From: erickson Date: Mon, 11 Aug 2008 13:08:26 +0000 (+0000) Subject: Merged revisions 10257-10259,10261-10263,10265-10266,10273,10278-10287,10290-10291... X-Git-Url: https://old-git.evergreen-ils.org/?a=commitdiff_plain;h=1e1e3ce1bf2c7b7b0faad38172f3827ae27116d5;p=Evergreen.git Merged revisions 10257-10259,10261-10263,10265-10266,10273,10278-10287,10290-10291,10294,10297-10300,10303-10308,10310,10315,10319-10321,10323-10325 via svnmerge from svn://svn.open-ils.org/ILS/trunk ........ r10257 | dbs | 2008-08-04 23:12:18 -0400 (Mon, 04 Aug 2008) | 2 lines Add an "install" target to place built i18n files into the correct location in the source tree ........ r10258 | dbs | 2008-08-04 23:15:03 -0400 (Mon, 04 Aug 2008) | 2 lines Updated translations from Tigran Zargaryan - for early testing of 1.4 release ........ r10259 | erickson | 2008-08-05 08:34:03 -0400 (Tue, 05 Aug 2008) | 1 line no need to bail out on no-copy. if precat, send null copy id to circ test ........ r10261 | erickson | 2008-08-05 08:42:34 -0400 (Tue, 05 Aug 2008) | 1 line catch server errors on checkout, clear the pending transaction flag, store the exception text in a hidden div as a debugging tool ........ r10262 | dbs | 2008-08-05 13:17:47 -0400 (Tue, 05 Aug 2008) | 2 lines Add test for JSON::XS (per Syd Weidman's suggestion) ........ r10263 | erickson | 2008-08-05 13:55:36 -0400 (Tue, 05 Aug 2008) | 1 line the existing if/else construct was forcing the proximity call regardless of whether the cached value existed. changed and tested ........ r10265 | erickson | 2008-08-05 14:01:59 -0400 (Tue, 05 Aug 2008) | 6 lines Added support for using in-database hold permit logic. This code abides by the legacy_script_support setting to determine which permit style should be used ........ r10266 | miker | 2008-08-05 21:47:35 -0400 (Tue, 05 Aug 2008) | 1 line allow NULL as a parameter in transforms and FROM clause functions ........ r10273 | miker | 2008-08-05 22:02:04 -0400 (Tue, 05 Aug 2008) | 1 line adding 1.2.2.3 to 1.2.3.0 upgrade script ........ r10278 | erickson | 2008-08-06 11:42:00 -0400 (Wed, 06 Aug 2008) | 1 line don't consider deleted volumes in callnumber browse ........ r10279 | dbs | 2008-08-06 15:17:49 -0400 (Wed, 06 Aug 2008) | 3 lines Add autoconf and automake for Debian (not part of build-essential? geez!) Add an explicit shell and vim formatting modeline ........ r10280 | dbs | 2008-08-06 15:19:09 -0400 (Wed, 06 Aug 2008) | 2 lines Sort our Debian package list as it's getting unruly ........ r10281 | erickson | 2008-08-06 17:33:40 -0400 (Wed, 06 Aug 2008) | 16 lines This is a remodeled version of Evergreen/src/support-scripts/eg_gen_overdue.pl. This version is built to work for both overdue and predue (courtesy) notices. What we have so far is circ searching and retrieval of associated data. TODO - Add some tuning for sub-day notice intervals - Plug in Template Toolkit for template generation - Set up and test emailing of templates - Handle global XML file generation (send all loaded data to a TT XML template as a last step?) - Add sample templates to repo - Add sample config to opensrf.xml.example ........ r10282 | erickson | 2008-08-06 18:05:28 -0400 (Wed, 06 Aug 2008) | 1 line don't re-fetch the user and circ lib on a batch. use editor for initial query (good for logging) ........ r10283 | miker | 2008-08-06 22:23:43 -0400 (Wed, 06 Aug 2008) | 1 line use decomposed characters where possible, for the sake of JSON::XS ........ r10284 | erickson | 2008-08-06 22:26:14 -0400 (Wed, 06 Aug 2008) | 4 lines Plugged in template processing ........ r10285 | erickson | 2008-08-06 22:30:47 -0400 (Wed, 06 Aug 2008) | 1 line adding some example templates. need to add locale blocks and do some more pre-run data munging ........ r10286 | erickson | 2008-08-06 22:34:51 -0400 (Wed, 06 Aug 2008) | 1 line added example config for over/pre due notices ........ r10287 | miker | 2008-08-06 22:35:48 -0400 (Wed, 06 Aug 2008) | 1 line used decomposed characters -- mostly for consistency with other recent changes ........ r10290 | erickson | 2008-08-06 22:57:07 -0400 (Wed, 06 Aug 2008) | 1 line changing default hold pull list limit to 100 ........ r10291 | erickson | 2008-08-07 08:52:47 -0400 (Thu, 07 Aug 2008) | 6 lines Removed global pre/post chomp template config. make judicious use of [%- -%] instead. Thanks for the tip, Mike ........ r10294 | erickson | 2008-08-07 09:28:50 -0400 (Thu, 07 Aug 2008) | 1 line pull org location from URL param "l" .. default to org tree root ........ r10297 | erickson | 2008-08-07 09:49:48 -0400 (Thu, 07 Aug 2008) | 1 line return to login page if auth fails, existing auth session has timed out, or existing auth session does not have required perms ........ r10298 | dbs | 2008-08-07 10:23:16 -0400 (Thu, 07 Aug 2008) | 2 lines Per Brandon Uhlmann's suggestion, speed up bibloading by automatically disabling/reenabling materialized record trigger ........ r10299 | dbs | 2008-08-07 10:30:43 -0400 (Thu, 07 Aug 2008) | 2 lines Per Brandon Uhlmann's suggestion, speed up loading of bibs by automatically disabling/reenabling materialized record trigger ........ r10300 | dbs | 2008-08-07 11:08:26 -0400 (Thu, 07 Aug 2008) | 2 lines Add a --required_field option to enable skipping records that don't contain a required field ........ r10303 | erickson | 2008-08-07 11:23:13 -0400 (Thu, 07 Aug 2008) | 1 line preserve the entire url with query string in redirects ........ r10304 | erickson | 2008-08-07 13:13:53 -0400 (Thu, 07 Aug 2008) | 1 line added logic for extracting title, author (and potentiall other attributes) from the bib and to break apart the due date so the template can display however it likes ........ r10305 | erickson | 2008-08-07 13:16:30 -0400 (Thu, 07 Aug 2008) | 1 line displaying title/author and extracted due date. falling back to billing addr if library mailing addr is unset ........ r10306 | erickson | 2008-08-07 15:33:15 -0400 (Thu, 07 Aug 2008) | 1 line added logic for generating a template from the combined overdue data (e.g. xml file). added sample combined xml file. ........ r10307 | erickson | 2008-08-07 16:36:50 -0400 (Thu, 07 Aug 2008) | 1 line fix up some spacing ........ r10308 | erickson | 2008-08-07 16:37:48 -0400 (Thu, 07 Aug 2008) | 1 line plugged in SMTP handling. also, with this version, if the SMTP transaction fails, the notice is added to the global notice set (if applicable) ........ r10310 | phasefx | 2008-08-08 10:39:38 -0400 (Fri, 08 Aug 2008) | 1 line yaz change; will complain of bad xml format otherwise ........ r10315 | phasefx | 2008-08-08 11:37:24 -0400 (Fri, 08 Aug 2008) | 1 line fix Cancel button in volume editor ........ r10319 | miker | 2008-08-09 23:49:21 -0400 (Sat, 09 Aug 2008) | 1 line make staged search result calcuation more configurable ........ r10320 | miker | 2008-08-10 01:34:55 -0400 (Sun, 10 Aug 2008) | 1 line more staged search configurability and acuracy ........ r10321 | miker | 2008-08-10 14:18:30 -0400 (Sun, 10 Aug 2008) | 1 line validate all controlled subfields in a tag ........ r10323 | miker | 2008-08-10 14:59:38 -0400 (Sun, 10 Aug 2008) | 1 line send actual value; no wrapping array, just a string; note about future improvement ........ r10324 | miker | 2008-08-10 20:11:18 -0400 (Sun, 10 Aug 2008) | 1 line allow blocked users to be stopped from placing holds (clear as mud, eh?) ........ r10325 | miker | 2008-08-10 21:05:06 -0400 (Sun, 10 Aug 2008) | 1 line add a note about how to reload this portion of the schema, and from whence to retrieve the default seed data ........ git-svn-id: svn://svn.open-ils.org/ILS/branches/acq-experiment@10326 dcc99617-32d9-48b4-a31d-7c20da2025e4 --- diff --git a/Open-ILS/examples/opensrf.xml.example b/Open-ILS/examples/opensrf.xml.example index f46eb8eb54..59ead0ceee 100644 --- a/Open-ILS/examples/opensrf.xml.example +++ b/Open-ILS/examples/opensrf.xml.example @@ -32,7 +32,7 @@ vim:et:ts=4:sw=4: /openils/var/data/ils_events.xml - + localhost @@ -45,6 +45,52 @@ vim:et:ts=4:sw=4: + + + localhost + evergreen@localhost + + + + + + overdue@localhost + + + /openils/var/data/overdue + /openils/var/data/templates/overdue_combined_xml.example + + + + 7 days + + noemail + + true + + /openils/var/data/templates/overdue_7day.example + + + + + + + + + 5 days + 13 days + + + 1 day + false + true + /openils/var/data/templates/predue_1day.example + + + + true - 500 - 20 + + + inclusion + + + 1000 + + + 10 + diff --git a/Open-ILS/examples/templates/overdue_7day.example b/Open-ILS/examples/templates/overdue_7day.example new file mode 100644 index 0000000000..697d2bf554 --- /dev/null +++ b/Open-ILS/examples/templates/overdue_7day.example @@ -0,0 +1,32 @@ +[%- USE date -%] +[%- SET user = circ_list.0.usr -%] +[%- SET lib = circ_list.0.circ_lib -%] +To: [% user.email %] +From: [% smtp_sender %] +Reply-To: [% smtp_reply %] +Subject: Overdue Notification + + +Dear [% user.first_given_name %] [% user.family_name %] + +Our records indicate these items are 7 days overdue: + +[% FOREACH circ = circ_list %] + [% get_bib_attr(circ, 'title') %], by [% get_bib_attr(circ, 'author') %] + Call Number: [% circ.target_copy.call_number.label %] + [%- SET due_date = parse_due_date(circ) %] + Due Date: [% date.format(due_date, '%Y-%m-%d') %] + Barcode: [% circ.target_copy.barcode %] +[% END %] + + +Please return the above items to avoid additional fines. Please do not +respond to this email. + +Contact your library for more information: + +[% lib.name %] +[% lib.mailing_address.street1 %] [% lib.mailing_address.street2 %] +[% lib.mailing_address.city %], [% lib.mailing_address.state %] +[% lib.mailing_address.post_code %] +[% lib.phone %] diff --git a/Open-ILS/examples/templates/overdue_combined_xml.example b/Open-ILS/examples/templates/overdue_combined_xml.example new file mode 100644 index 0000000000..024821fffd --- /dev/null +++ b/Open-ILS/examples/templates/overdue_combined_xml.example @@ -0,0 +1,52 @@ +[% USE date -%] + + + [%- FOREACH circ_set = overdues %] + + [%- SET user = circ_set.circ_list.0.usr -%] + [%- SET lib = circ_set.circ_list.0.circ_lib -%] + [%- SET user_addr = user.mailing_address -%] + [%- IF !user_addr -%] + [%- SET user_addr = user.billing_address -%] + [%- END %] + [%- SET lib_addr = lib.mailing_address -%] + [%- IF !lib_addr -%] + [%- SET lib_addr = lib.billing_address -%] + [%- END %] + + [% user.card.barcode %] + [% user.first_given_name %] + [% user.family_name %] + [% escape_xml(user_addr.street1) %] + [% escape_xml(user_addr.street2) %] + [% escape_xml(user_addr.city) %] + [% user_addr.state %] + [% user_addr.post_code %] + [% user.email %] + [% user.id %] + + + [% escape_xml(lib.name) %] + [% lib.phone %] + [% escape_xml(lib_addr.street1) %] + [% escape_xml(lib_addr.street2) %] + [% escape_xml(lib_addr.city) %] + [% lib_addr.state %] + [% lib_addr.post_code %] + [% lib.email %] + [% lib.id %] + + [%- FOREACH circ = circ_set.circ_list %] + [%- SET due_date = parse_due_date(circ) %] + + [% escape_xml(get_bib_attr(circ, 'title')) %] + [% escape_xml(get_bib_attr(circ, 'author')) %] + [% date.format(due_date, '%Y-%m-%d') %] + [% escape_xml(circ.target_copy.call_number.label) %] + [% escape_xml(circ.target_copy.barcode) %] + [% circ.id %] + + [%- END %] + + [%- END %] + diff --git a/Open-ILS/examples/templates/predue_1day.example b/Open-ILS/examples/templates/predue_1day.example new file mode 100644 index 0000000000..c0823926c6 --- /dev/null +++ b/Open-ILS/examples/templates/predue_1day.example @@ -0,0 +1,33 @@ +[%- USE date -%] +[%- SET user = circ_list.0.usr -%] +[%- SET lib = circ_list.0.circ_lib -%] +To: [% user.email %] +From: [% smtp_sender %] +Reply-To: [% smtp_reply %] +Subject: Courtesy Notice + +Dear [% user.first_given_name %] [% user.family_name %] + +Our records indicate these items are due tomorrow: + +[% FOREACH circ = circ_list %] + [% get_bib_attr(circ, 'title') %], by [% get_bib_attr(circ, 'author') %] + Call Number: [% circ.target_copy.call_number.label %] + [%- SET due_date = parse_due_date(circ) %] + Due Date: [% date.format(due_date, '%Y-%m-%d') %] + Barcode: [% circ.target_copy.barcode %] +[% END %] + + +Please return the above items by tomorrow to avoid additional fines. Please do not +respond to this email. + +Contact your library for more information: + +[% lib.name %] +[%- SET addr = lib.mailing_address -%] +[%- IF !addr -%] [%- SET addr = lib.billing_address -%] [%- END %] +[% lib.mailing_address.street1 %] [% lib.mailing_address.street2 %] +[% lib.mailing_address.city %], [% lib.mailing_address.state %] +[% lib.mailing_address.post_code %] +[% lib.phone %] diff --git a/Open-ILS/src/c-apps/oils_cstore.c b/Open-ILS/src/c-apps/oils_cstore.c index c580fa3025..6aaab599d8 100644 --- a/Open-ILS/src/c-apps/oils_cstore.c +++ b/Open-ILS/src/c-apps/oils_cstore.c @@ -1128,15 +1128,18 @@ static char* searchValueTransform( const jsonObject* array ) { else buffer_add(sql_buf, ", "); - if ( dbi_conn_quote_string(dbhandle, &val) ) { + if (func_item->type == JSON_NULL) { + buffer_add( sql_buf, "NULL" ); + } else if ( dbi_conn_quote_string(dbhandle, &val) ) { buffer_fadd( sql_buf, "%s", val ); - free(val); } else { osrfLogError(OSRF_LOG_MARK, "%s: Error quoting key string [%s]", MODULENAME, val); free(val); buffer_free(sql_buf); return NULL; } + + free(val); } buffer_add( diff --git a/Open-ILS/src/extras/Makefile.install b/Open-ILS/src/extras/Makefile.install index 980f7e0747..7b4a2219b7 100644 --- a/Open-ILS/src/extras/Makefile.install +++ b/Open-ILS/src/extras/Makefile.install @@ -28,13 +28,15 @@ # # --------------------------------------------------------------------- +# Make any assumptions about the shell being used explicit +SHELL=/bin/bash + # XXX # Gentoo needs explicit versions on many of these packages # to simulate a "blessed" set of packages # Also, I (think) Gentoo has a javascript::spidermonkey package that does # not require fetching the sources externally ... needs testing/updating in here - LIBJS=js-1.7.0 LIBJS_PERL=JavaScript-SpiderMonkey-0.19 LIBJS_URL=ftp://ftp.mozilla.org/pub/mozilla.org/js/$(LIBJS).tar.gz @@ -79,63 +81,65 @@ APT_TOOL=aptitude # Debian dependencies DEBS = \ + apache2-mpm-prefork\ + apache2-prefork-dev\ + aspell\ + aspell-en\ + autoconf\ + automake\ build-essential\ - syslog-ng\ - psmisc\ - ntpdate\ + ejabberd\ less\ - memcached\ - libxml2-dev\ - libmodule-build-perl\ - libexpat1-dev\ - libmemcache-dev\ - libperl-dev\ - libcache-memcached-perl\ + libapache2-mod-perl2\ + libbusiness-creditcard-perl\ + libbusiness-onlinepayment-authorizenet-perl\ + libbusiness-onlinepayment-perl\ + libcache-memcached-perl\ + libclass-dbi-abstractsearch-perl\ + libclass-dbi-pg-perl\ + libclass-dbi-sqlite-perl\ + libdatetime-format-builder-perl\ + libdatetime-format-mail-perl\ + libdatetime-perl\ + libdatetime-timezone-perl\ + libemail-send-perl\ + liberror-perl\ + libexpat1-dev\ + libfile-find-rule-perl\ + libfreezethaw-perl\ + libgd-graph3d-perl\ + liblog-log4perl-perl\ + libmarc-record-perl\ + libmemcache-dev\ + libmodule-build-perl\ + libnet-jabber-perl\ + libole-storage-lite-perl\ + libperl-dev\ + libpq-dev\ + libreadline5-dev\ + librpc-xml-perl\ + libspreadsheet-writeexcel-perl\ + libtemplate-perl\ + libtest-pod-perl\ + libtext-aspell-perl\ libtext-csv-perl\ - libxml-libxml-perl\ - libxslt1-dev\ - libxml-libxslt-perl\ - libwww-perl\ - liberror-perl\ - libclass-dbi-pg-perl\ - libclass-dbi-abstractsearch-perl\ - libtemplate-perl\ - libtext-aspell-perl\ - libdatetime-format-mail-perl\ - libdatetime-timezone-perl\ - libdatetime-perl\ - libunix-syslog-perl\ - libgd-graph3d-perl\ - libuniversal-require-perl\ - libclass-dbi-sqlite-perl\ - liblog-log4perl-perl\ - libnet-jabber-perl\ - libtest-pod-perl\ - libfile-find-rule-perl\ - libdatetime-format-builder-perl\ - libmarc-record-perl\ - librpc-xml-perl\ - aspell\ - aspell-en\ - libxml-simple-perl\ - libpq-dev\ - libemail-send-perl\ - ejabberd\ - libtool\ - apache2-mpm-prefork\ - apache2-prefork-dev\ - libapache2-mod-perl2\ - libreadline5-dev\ libtext-csv-perl\ - libspreadsheet-writeexcel-perl\ - libole-storage-lite-perl\ libtie-ixhash-perl\ + libtool\ + libuniversal-require-perl\ + libunix-syslog-perl\ + libwww-perl\ + libxml2-dev\ + libxml-libxml-perl\ + libxml-libxslt-perl\ + libxml-simple-perl\ + libxslt1-dev\ + memcached\ + ntpdate\ + psmisc\ python-dev\ python-setuptools\ - libfreezethaw-perl\ - libbusiness-creditcard-perl\ - libbusiness-onlinepayment-perl\ - libbusiness-onlinepayment-authorizenet-perl + syslog-ng CENTOS = \ apr-util-devel \ @@ -452,3 +456,5 @@ create_ld_local: echo '/usr/local/lib' > /etc/ld.so.conf.d/local.conf; \ ldconfig; \ fi; + +# vim:noet:sw=4:ts=4: diff --git a/Open-ILS/src/extras/import/marc2are.pl b/Open-ILS/src/extras/import/marc2are.pl index 6ae28e261f..401a5dbf32 100755 --- a/Open-ILS/src/extras/import/marc2are.pl +++ b/Open-ILS/src/extras/import/marc2are.pl @@ -69,7 +69,7 @@ while ( try { $rec = $batch->next } otherwise { $rec = -1 } ) { $xml =~ s/^<\?xml.+\?\s*>//go; $xml =~ s/>\s+ignore_errors(1); -my ($id_field, $id_subfield, $recid, $user, $config, $idlfile, $marctype, $keyfile, $dontuse_file, $enc, $force_enc, @files, @trash_fields, $quiet) = +my ($id_field, $id_subfield, $recid, $user, $config, $idlfile, $marctype, $keyfile, $dontuse_file, $enc, $force_enc, @files, @trash_fields, @req_fields, $quiet) = ('', 'a', 0, 1, '/openils/conf/opensrf_core.xml', '/openils/conf/fm_IDL.xml', 'USMARC'); my ($db_driver,$db_host,$db_name,$db_user,$db_pw) = @@ -39,6 +39,7 @@ GetOptions( 'keyfile=s' => \$keyfile, 'config=s' => \$config, 'file=s' => \@files, + 'required_field=s' => \@req_fields, 'trash=s' => \@trash_fields, 'xml_idl=s' => \$idlfile, 'dontuse=s' => \$dontuse_file, @@ -130,8 +131,13 @@ my %used_ids; my $starttime = time; my $rec; my $count = 0; -while ( try { $rec = $batch->next } otherwise { $rec = -1 } ) { +PROCESS: while ( try { $rec = $batch->next } otherwise { $rec = -1 } ) { next if ($rec == -1); + + # Skip records that don't contain a required field (like '245', for example) + foreach my $req_field(@req_fields) { + next PROCESS if !$rec->field("$req_field"); + } my $id; $recid++; @@ -187,7 +193,7 @@ while ( try { $rec = $batch->next } otherwise { $rec = -1 } ) { $xml =~ s/^<\?xml.+\?\s*>//go; $xml =~ s/>\s+print("SET CLIENT_ENCODING TO 'UNICODE';\n\n"); $main_out->print("BEGIN;\n\n"); my %out_files; -my %out_headers; for my $h (@order) { $out_files{$h} = FileHandle->new(">$output.$h.sql"); binmode($out_files{$h},'utf8'); @@ -82,6 +82,10 @@ while ( my $rec = <> ) { my $fields = join(',', @{ $fieldcache{$hint}{fields} }); $main_out->print( "DELETE FROM $fieldcache{$hint}{table};\n" ) if (grep {$_ eq $hint } @wipe); + # Speed up loading of bib records + if ($hint eq 'mfr') { + $main_out->print("\nSELECT reporter.disable_materialized_simple_record_trigger();\n"); + } $main_out->print( "COPY $fieldcache{$hint}{table} ($fields) FROM '$pwd/$output.$hint.sql';\n" ); } @@ -114,5 +118,8 @@ while ( my $rec = <> ) { $count++; } +if (grep /^mfr$/, %out_files) { + $main_out->print("SELECT reporter.enable_materialized_simple_record_trigger();\n"); +} $main_out->print("-- COMMIT;\n\n"); $main_out->close; diff --git a/Open-ILS/src/extras/import/pg_loader.pl b/Open-ILS/src/extras/import/pg_loader.pl index 679f44edd6..ef07a05ce0 100755 --- a/Open-ILS/src/extras/import/pg_loader.pl +++ b/Open-ILS/src/extras/import/pg_loader.pl @@ -89,6 +89,10 @@ for my $h (@order) { my $fields = join(',', @{ $fieldcache{$h}{fields} }); $output->print( "DELETE FROM $fieldcache{$h}{table};\n" ) if (grep {$_ eq $h } @wipe); + # Speed up loading of bib records + if ($h eq 'mfr') { + $output->print("\nSELECT reporter.disable_materialized_simple_record_trigger();\n"); + } $output->print( "COPY $fieldcache{$h}{table} ($fields) FROM STDIN;\n" ); for my $line (@{ $lineset{$h} }) { @@ -115,6 +119,10 @@ for my $h (@order) { $output->print('\.'."\n\n"); + if ($h eq 'mfr') { + $output->print("SELECT reporter.enable_materialized_simple_record_trigger();\n"); + } + $output->print("SELECT setval('$fieldcache{$h}{sequence}'::TEXT, (SELECT MAX($fieldcache{$h}{pkey}) FROM $fieldcache{$h}{table}), TRUE);\n\n") if (!grep { $_ eq $h} @auto); } diff --git a/Open-ILS/src/extras/org_tree_html_options.pl b/Open-ILS/src/extras/org_tree_html_options.pl index 4aaa8b9531..475ead0346 100644 --- a/Open-ILS/src/extras/org_tree_html_options.pl +++ b/Open-ILS/src/extras/org_tree_html_options.pl @@ -42,7 +42,7 @@ sub entityize { $stuff =~ s/\/>/og; $stuff =~ s/\&/&/og; - $stuff = NFC($stuff); + $stuff = NFD($stuff); $stuff =~ s/([\x{0080}-\x{fffd}])/sprintf('&#x%X;',ord($1))/sgoe; return $stuff; } diff --git a/Open-ILS/src/perlmods/OpenILS/Application/Circ/Circulate.pm b/Open-ILS/src/perlmods/OpenILS/Application/Circ/Circulate.pm index ca91fa1572..8cefede17b 100644 --- a/Open-ILS/src/perlmods/OpenILS/Application/Circ/Circulate.pm +++ b/Open-ILS/src/perlmods/OpenILS/Application/Circ/Circulate.pm @@ -503,8 +503,7 @@ sub mk_env { } elsif( $self->copy_barcode ) { $copy = $e->search_asset_copy( - [{barcode => $self->copy_barcode, deleted => 'f'}, $flesh ])->[0] - or return $e->event; + [{barcode => $self->copy_barcode, deleted => 'f'}, $flesh ])->[0]; } if($copy) { @@ -822,7 +821,7 @@ sub run_indb_circ_test { { from => [ $dbfunc, $self->editor->requestor->ws_ou, - $self->copy->id, + ($self->is_precat) ? undef : $self->copy->id, $self->patron->id, ] } diff --git a/Open-ILS/src/perlmods/OpenILS/Application/Circ/Holds.pm b/Open-ILS/src/perlmods/OpenILS/Application/Circ/Holds.pm index f02962c495..7456a86368 100644 --- a/Open-ILS/src/perlmods/OpenILS/Application/Circ/Holds.pm +++ b/Open-ILS/src/perlmods/OpenILS/Application/Circ/Holds.pm @@ -1165,10 +1165,10 @@ sub _check_metarecord_hold_is_possible { my $home_org = $patron->home_ou; my $req_org = $request_lib->id; - my $home_prox = - ($prox_cache{$home_org}) ? - $prox_cache{$home_org} : - $prox_cache{$home_org} = $e->search_actor_org_unit_proximity({from_org => $home_org}); + $prox_cache{$home_org} = + $e->search_actor_org_unit_proximity({from_org => $home_org}) + unless $prox_cache{$home_org}; + my $home_prox = $prox_cache{$home_org}; my %buckets; my %hash = map { ($_->to_org => $_->prox) } @$home_prox; @@ -1183,10 +1183,11 @@ sub _check_metarecord_hold_is_possible { # directly before the farthest away copies. That way, they are not # given priority, but they are checked before the farthest copies. # ----------------------------------------------------------------------- - my $req_prox = - ($prox_cache{$req_org}) ? - $prox_cache{$req_org} : - $prox_cache{$req_org} = $e->search_actor_org_unit_proximity({from_org => $req_org}); + + $prox_cache{$req_org} = + $e->search_actor_org_unit_proximity({from_org => $req_org}) + unless $prox_cache{$req_org}; + my $req_prox = $prox_cache{$req_org}; my %buckets2; my %hash2 = map { ($_->to_org => $_->prox) } @$req_prox; @@ -1297,10 +1298,12 @@ sub _check_title_hold_is_possible { my $home_org = $patron->home_ou; my $req_org = $request_lib->id; - my $home_prox = - ($prox_cache{$home_org}) ? - $prox_cache{$home_org} : - $prox_cache{$home_org} = $e->search_actor_org_unit_proximity({from_org => $home_org}); + $logger->info("prox cache $home_org " . $prox_cache{$home_org}); + + $prox_cache{$home_org} = + $e->search_actor_org_unit_proximity({from_org => $home_org}) + unless $prox_cache{$home_org}; + my $home_prox = $prox_cache{$home_org}; my %buckets; my %hash = map { ($_->to_org => $_->prox) } @$home_prox; @@ -1315,10 +1318,11 @@ sub _check_title_hold_is_possible { # directly before the farthest away copies. That way, they are not # given priority, but they are checked before the farthest copies. # ----------------------------------------------------------------------- - my $req_prox = - ($prox_cache{$req_org}) ? - $prox_cache{$req_org} : - $prox_cache{$req_org} = $e->search_actor_org_unit_proximity({from_org => $req_org}); + $prox_cache{$req_org} = + $e->search_actor_org_unit_proximity({from_org => $req_org}) + unless $prox_cache{$req_org}; + my $req_prox = $prox_cache{$req_org}; + my %buckets2; my %hash2 = map { ($_->to_org => $_->prox) } @$req_prox; diff --git a/Open-ILS/src/perlmods/OpenILS/Application/Search/Biblio.pm b/Open-ILS/src/perlmods/OpenILS/Application/Search/Biblio.pm index 3056109f20..09ebf4fe1e 100644 --- a/Open-ILS/src/perlmods/OpenILS/Application/Search/Biblio.pm +++ b/Open-ILS/src/perlmods/OpenILS/Application/Search/Biblio.pm @@ -780,6 +780,14 @@ sub staged_search { # restrict total tested to superpage size * number of superpages $search_hash->{core_limit} = $superpage_size * $max_superpages; + # Set the configured estimation strategy, defaults to 'inclusion'. + my $estimation_strategy = OpenSRF::Utils::SettingsClient + ->new + ->config_value( + apps => 'open-ils.search', app_settings => 'estimation_strategy' + ) || 'inclusion'; + $search_hash->{estimation_strategy} = $estimation_strategy; + # pull any existing results from the cache my $key = search_cache_key($method, $search_hash); my $cache_data = $cache->get_cache($key) || {}; @@ -790,6 +798,7 @@ sub staged_search { my $page; # current superpage my $est_hit_count = 0; my $current_page_summary = {}; + my $global_summary = {checked => 0, visible => 0, excluded => 0, deleted => 0, total => 0}; for($page = 0; $page < $max_superpages; $page++) { @@ -838,6 +847,14 @@ sub staged_search { $logger->debug("staged search: located $current_count, with estimated hits=". $summary->{estimated_hit_count}." : visible=".$summary->{visible}.", checked=".$summary->{checked}); + if (defined($summary->{estimated_hit_count})) { + $global_summary->{checked} += $summary->{checked}; + $global_summary->{visible} += $summary->{visible}; + $global_summary->{excluded} += $summary->{excluded}; + $global_summary->{deleted} += $summary->{deleted}; + $global_summary->{total} = $summary->{total}; + } + # we've found all the possible hits last if $current_count == $summary->{visible} and not defined $summary->{estimated_hit_count}; @@ -851,6 +868,23 @@ sub staged_search { my @results = grep {defined $_} @$all_results[$user_offset..($user_offset + $user_limit - 1)]; + # refine the estimate if we have more than one superpage + if ($page > 0) { + if ($global_summary->{checked} >= $global_summary->{total}) { + $est_hit_count = $global_summary->{visible}; + } else { + my $updated_hit_count = $U->storagereq( + 'open-ils.storage.fts_paging_estimate', + $global_summary->{checked}, + $global_summary->{visible}, + $global_summary->{excluded}, + $global_summary->{deleted}, + $global_summary->{total} + ); + $est_hit_count = $updated_hit_count->{$estimation_strategy}; + } + } + return { count => $est_hit_count, core_limit => $search_hash->{core_limit}, diff --git a/Open-ILS/src/perlmods/OpenILS/Application/Search/Z3950.pm b/Open-ILS/src/perlmods/OpenILS/Application/Search/Z3950.pm index ef38212d41..3fd6428c8e 100755 --- a/Open-ILS/src/perlmods/OpenILS/Application/Search/Z3950.pm +++ b/Open-ILS/src/perlmods/OpenILS/Application/Search/Z3950.pm @@ -17,7 +17,7 @@ use OpenILS::Application::AppUtils; use OpenSRF::Utils::Logger qw/$logger/; use OpenILS::Utils::CStoreEditor q/:funcs/; -my $output = "USMARC"; +my $output = "usmarc"; my $U = 'OpenILS::Application::AppUtils'; my $sclient; diff --git a/Open-ILS/src/perlmods/OpenILS/Application/Storage/Publisher/metabib.pm b/Open-ILS/src/perlmods/OpenILS/Application/Storage/Publisher/metabib.pm index 9be3c6a83f..ab681bcd16 100644 --- a/Open-ILS/src/perlmods/OpenILS/Application/Storage/Publisher/metabib.pm +++ b/Open-ILS/src/perlmods/OpenILS/Application/Storage/Publisher/metabib.pm @@ -2369,6 +2369,9 @@ sub staged_fts { } + # inclusion, exclusion, delete_adjusted_inclusion, delete_adjusted_exclusion + my $estimation_strategy = $args{estimation_strategy} || 'inclusion'; + my $ou = $args{org_unit}; my $limit = $args{limit} || 10; my $offset = $args{offset} || 0; @@ -2566,11 +2569,10 @@ sub staged_fts { my $estimate = $visible; if ( $total > $checked && $checked ) { - my $deleted_ratio = $deleted / $checked; - my $exclution_ratio = $excluded / $checked; - my $delete_adjusted_total = $total - ( $total * $deleted_ratio ); - $estimate = $$summary_row{estimated_hit_count} = int($delete_adjusted_total - ( $delete_adjusted_total * $exclution_ratio )); + $$summary_row{hit_estimate} = FTS_paging_estimate($self, $client, $checked, $visible, $excluded, $deleted, $total); + $estimate = $$summary_row{estimated_hit_count} = $$summary_row{hit_estimate}{$estimation_strategy}; + } delete $$summary_row{id}; @@ -2622,6 +2624,78 @@ __PACKAGE__->register_method( cachable => 1, ); +sub FTS_paging_estimate { + my $self = shift; + my $client = shift; + + my $checked = shift; + my $visible = shift; + my $excluded = shift; + my $deleted = shift; + my $total = shift; + + my $deleted_ratio = $deleted / $checked; + my $delete_adjusted_total = $total - ( $total * $deleted_ratio ); + + my $exclusion_ratio = $excluded / $checked; + my $delete_adjusted_exclusion_ratio = $excluded / ($checked - $deleted); + + my $inclusion_ratio = $visible / $checked; + my $delete_adjusted_inclusion_ratio = $visible / ($checked - $deleted); + + return { + exclusion => int($delete_adjusted_total - ( $delete_adjusted_total * $exclusion_ratio )), + inclusion => int($delete_adjusted_total * $inclusion_ratio), + delete_adjusted_exclusion => int($delete_adjusted_total - ( $delete_adjusted_total * $delete_adjusted_exclusion_ratio )), + delete_adjusted_inclusion => int($delete_adjusted_total * $delete_adjusted_inclusion_ratio) + }; +} +__PACKAGE__->register_method( + api_name => "open-ils.storage.fts_paging_estimate", + method => 'FTS_paging_estimate', + argc => 5, + strict => 1, + api_level => 1, + signature => { + 'return'=> q# + Hash of estimation values based on four variant estimation strategies: + exclusion -- Estimate based on the ratio of excluded records on the current superpage; + inclusion -- Estimate based on the ratio of visible records on the current superpage; + delete_adjusted_exclusion -- Same as exclusion strategy, but the ratio is adjusted by deleted count; + delete_adjusted_inclusion -- Same as inclusion strategy, but the ratio is adjusted by deleted count; + #, + desc => q# + Helper method used to determin the approximate number of + hits for a search that spans multiple superpages. For + sparse superpages, the inclusion estimate will likely be the + best estimate. The exclusion strategy is the original, but + inclusion is the default. + #, + params => [ + { name => 'checked', + desc => 'Number of records check -- nominally the size of a superpage, or a remaining amount from the last superpage.', + type => 'number' + }, + { name => 'visible', + desc => 'Number of records visible to the search location on the current superpage.', + type => 'number' + }, + { name => 'excluded', + desc => 'Number of records excluded from the search location on the current superpage.', + type => 'number' + }, + { name => 'deleted', + desc => 'Number of deleted records on the current superpage.', + type => 'number' + }, + { name => 'total', + desc => 'Total number of records up to check_limit (superpage_size * max_superpages).', + type => 'number' + } + ] + } +); + sub xref_count { my $self = shift; diff --git a/Open-ILS/src/perlmods/OpenILS/Application/SuperCat.pm b/Open-ILS/src/perlmods/OpenILS/Application/SuperCat.pm index ede7e269e3..e33dcbb6f4 100644 --- a/Open-ILS/src/perlmods/OpenILS/Application/SuperCat.pm +++ b/Open-ILS/src/perlmods/OpenILS/Application/SuperCat.pm @@ -261,6 +261,7 @@ sub cn_browse { "open-ils.cstore.direct.asset.call_number.search.atomic", { label => { "<" => { transform => "upper", value => ["upper", $label] } }, owning_lib => \@ou_ids, + deleted => 'f', }, { flesh => 1, flesh_fields => { acn => [qw/record owning_lib/] }, @@ -277,6 +278,7 @@ sub cn_browse { "open-ils.cstore.direct.asset.call_number.search.atomic", { label => { ">=" => { transform => "upper", value => ["upper", $label] } }, owning_lib => \@ou_ids, + deleted => 'f', }, { flesh => 1, flesh_fields => { acn => [qw/record owning_lib/] }, diff --git a/Open-ILS/src/perlmods/OpenILS/Utils/PermitHold.pm b/Open-ILS/src/perlmods/OpenILS/Utils/PermitHold.pm index fcef298dc6..d385fb9490 100644 --- a/Open-ILS/src/perlmods/OpenILS/Utils/PermitHold.pm +++ b/Open-ILS/src/perlmods/OpenILS/Utils/PermitHold.pm @@ -9,10 +9,12 @@ use DateTime::Format::ISO8601; use OpenILS::Application::Circ::ScriptBuilder; use OpenSRF::Utils::Logger qw(:logger); use OpenILS::Event; +use OpenILS::Utils::CStoreEditor qw/:funcs/; my $U = "OpenILS::Application::AppUtils"; my $script; # - the permit script my $script_libs; # - extra script libs +my $legacy_script_support; # mental note: open-ils.storage.biblio.record_entry.ranged_tree @@ -23,6 +25,16 @@ sub permit_copy_hold { my $params = shift; my @allevents; + unless(defined $legacy_script_support) { + my $conf = OpenSRF::Utils::SettingsClient->new; + $legacy_script_support = $conf->config_value( + apps => 'open-ils.circ' => app_settings => 'legacy_script_support'); + $legacy_script_support = ($legacy_script_support and + $legacy_script_support =~ /true/i) ? 1 : 0; + } + + return indb_hold_permit($params) unless $legacy_script_support; + my $ctx = { patron_id => $$params{patron_id}, patron => $$params{patron}, @@ -181,4 +193,50 @@ sub check_age_protect { return undef; } +my $LEGACY_HOLD_EVENT_MAP = { + 'config.hold_matrix_test.holdable' => 'ITEM_NOT_HOLDABLE', + 'transit_range' => 'ITEM_NOT_HOLDABLE', + 'no_matchpoint' => 'NO_POLICY_MATCHPOINT', + 'config.hold_matrix_test.max_holds' => 'MAX_HOLDS', + 'config.rule_age_hold_protect.prox' => 'ITEM_AGE_PROTECTED' +}; + +sub indb_hold_permit { + my $params = shift; + + my $patron_id = + ref($$params{patron}) ? $$params{patron}->id : $$params{patron_id}; + my $request_lib = + ref($$params{request_lib}) ? $$params{request_lib}->id : $$params{request_lib}; + + my $HOLD_TEST = { + from => [ + 'action.hold_request_permit_test', + $$params{pickup_lib}, + $request_lib, + $$params{copy}->id, + $patron_id, + $$params{requestor}->id + ] + }; + + my $e = new_editor(xact=>1); + my $results = $e->json_query($HOLD_TEST); + $e->rollback; + + unless($$params{show_event_list}) { + return 1 if $U->is_true($results->[0]->{success}); + return 0; + } + + return [OpenILS::Event->new('NO_POLICY_MATCHPOINT')] unless @$results; + return [] if $U->is_true($results->[0]->{success}); + + my @events; + push(@events, OpenILS::Event->new( + $LEGACY_HOLD_EVENT_MAP->{$_->{fail_part}})) for @$results; + return \@events; +} + + 23; diff --git a/Open-ILS/src/perlmods/OpenILS/WWW/Proxy.pm b/Open-ILS/src/perlmods/OpenILS/WWW/Proxy.pm index 84c8f4d1eb..96f3db149b 100644 --- a/Open-ILS/src/perlmods/OpenILS/WWW/Proxy.pm +++ b/Open-ILS/src/perlmods/OpenILS/WWW/Proxy.pm @@ -77,8 +77,9 @@ sub handler { # push everyone to the secure site if (!$ssl_off && $url =~ /^http:/o) { - $url =~ s/^http:/https:/o; - print "Location: $url\n\n"; + my $base = $cgi->url(-base=>1); + $base =~ s/^http:/https:/o; + print "Location: $base".$apache->unparsed_uri."\n\n"; return Apache2::Const::OK; } @@ -104,7 +105,7 @@ sub handler { $auth_ses = oils_login($u, $p, $ltype); if ($auth_ses) { print $cgi->redirect( - -uri=>$url, + -uri=> $apache->unparsed_uri, -cookie=>$cgi->cookie( -name=>'ses', -value=>$auth_ses, @@ -112,11 +113,13 @@ sub handler { ) ); return Apache2::Const::REDIRECT; - } + } else { + return back_to_login($apache, $cgi); + } } my $user = verify_login($auth_ses); - return Apache2::Const::FORBIDDEN unless ($user); + return back_to_login($apache, $cgi) unless $user; $ws_ou ||= $user->home_ou; @@ -127,12 +130,26 @@ sub handler { ->request('open-ils.actor.user.perm.check', $auth_ses, $user->id, $ws_ou, $perms) ->gather(1); - return Apache2::Const::FORBIDDEN if (@$failures > 0); + return back_to_login($apache, $cgi) if (@$failures > 0); # they're good, let 'em through return Apache2::Const::DECLINED; } +sub back_to_login { + my $apache = shift; + my $cgi = shift; + print $cgi->redirect( + -uri=>$apache->unparsed_uri, + -cookie=>$cgi->cookie( + -name=>'ses', + -value=>'', + -path=>'/',-expires=>'-1h' + ) + ); + return Apache2::Const::REDIRECT; +} + # returns the user object if the session is valid, 0 otherwise sub verify_login { my $auth_token = shift; diff --git a/Open-ILS/src/sql/Pg/1.2.2.3-1.2.3.0-upgrade.sql b/Open-ILS/src/sql/Pg/1.2.2.3-1.2.3.0-upgrade.sql new file mode 100644 index 0000000000..1d06a44352 --- /dev/null +++ b/Open-ILS/src/sql/Pg/1.2.2.3-1.2.3.0-upgrade.sql @@ -0,0 +1,522 @@ +/* + * Copyright (C) 2007-2008 Equinox Software, Inc. + * Mike Rylander + * + * This program is free software; you can redistribute it and/or + * modify it under the terms of the GNU General Public License + * as published by the Free Software Foundation; either version 2 + * of the License, or (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + */ + + + +BEGIN; + +ALTER TABLE config.rule_max_fine ADD COLUMN is_percent BOOL NOT NULL DEFAULT FALSE; + +CREATE OR REPLACE FUNCTION biblio.next_autogen_tcn_value () RETURNS TEXT AS $$ + BEGIN RETURN 'AUTOGENERATED-' || nextval('biblio.autogen_tcn_value_seq'::TEXT); END; +$$ LANGUAGE PLPGSQL; + + +CREATE OR REPLACE FUNCTION search.staged_fts ( + + param_search_ou INT, + param_depth INT, + param_searches TEXT, -- JSON hash, to be turned into a resultset via search.parse_search_args + param_statuses INT[], + param_locations INT[], + param_audience TEXT[], + param_language TEXT[], + param_lit_form TEXT[], + param_types TEXT[], + param_forms TEXT[], + param_vformats TEXT[], + param_pref_lang TEXT, + param_pref_lang_multiplier REAL, + param_sort TEXT, + param_sort_desc BOOL, + metarecord BOOL, + staff BOOL, + param_rel_limit INT, + param_chk_limit INT, + param_skip_chk INT + +) RETURNS SETOF search.search_result AS $func$ +DECLARE + + current_res search.search_result%ROWTYPE; + query_part search.search_args%ROWTYPE; + phrase_query_part search.search_args%ROWTYPE; + rank_adjust_id INT; + core_rel_limit INT; + core_chk_limit INT; + core_skip_chk INT; + rank_adjust search.relevance_adjustment%ROWTYPE; + query_table TEXT; + tmp_text TEXT; + tmp_int INT; + current_rank TEXT; + ranks TEXT[] := '{}'; + query_table_alias TEXT; + from_alias_array TEXT[] := '{}'; + used_ranks TEXT[] := '{}'; + mb_field INT; + mb_field_list INT[]; + search_org_list INT[]; + select_clause TEXT := 'SELECT'; + from_clause TEXT := ' FROM metabib.metarecord_source_map m JOIN metabib.rec_descriptor mrd ON (m.source = mrd.record) '; + where_clause TEXT := ' WHERE 1=1 '; + mrd_used BOOL := FALSE; + sort_desc BOOL := FALSE; + + core_result RECORD; + core_cursor REFCURSOR; + core_rel_query TEXT; + vis_limit_query TEXT; + inner_where_clause TEXT; + + total_count INT := 0; + check_count INT := 0; + deleted_count INT := 0; + visible_count INT := 0; + excluded_count INT := 0; + +BEGIN + + core_rel_limit := COALESCE( param_rel_limit, 25000 ); + core_chk_limit := COALESCE( param_chk_limit, 1000 ); + core_skip_chk := COALESCE( param_skip_chk, 1 ); + + IF metarecord THEN + select_clause := select_clause || ' m.metarecord as id, array_accum(distinct m.source) as records,'; + ELSE + select_clause := select_clause || ' m.source as id, array_accum(distinct m.source) as records,'; + END IF; + + -- first we need to construct the base query + FOR query_part IN SELECT * FROM search.parse_search_args(param_searches) WHERE term_type = 'fts_query' LOOP + + inner_where_clause := 'index_vector @@ ' || query_part.term; + + IF query_part.field_name IS NOT NULL THEN + + SELECT id INTO mb_field + FROM config.metabib_field + WHERE field_class = query_part.field_class + AND name = query_part.field_name; + + IF FOUND THEN + inner_where_clause := inner_where_clause || + ' AND ' || 'field = ' || mb_field; + END IF; + + END IF; + + -- moving on to the rank ... + SELECT * INTO query_part + FROM search.parse_search_args(param_searches) + WHERE term_type = 'fts_rank' + AND table_alias = query_part.table_alias; + + current_rank := query_part.term || ' * ' || query_part.table_alias || '_weight.weight'; + + IF query_part.field_name IS NOT NULL THEN + + SELECT array_accum(distinct id) INTO mb_field_list + FROM config.metabib_field + WHERE field_class = query_part.field_class + AND name = query_part.field_name; + + ELSE + + SELECT array_accum(distinct id) INTO mb_field_list + FROM config.metabib_field + WHERE field_class = query_part.field_class; + + END IF; + + FOR rank_adjust IN SELECT * FROM search.relevance_adjustment WHERE active AND field IN ( SELECT * FROM search.explode_array( mb_field_list ) ) LOOP + + IF NOT rank_adjust.bump_type = ANY (used_ranks) THEN + + IF rank_adjust.bump_type = 'first_word' THEN + SELECT term INTO tmp_text + FROM search.parse_search_args(param_searches) + WHERE table_alias = query_part.table_alias AND term_type = 'word' + ORDER BY id + LIMIT 1; + + tmp_text := query_part.table_alias || '.value ILIKE ' || quote_literal( tmp_text || '%' ); + + ELSIF rank_adjust.bump_type = 'word_order' THEN + SELECT array_to_string( array_accum( term ), '%' ) INTO tmp_text + FROM search.parse_search_args(param_searches) + WHERE table_alias = query_part.table_alias AND term_type = 'word'; + + tmp_text := query_part.table_alias || '.value ILIKE ' || quote_literal( '%' || tmp_text || '%' ); + + ELSIF rank_adjust.bump_type = 'full_match' THEN + SELECT array_to_string( array_accum( term ), E'\\s+' ) INTO tmp_text + FROM search.parse_search_args(param_searches) + WHERE table_alias = query_part.table_alias AND term_type = 'word'; + + tmp_text := query_part.table_alias || '.value ~ ' || quote_literal( '^' || tmp_text || E'\\W*$' ); + + END IF; + + + IF tmp_text IS NOT NULL THEN + current_rank := current_rank || ' * ( CASE WHEN ' || tmp_text || + ' THEN ' || rank_adjust.multiplier || '::REAL ELSE 1.0 END )'; + END IF; + + used_ranks := array_append( used_ranks, rank_adjust.bump_type ); + + END IF; + + END LOOP; + + ranks := array_append( ranks, current_rank ); + used_ranks := '{}'; + + FOR phrase_query_part IN + SELECT * + FROM search.parse_search_args(param_searches) + WHERE term_type = 'phrase' + AND table_alias = query_part.table_alias LOOP + + tmp_text := replace( phrase_query_part.term, '*', E'\\*' ); + tmp_text := replace( tmp_text, '?', E'\\?' ); + tmp_text := replace( tmp_text, '+', E'\\+' ); + tmp_text := replace( tmp_text, '|', E'\\|' ); + tmp_text := replace( tmp_text, '(', E'\\(' ); + tmp_text := replace( tmp_text, ')', E'\\)' ); + tmp_text := replace( tmp_text, '[', E'\\[' ); + tmp_text := replace( tmp_text, ']', E'\\]' ); + + inner_where_clause := inner_where_clause || ' AND ' || 'value ~* ' || quote_literal( E'(^|\\W+)' || regexp_replace(tmp_text, E'\\s+',E'\\\\s+','g') || E'(\\W+|\$)' ); + + END LOOP; + + query_table := search.pick_table(query_part.field_class); + + from_clause := from_clause || + ' JOIN ( SELECT * FROM ' || query_table || ' WHERE ' || inner_where_clause || + CASE WHEN core_rel_limit > 0 THEN ' LIMIT ' || core_rel_limit::TEXT ELSE '' END || ' ) AS ' || query_part.table_alias || + ' ON ( m.source = ' || query_part.table_alias || '.source )' || + ' JOIN config.metabib_field AS ' || query_part.table_alias || '_weight' || + ' ON ( ' || query_part.table_alias || '.field = ' || query_part.table_alias || '_weight.id AND ' || query_part.table_alias || '_weight.search_field)'; + + from_alias_array := array_append(from_alias_array, query_part.table_alias); + + END LOOP; + + IF param_pref_lang IS NOT NULL AND param_pref_lang_multiplier IS NOT NULL THEN + current_rank := ' CASE WHEN mrd.item_lang = ' || quote_literal( param_pref_lang ) || + ' THEN ' || param_pref_lang_multiplier || '::REAL ELSE 1.0 END '; + + --ranks := array_append( ranks, current_rank ); + END IF; + + current_rank := ' AVG( ( (' || array_to_string( ranks, ') + (' ) || ') ) * ' || current_rank || ' ) '; + select_clause := select_clause || current_rank || ' AS rel,'; + + sort_desc = param_sort_desc; + + IF param_sort = 'pubdate' THEN + + tmp_text := '999999'; + IF param_sort_desc THEN tmp_text := '0'; END IF; + + current_rank := $$ + ( COALESCE( FIRST (( + SELECT SUBSTRING(frp.value FROM E'\\d{4}') + FROM metabib.full_rec frp + WHERE frp.record = m.source + AND frp.tag = '260' + AND frp.subfield = 'c' + LIMIT 1 + )), $$ || quote_literal(tmp_text) || $$ )::INT ) + $$; + + ELSIF param_sort = 'title' THEN + + tmp_text := 'zzzzzz'; + IF param_sort_desc THEN tmp_text := ' '; END IF; + + current_rank := $$ + ( COALESCE( FIRST (( + SELECT LTRIM(SUBSTR( frt.value, COALESCE(SUBSTRING(frt.ind2 FROM E'\\d+'),'0')::INT + 1 )) + FROM metabib.full_rec frt + WHERE frt.record = m.source + AND frt.tag = '245' + AND frt.subfield = 'a' + LIMIT 1 + )),$$ || quote_literal(tmp_text) || $$)) + $$; + + ELSIF param_sort = 'author' THEN + + tmp_text := 'zzzzzz'; + IF param_sort_desc THEN tmp_text := ' '; END IF; + + current_rank := $$ + ( COALESCE( FIRST (( + SELECT LTRIM(fra.value) + FROM metabib.full_rec fra + WHERE fra.record = m.source + AND fra.tag LIKE '1%' + AND fra.subfield = 'a' + ORDER BY fra.tag::text::int + LIMIT 1 + )),$$ || quote_literal(tmp_text) || $$)) + $$; + + ELSIF param_sort = 'create_date' THEN + current_rank := $$( FIRST (( SELECT create_date FROM biblio.record_entry rbr WHERE rbr.id = m.source)) )$$; + ELSIF param_sort = 'edit_date' THEN + current_rank := $$( FIRST (( SELECT edit_date FROM biblio.record_entry rbr WHERE rbr.id = m.source)) )$$; + ELSE + sort_desc := NOT COALESCE(param_sort_desc, FALSE); + END IF; + + select_clause := select_clause || current_rank || ' AS rank'; + + -- now add the other qualifiers + IF param_audience IS NOT NULL AND array_upper(param_audience, 1) > 0 THEN + where_clause = where_clause || $$ AND mrd.audience IN ('$$ || array_to_string(param_audience, $$','$$) || $$') $$; + END IF; + + IF param_language IS NOT NULL AND array_upper(param_language, 1) > 0 THEN + where_clause = where_clause || $$ AND mrd.item_lang IN ('$$ || array_to_string(param_language, $$','$$) || $$') $$; + END IF; + + IF param_lit_form IS NOT NULL AND array_upper(param_lit_form, 1) > 0 THEN + where_clause = where_clause || $$ AND mrd.lit_form IN ('$$ || array_to_string(param_lit_form, $$','$$) || $$') $$; + END IF; + + IF param_types IS NOT NULL AND array_upper(param_types, 1) > 0 THEN + where_clause = where_clause || $$ AND mrd.item_type IN ('$$ || array_to_string(param_types, $$','$$) || $$') $$; + END IF; + + IF param_forms IS NOT NULL AND array_upper(param_forms, 1) > 0 THEN + where_clause = where_clause || $$ AND mrd.item_form IN ('$$ || array_to_string(param_forms, $$','$$) || $$') $$; + END IF; + + IF param_vformats IS NOT NULL AND array_upper(param_vformats, 1) > 0 THEN + where_clause = where_clause || $$ AND mrd.vr_format IN ('$$ || array_to_string(param_vformats, $$','$$) || $$') $$; + END IF; + + core_rel_query := select_clause || from_clause || where_clause || + ' GROUP BY 1 ORDER BY 4' || CASE WHEN sort_desc THEN ' DESC' ELSE ' ASC' END || ';'; + --RAISE NOTICE 'Base Query: %', core_rel_query; + + IF param_depth IS NOT NULL THEN + SELECT array_accum(distinct id) INTO search_org_list FROM actor.org_unit_descendants( param_search_ou, param_depth ); + ELSE + SELECT array_accum(distinct id) INTO search_org_list FROM actor.org_unit_descendants( param_search_ou ); + END IF; + + OPEN core_cursor FOR EXECUTE core_rel_query; + + LOOP + + FETCH core_cursor INTO core_result; + EXIT WHEN NOT FOUND; + + + IF total_count % 1000 = 0 THEN + -- RAISE NOTICE ' % total, % checked so far ... ', total_count, check_count; + END IF; + + IF core_chk_limit > 0 AND total_count - core_skip_chk + 1 >= core_chk_limit THEN + total_count := total_count + 1; + CONTINUE; + END IF; + + total_count := total_count + 1; + + CONTINUE WHEN param_skip_chk IS NOT NULL and total_count < param_skip_chk; + + check_count := check_count + 1; + + PERFORM 1 FROM biblio.record_entry b WHERE NOT b.deleted AND b.id IN ( SELECT * FROM search.explode_array( core_result.records ) ); + IF NOT FOUND THEN + -- RAISE NOTICE ' % were all deleted ... ', core_result.records; + deleted_count := deleted_count + 1; + CONTINUE; + END IF; + + PERFORM 1 + FROM biblio.record_entry b + JOIN config.bib_source s ON (b.source = s.id) + WHERE s.transcendant + AND b.id IN ( SELECT * FROM search.explode_array( core_result.records ) ); + + IF FOUND THEN + -- RAISE NOTICE ' % were all transcendant ... ', core_result.records; + visible_count := visible_count + 1; + + current_res.id = core_result.id; + current_res.rel = core_result.rel; + + tmp_int := 1; + IF metarecord THEN + SELECT COUNT(DISTINCT s.source) INTO tmp_int FROM metabib.metarecord_source_map s WHERE s.metarecord = core_result.id; + END IF; + + IF tmp_int = 1 THEN + current_res.record = core_result.records[1]; + ELSE + current_res.record = NULL; + END IF; + + RETURN NEXT current_res; + + CONTINUE; + END IF; + + IF param_statuses IS NOT NULL AND array_upper(param_statuses, 1) > 0 THEN + + PERFORM 1 + FROM asset.call_number cn + JOIN asset.copy cp ON (cp.call_number = cn.id) + WHERE NOT cn.deleted + AND NOT cp.deleted + AND cp.status IN ( SELECT * FROM search.explode_array( param_statuses ) ) + AND cn.record IN ( SELECT * FROM search.explode_array( core_result.records ) ) + AND cp.circ_lib IN ( SELECT * FROM search.explode_array( search_org_list ) ) + LIMIT 1; + + IF NOT FOUND THEN + -- RAISE NOTICE ' % were all status-excluded ... ', core_result.records; + excluded_count := excluded_count + 1; + CONTINUE; + END IF; + + END IF; + + IF param_locations IS NOT NULL AND array_upper(param_locations, 1) > 0 THEN + + PERFORM 1 + FROM asset.call_number cn + JOIN asset.copy cp ON (cp.call_number = cn.id) + WHERE NOT cn.deleted + AND NOT cp.deleted + AND cp.location IN ( SELECT * FROM search.explode_array( param_locations ) ) + AND cn.record IN ( SELECT * FROM search.explode_array( core_result.records ) ) + AND cp.circ_lib IN ( SELECT * FROM search.explode_array( search_org_list ) ) + LIMIT 1; + + IF NOT FOUND THEN + -- RAISE NOTICE ' % were all copy_location-excluded ... ', core_result.records; + excluded_count := excluded_count + 1; + CONTINUE; + END IF; + + END IF; + + IF staff IS NULL OR NOT staff THEN + + PERFORM 1 + FROM asset.call_number cn + JOIN asset.copy cp ON (cp.call_number = cn.id) + JOIN actor.org_unit a ON (cp.circ_lib = a.id) + JOIN asset.copy_location cl ON (cp.location = cl.id) + JOIN config.copy_status cs ON (cp.status = cs.id) + WHERE NOT cn.deleted + AND NOT cp.deleted + AND cs.holdable + AND cl.opac_visible + AND cp.opac_visible + AND a.opac_visible + AND cp.circ_lib IN ( SELECT * FROM search.explode_array( search_org_list ) ) + AND cn.record IN ( SELECT * FROM search.explode_array( core_result.records ) ) + LIMIT 1; + + IF NOT FOUND THEN + -- RAISE NOTICE ' % were all visibility-excluded ... ', core_result.records; + excluded_count := excluded_count + 1; + CONTINUE; + END IF; + + ELSE + + PERFORM 1 + FROM asset.call_number cn + JOIN asset.copy cp ON (cp.call_number = cn.id) + JOIN actor.org_unit a ON (cp.circ_lib = a.id) + JOIN asset.copy_location cl ON (cp.location = cl.id) + JOIN config.copy_status cs ON (cp.status = cs.id) + WHERE NOT cn.deleted + AND NOT cp.deleted + AND cp.circ_lib IN ( SELECT * FROM search.explode_array( search_org_list ) ) + AND cn.record IN ( SELECT * FROM search.explode_array( core_result.records ) ) + LIMIT 1; + + IF NOT FOUND THEN + + PERFORM 1 + FROM asset.call_number cn + WHERE cn.record IN ( SELECT * FROM search.explode_array( core_result.records ) ) + LIMIT 1; + + IF FOUND THEN + -- RAISE NOTICE ' % were all visibility-excluded ... ', core_result.records; + excluded_count := excluded_count + 1; + CONTINUE; + END IF; + + END IF; + + END IF; + + visible_count := visible_count + 1; + + current_res.id = core_result.id; + current_res.rel = core_result.rel; + + tmp_int := 1; + IF metarecord THEN + SELECT COUNT(DISTINCT s.source) INTO tmp_int FROM metabib.metarecord_source_map s WHERE s.metarecord = core_result.id; + END IF; + + IF tmp_int = 1 THEN + current_res.record = core_result.records[1]; + ELSE + current_res.record = NULL; + END IF; + + RETURN NEXT current_res; + + IF visible_count % 1000 = 0 THEN + -- RAISE NOTICE ' % visible so far ... ', visible_count; + END IF; + + END LOOP; + + current_res.id = NULL; + current_res.rel = NULL; + current_res.record = NULL; + current_res.total = total_count; + current_res.checked = check_count; + current_res.deleted = deleted_count; + current_res.visible = visible_count; + current_res.excluded = excluded_count; + + CLOSE core_cursor; + + RETURN NEXT current_res; + +END; +$func$ LANGUAGE PLPGSQL; + +COMMIT; + diff --git a/Open-ILS/src/sql/Pg/110.hold_matrix.sql b/Open-ILS/src/sql/Pg/110.hold_matrix.sql index b80dcdd60b..857f72e071 100644 --- a/Open-ILS/src/sql/Pg/110.hold_matrix.sql +++ b/Open-ILS/src/sql/Pg/110.hold_matrix.sql @@ -1,3 +1,15 @@ +/* + +-- If, for some reason, you need to reload this chunk of the schema +-- just use the following two statements to remove the tables before +-- running the rest of the file. See 950.data.seed-values.sql for +-- the one default entry to add back to config.hold_matrix_matchpoint. + +DROP TABLE config.hold_matrix_matchpoint CASCADE; +DROP TABLE config.hold_matrix_test CASCADE; + +*/ + BEGIN; @@ -42,6 +54,7 @@ CREATE TABLE config.hold_matrix_test ( transit_range INT REFERENCES actor.org_unit_type (id) DEFERRABLE INITIALLY DEFERRED, -- Can circ inside range of cn.owner/cp.circ_lib at depth of the org_unit_type specified here max_holds INT, -- Total hold requests must be less than this, NULL means skip (always pass) include_frozen_holds BOOL NOT NULL DEFAULT TRUE, -- Include frozen hold requests in the count for max_holds test + stop_blocked_user BOOL NOT NULL DEFAULT FALSE, -- Stop users who cannot check out items from placing holds age_hold_protect_rule INT REFERENCES config.rule_age_hold_protect (id) DEFERRABLE INITIALLY DEFERRED -- still not sure we want to move this off the copy ); @@ -164,6 +177,7 @@ DECLARE hold_count INT; hold_transit_prox INT; frozen_hold_count INT; + patron_penalties INT; done BOOL := FALSE; BEGIN SELECT INTO user_object * FROM actor.usr WHERE id = match_user; @@ -177,6 +191,15 @@ BEGIN RETURN; END IF; + -- Fail if user is barred + IF user_object.barred IS TRUE THEN + result.fail_part := 'actor.usr.barred'; + result.success := FALSE; + done := TRUE; + RETURN NEXT result; + RETURN; + END IF; + SELECT INTO item_object * FROM asset.copy WHERE id = match_item; -- Fail if we couldn't find a copy @@ -229,6 +252,19 @@ BEGIN END IF; END IF; + IF hold_test.stop_blocked_user IS TRUE THEN + SELECT INTO patron_penalties COUNT(*) + FROM actor.usr_standing_penalty + WHERE usr = match_user; + + IF items_out > 0 THEN + result.fail_part := 'config.hold_matrix_test.stop_blocked_user'; + result.success := FALSE; + done := TRUE; + RETURN NEXT result; + END IF; + END IF; + IF hold_test.max_holds IS NOT NULL THEN SELECT INTO hold_count COUNT(*) FROM action.hold_request diff --git a/Open-ILS/src/sql/Pg/300.schema.staged_search.sql b/Open-ILS/src/sql/Pg/300.schema.staged_search.sql index 8014426eec..158e16170b 100644 --- a/Open-ILS/src/sql/Pg/300.schema.staged_search.sql +++ b/Open-ILS/src/sql/Pg/300.schema.staged_search.sql @@ -258,7 +258,7 @@ BEGIN tmp_text := '999999'; IF param_sort_desc THEN tmp_text := '0'; END IF; - current_rank := $$ COALESCE( mrd.date1, $$ || quote_literal(tmp_text) || $$ )::INT ) $$; + current_rank := $$ COALESCE( FIRST(NULLIF(REGEXP_REPLACE(mrd.date1, E'\\D+', '9', 'g'),'')), $$ || quote_literal(tmp_text) || $$ )::INT $$; ELSIF param_sort = 'title' THEN diff --git a/Open-ILS/src/support-scripts/generate_circ_notices.pl b/Open-ILS/src/support-scripts/generate_circ_notices.pl new file mode 100755 index 0000000000..1b78dd7f5a --- /dev/null +++ b/Open-ILS/src/support-scripts/generate_circ_notices.pl @@ -0,0 +1,355 @@ +#!/usr/bin/perl +# --------------------------------------------------------------- +# Copyright (C) 2008 Georgia Public Library Service +# Bill Erickson +# +# This program is free software; you can redistribute it and/or +# modify it under the terms of the GNU General Public License +# as published by the Free Software Foundation; either version 2 +# of the License, or (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# --------------------------------------------------------------- +use strict; use warnings; +require 'oils_header.pl'; +use vars qw/$logger/; +use DateTime; +use Template; +use Data::Dumper; +use Email::Send; +use Getopt::Long; +use Unicode::Normalize; +use DateTime::Format::ISO8601; +use OpenSRF::Utils qw/:datetime/; +use OpenSRF::Utils::JSON; +use OpenSRF::Utils::SettingsClient; +use OpenSRF::AppSession; +use OpenILS::Const qw/:const/; +use OpenILS::Application::AppUtils; +use OpenILS::Const qw/:const/; +my $U = 'OpenILS::Application::AppUtils'; + +my $settings = undef; +my $e = OpenILS::Utils::CStoreEditor->new; + +my @global_overdue_circs; # all circ collections stored here go into the final global XML file + +my ($osrf_config, $send_email, $gen_day_intervals, $days_back, $gen_global_templates) = + ('/openils/conf/opensrf_core.xml', 0, 0, 0, 0); + +GetOptions( + 'osrf_osrf_config=s' => \$osrf_config, + 'send-email' => \$send_email, + 'generate-day-intervals' => \$gen_day_intervals, + 'generate-global-templates' => \$gen_global_templates, + 'days-back=s' => \$days_back, +); + +sub help { + print < + + --send-emails If set, generate email notices + + --generate-day-intervals If set, notices which have a notify_interval of >= 1 day will be processed. + + --days-back This is used to set the effective run date of the script. + This is useful if you don't want to generate notices on certain days. For example, if you don't + generate notices on the weekend, you would run this script on weekdays and set --days-back to + 0,1,2 when it's run on Monday to capture any notices from Saturday and Sunday. +HELP +} + + +sub main { + osrf_connect($osrf_config); + $settings = OpenSRF::Utils::SettingsClient->new; + + my $sender_address = $settings->config_value(notifications => 'sender_address'); + my $od_sender_addr = $settings->config_value(notifications => overdue => 'sender_address') || $sender_address; + my $pd_sender_addr = $settings->config_value(notifications => predue => 'sender_address') || $sender_address; + my $overdue_notices = $settings->config_value(notifications => overdue => 'notice'); + my $predue_notices = $settings->config_value(notifications => predue => 'notice'); + + $overdue_notices = [$overdue_notices] unless ref $overdue_notices eq 'ARRAY'; + $predue_notices = [$predue_notices] unless ref $predue_notices eq 'ARRAY'; + + my @overdues = sort { + OpenSRF::Utils->interval_to_seconds($a->{notify_interval}) <=> + OpenSRF::Utils->interval_to_seconds($b->{notify_interval}) } @$overdue_notices; + + my @predues = sort { + OpenSRF::Utils->interval_to_seconds($a->{notify_interval}) <=> + OpenSRF::Utils->interval_to_seconds($b->{notify_interval}) } @$predue_notices; + + generate_notice_set($_, 'overdue') for @overdues; + generate_notice_set($_, 'predue') for @predues; + + generate_global_overdue_file() if $gen_global_templates; +} + +sub generate_global_overdue_file { + $logger->info("notice: processing ".scalar(@global_overdue_circs)." for global template"); + return unless @global_overdue_circs; + + my $tt = Template->new({ABSOLUTE => 1}); + + $tt->process( + $settings->config_value(notifications => overdue => 'combined_template'), + { + overdues => \@global_overdue_circs, + get_bib_attr => \&get_bib_attr, + parse_due_date => \&parse_due_date, # let the templates decide date format + escape_xml => \&escape_xml, + }, + \&global_overdue_output + ) or $logger->error('notice: Template error '.$tt->error); +} + +sub global_overdue_output { + print shift(); +} + + +sub generate_notice_set { + my($notice, $type) = @_; + + my $notify_interval = OpenSRF::Utils->interval_to_seconds($notice->{notify_interval}); + $notify_interval = -$notify_interval if $type eq 'overdue'; + + my ($start_date, $end_date) = make_date_range(-$days_back + $notify_interval); + + $logger->info("notice: retrieving circs with due date in range $start_date -> $end_date"); + + my $QUERY = { + select => { + circ => ['id'] + }, + from => 'circ', + where => { + '+circ' => { + checkin_time => undef, + '-or' => [ + {stop_fines => ["LOST","LONGOVERDUE","CLAIMSRETURNED"]}, + {stop_fines => undef} + ], + due_date => {between => [$start_date, $end_date]}, + } + } + }; + + # if a circ duration is defined for this type of notice + if(my $durs = $notice->{circ_duration_range}) { + $QUERY->{where}->{'+circ'}->{duration} = {between => [$durs->{from}, $durs->{to}]}; + } + + my $circs = $e->json_query($QUERY, {timeout => 18000, substream => 1}); + process_circs($notice, $type, map {$_->{id}} @$circs); +} + + +sub process_circs { + my $notice = shift; + my $type = shift; + my @circs = @_; + + return unless @circs; + + $logger->info("notice: processing $type notices with notify interval ". + $notice->{notify_interval}." and ".scalar(@circs)." circs"); + + my $org; + my $patron; + my @current; + + my $x = 0; + for my $circ (@circs) { + $circ = $e->retrieve_action_circulation($circ); + + if( !defined $org or + $circ->circ_lib != $org or $circ->usr ne $patron ) { + $org = $circ->circ_lib; + $patron = $circ->usr; + generate_notice($notice, $type, @current) if @current; + @current = (); + } + + push(@current, $circ); + $x++; + } + + $logger->info("notice: processed $x circs"); + generate_notice($notice, $type, @current); +} + +my %ORG_CACHE; + +sub generate_notice { + my $notice = shift; + my $type = shift; + my @circs = @_; + return unless @circs; + my $circ_list = fetch_circ_data(@circs); + my $tt = Template->new({ABSOLUTE => 1}); + + my $sender = $settings->config_value( + notifications => $type => 'sender_address') || + $settings->config_value(notifications => 'sender_address'); + + my $context = { + circ_list => $circ_list, + get_bib_attr => \&get_bib_attr, + parse_due_date => \&parse_due_date, # let the templates decide date format + smtp_sender => $sender, + smtp_repley => $sender, # XXX + notice => $notice, + }; + + push(@global_overdue_circs, $context) if + $type eq 'overdue' and $notice->{file_append} =~ /always/i; + + if($send_email and $notice->{email_notify} and + my $email = $circ_list->[0]->usr->email) { + + if(my $tmpl = $notice->{email_template}) { + $tt->process($tmpl, $context, + sub { + email_template_output($notice, $type, $context, $email, @_); + } + ) or $logger->error('notice: Template error '.$tt->error); + } + } else { + push(@global_overdue_circs, $context) + if $type eq 'overdue' and $notice->{file_append} =~ /noemail/i; + } +} + +sub get_bib_attr { + my $circ = shift; + my $attr = shift; + my $copy = $circ->target_copy; + if($copy->call_number->id == OILS_PRECAT_CALL_NUMBER) { + return $copy->dummy_title || '' if $attr eq 'title'; + return $copy->dummy_author || '' if $attr eq 'author'; + } else { + my $mvr = $U->record_to_mvr($copy->call_number->record); + return $mvr->title || '' if $attr eq 'title'; + return $mvr->author || '' if $attr eq 'author'; + } +} + +# provides a date that Template::Plugin::Date can parse +sub parse_due_date { + my $circ = shift; + my $due = DateTime::Format::ISO8601->new->parse_datetime(clense_ISO8601($circ->due_date)); + return sprintf( + "%0.2d:%0.2d:%0.2d %0.2d-%0.2d-%0.4d", + $due->hour, + $due->minute, + $due->second, + $due->day, + $due->month, + $due->year + ); +} + +sub escape_xml { + my $str = shift; + $str =~ s/&/&/sog; + $str =~ s//>/sog; + return $str; +} + + +sub email_template_output { + my $notice = shift; + my $type = shift; + my $context = shift; + my $email = shift; + my $msg = shift; + + my $sender = Email::Send->new({mailer => 'SMTP'}); + my $smtp_server = $settings->config_value(notifications => 'smtp_server'); + $logger->debug("notice: smtp server is $smtp_server"); + $sender->mailer_args([Host => $smtp_server]); + my $stat = $sender->send($msg); + + if( $stat and $stat->type eq 'success' ) { + $logger->info("notice: successfully sent $type email to $email"); + } else { + $logger->warn("notice: unable to send $type email to $email: ".Dumper($stat)); + # if we were unable to send the email, add this notice set to the global notify set + push(@global_overdue_circs, $context) + if $type eq 'overdue' and $notice->{file_append} =~ /noemail/i; + } +} + +sub fetch_circ_data { + my @circs = @_; + + my $circ_lib_id = $circs[0]->circ_lib; + my $usr_id = $circs[0]->usr; + $logger->debug("notice: printing user:$usr_id circ_lib:$circ_lib_id"); + + my $usr = $e->retrieve_actor_user([ + $usr_id, + { flesh => 1, + flesh_fields => { + au => [qw/card billing_address mailing_address/] + } + } + ]); + + my $circ_lib = $ORG_CACHE{$circ_lib_id} || + $e->retrieve_actor_org_unit([ + $circ_lib_id, + { flesh => 1, + flesh_fields => { + aou => [qw/billing_address mailing_address/], + } + } + ]); + $ORG_CACHE{$circ_lib_id} = $circ_lib; + + my $circ_objs = $e->search_action_circulation([ + {id => [map {$_->id} @circs]}, + { flesh => 3, + flesh_fields => { + circ => [q/target_copy/], + acp => ['call_number'], + acn => ['record'], + } + } + ]); + + $_->circ_lib($circ_lib) for @$circ_objs; + $_->usr($usr) for @$circ_objs; + + return $circ_objs +} + + +sub make_date_range { + my $offset = shift; + #my $is_day_precision = shift; # window? + + my $epoch = CORE::time + $offset; + my $date = DateTime->from_epoch(epoch => $epoch, time_zone => 'local'); + + $date->set_hour(0); + $date->set_minute(0); + $date->set_second(0); + my $start = "$date"; + + $date->set_hour(23); + $date->set_minute(59); + $date->set_second(59); + + return ($start, "$date"); +} + +main(); diff --git a/Open-ILS/src/support-scripts/settings-tester.pl b/Open-ILS/src/support-scripts/settings-tester.pl index 9cd2e54321..cf940948da 100755 --- a/Open-ILS/src/support-scripts/settings-tester.pl +++ b/Open-ILS/src/support-scripts/settings-tester.pl @@ -385,3 +385,4 @@ Spreadsheet::WriteExcel::Big Tie::IxHash Parse::RecDescent SRU +JSON::XS diff --git a/Open-ILS/web/opac/extras/selfcheck/selfcheck.js b/Open-ILS/web/opac/extras/selfcheck/selfcheck.js index e723aad3da..45ae925722 100644 --- a/Open-ILS/web/opac/extras/selfcheck/selfcheck.js +++ b/Open-ILS/web/opac/extras/selfcheck/selfcheck.js @@ -33,6 +33,7 @@ var successfulItems = {}; var scanTimeout = 800; var scanTimeoutId; var patronBarcodeRegex; +var orgUnit; function selfckInit() { @@ -41,13 +42,12 @@ function selfckInit() { selfckSetupPrinter(); - /* - XXX we need org information (from the proxy?) - var t = fetchOrgSettingDefault(1, 'circ.selfcheck.patron_login_timeout'); + orgUnit = findOrgUnitSN(cgi.param('l')) || globalOrgTree; + + var t = fetchOrgSettingDefault(orgUnit.id(), 'circ.selfcheck.patron_login_timeout'); patronTimeout = (t) ? parseInt(t) * 1000 : patronTimeout; - */ - var reg = fetchOrgSettingDefault(globalOrgTree.id(), 'opac.barcode_regex'); + var reg = fetchOrgSettingDefault(orgUnit.id(), 'opac.barcode_regex'); if(reg) patronBarcodeRegex = new RegExp(reg); if(!staff) { @@ -251,7 +251,16 @@ function selfckCheckout() { * attempts renewal. Any other events will display a message */ function selfckHandleCoResult(r) { - var evt = r.getResultObject(); + var evt; + + try { + evt = r.getResultObject(); + } catch(E) { + pendingXact = false; + selfckShowMsgNode({textcode:'UNKNOWN'}); + appendClear($('selfck-errors'), text(E.toString())); + return; + } if(evt.textcode == 'SUCCESS') { selfckDislplayCheckout(evt); diff --git a/Open-ILS/web/opac/extras/selfcheck/selfcheck.xml b/Open-ILS/web/opac/extras/selfcheck/selfcheck.xml index 4337899087..9d09b354d9 100644 --- a/Open-ILS/web/opac/extras/selfcheck/selfcheck.xml +++ b/Open-ILS/web/opac/extras/selfcheck/selfcheck.xml @@ -87,7 +87,7 @@ &selfck.event.co_success; - &selfck.event.unknown; + &selfck.event.co_unknown; &selfck.event.patron_not_found; &selfck.event.item_nocirc; &selfck.event.item_noncat; @@ -149,6 +149,8 @@ +
+ diff --git a/Open-ILS/web/opac/locale/en-US/lang.dtd b/Open-ILS/web/opac/locale/en-US/lang.dtd index 8ba6519715..3ebd54f8e7 100644 --- a/Open-ILS/web/opac/locale/en-US/lang.dtd +++ b/Open-ILS/web/opac/locale/en-US/lang.dtd @@ -2027,6 +2027,7 @@ + diff --git a/Open-ILS/xul/staff_client/server/admin/hold_pull_list.js b/Open-ILS/xul/staff_client/server/admin/hold_pull_list.js index 54cb3d2215..0567d5b4fa 100644 --- a/Open-ILS/xul/staff_client/server/admin/hold_pull_list.js +++ b/Open-ILS/xul/staff_client/server/admin/hold_pull_list.js @@ -5,7 +5,7 @@ var FETCH_USER = 'open-ils.actor:open-ils.actor.user.fleshed.retrieve'; var FETCH_VOLUME = 'open-ils.search:open-ils.search.callnumber.retrieve'; var myPerms = [ 'VIEW_HOLD' ]; -var HOLD_LIST_LIMIT = 50; +var HOLD_LIST_LIMIT = 100; var numHolds = 0; var listOffset = 0; diff --git a/Open-ILS/xul/staff_client/server/admin/hold_pull_list_classic.js b/Open-ILS/xul/staff_client/server/admin/hold_pull_list_classic.js index cbfc36de91..94683f7e6b 100644 --- a/Open-ILS/xul/staff_client/server/admin/hold_pull_list_classic.js +++ b/Open-ILS/xul/staff_client/server/admin/hold_pull_list_classic.js @@ -6,7 +6,7 @@ var FETCH_USER = 'open-ils.actor:open-ils.actor.user.fleshed.retrieve'; var FETCH_VOLUME = 'open-ils.search:open-ils.search.callnumber.retrieve'; var myPerms = [ 'VIEW_HOLD' ]; -var HOLD_LIST_LIMIT = 50; +var HOLD_LIST_LIMIT = 100; var numHolds = 0; var listOffset = 0; diff --git a/Open-ILS/xul/staff_client/server/cat/copy_browser.js b/Open-ILS/xul/staff_client/server/cat/copy_browser.js index 6e307615c4..28f17f4326 100644 --- a/Open-ILS/xul/staff_client/server/cat/copy_browser.js +++ b/Open-ILS/xul/staff_client/server/cat/copy_browser.js @@ -536,11 +536,12 @@ cat.copy_browser.prototype = { window.xulG.url_prefix(urls.XUL_VOLUME_EDITOR), title, 'chrome,modal,resizable', - { 'volumes' : list } + { 'volumes' : JSON2js(js2JSON(list)) } ); /* FIXME -- need to unique the temp space, and not rely on modalness of window */ //obj.data.stash_retrieve(); + if (typeof my_xulG.update_these_volumes == 'undefined') { return; } var volumes = my_xulG.volumes; if (!volumes) return; diff --git a/Open-ILS/xul/staff_client/server/cat/marcedit.js b/Open-ILS/xul/staff_client/server/cat/marcedit.js index 7c82fed073..293c971bfd 100644 --- a/Open-ILS/xul/staff_client/server/cat/marcedit.js +++ b/Open-ILS/xul/staff_client/server/cat/marcedit.js @@ -1893,35 +1893,45 @@ function validateAuthority (button) { var tag = row.firstChild; if (!control_map[tag.value]) continue + button.setAttribute('label', label + ' - ' + tag.value); var ind1 = tag.nextSibling; var ind2 = ind1.nextSibling; var subfields = ind2.nextSibling.childNodes; + var tags = {}; + for (var j = 0; j < subfields.length; j++) { var sf = subfields[j]; + var sf_code = sf.childNodes[1].value; + var sf_value = sf.childNodes[2].value; - if (!control_map[tag.value][sf.childNodes[1].value]) continue; - - button.setAttribute('label', label + ' - ' + tag.value + sf.childNodes[1].value); + if (!control_map[tag.value][sf_code]) continue; var found = 0; - for (var k in control_map[tag.value][sf.childNodes[1].value]) { - var x = searchAuthority(sf.childNodes[2].value, k, control_map[tag.value][sf.childNodes[1].value][k], 1); - var res = new XML( x.responseText ); - - if (res.gw::payload.gw::array.gw::string.length()) { - found = 1; - break; - } + for (var a_tag in control_map[tag.value][sf_code]) { + if (!tags[a_tag]) tags[a_tag] = []; + tags[a_tag].push({ term : sf_value, subfield : sf_code }); } - if (!found) { + } + + for (var val_tag in tags) { + var auth_data = validateBibField( [val_tag], tags[val_tag]); + var res = new XML( auth_data.responseText ); + found = parseInt(res.gw::payload.gw::string.toString()); + if (found) break; + } + + // XXX If adt, etc should be validated separately from vxz, etc then move this up into the above for loop + for (var j = 0; j < subfields.length; j++) { + var sf = subfields[j]; + if (!found) { sf.childNodes[2].inputField.style.color = 'red'; } else { sf.childNodes[2].inputField.style.color = 'black'; } - } + } } button.setAttribute('label', label); @@ -1930,6 +1940,19 @@ function validateAuthority (button) { } +function validateBibField (tags, searches) { + var url = "/gateway?input_format=json&format=xml&service=open-ils.search&method=open-ils.search.authority.validate.tag"; + url += '¶m="tags"¶m=' + js2JSON(tags); + url += '¶m="searches"¶m=' + js2JSON(searches); + + + var req = new XMLHttpRequest(); + req.open('GET',url,false); + req.send(null); + + return req; + +} function searchAuthority (term, tag, sf, limit) { var url = "/gateway?input_format=json&format=xml&service=open-ils.search&method=open-ils.search.authority.fts"; url += '¶m="term"¶m="' + term + '"'; diff --git a/Open-ILS/xul/staff_client/server/cat/marcedit.xul b/Open-ILS/xul/staff_client/server/cat/marcedit.xul index d42c4c2fc8..126da262f2 100644 --- a/Open-ILS/xul/staff_client/server/cat/marcedit.xul +++ b/Open-ILS/xul/staff_client/server/cat/marcedit.xul @@ -13,6 +13,7 @@