JBAS-1437 Backstage importer process bibs first

When processing a ZIP file, process bib files before authority files.
On large data sets, processing bibs first allows staff to resume normal
bib editing operations earlier.

Signed-off-by: Bill Erickson <berickxx@gmail.com>

JBAS-1437 Backstage importer --marc-file option

Importer now supports processing a single MARC file in addition to
ZIP files as before.  Zip file option is now specified via --zip-file.

Signed-off-by: Bill Erickson <berickxx@gmail.com>

JBAS-1437 Bib batch linker continued

Bug fixes, documentation.

Signed-off-by: Bill Erickson <berickxx@gmail.com>

JBAS-1437 Remove legacy linking/backstage scripts

Signed-off-by: Bill Erickson <berickxx@gmail.com>

JBAS-1437 Move authority/BS scripts to authority-control

Signed-off-by: Bill Erickson <berickxx@gmail.com>

JBAS-1437 bib linking log more info

Signed-off-by: Bill Erickson <berickxx@gmail.com>

JBAS-1437 Week-spanning bib link batcher script

Signed-off-by: Bill Erickson <berickxx@gmail.com>

JBAS-1437 Bibs-to-link finder and batch generator

Signed-off-by: Bill Erickson <berickxx@gmail.com>

JBAS-1437 B2A linking additional query options

* --all uses direct DB connection
  * Fetching all record ID's via cstore is a bad idea.
* order by ID
* better logging
* --sort-desc option

Signed-off-by: Bill Erickson <berickxx@gmail.com>

JBAS-1437 B2A link clean up bogus fast headings; leave OK ones

Signed-off-by: Bill Erickson <berickxx@gmail.com>

JBAS-1437 Backstage scripts default DB settings

Signed-off-by: Bill Erickson <berickxx@gmail.com>

JBAS-1437 Auth linker uses 'KCLS' control number identifer

Affects bib-to-auth and auth-to-auth linking.

Signed-off-by: Bill Erickson <berickxx@gmail.com>

JBAS-1437 Backstage update authority 005 date values

Signed-off-by: Bill Erickson <berickxx@gmail.com>

JBAS-1437 Bib ignores 'fast' headings.

Avoid any attempts at controlling fields whose thesaurus value (ind2=7
and $2) is 'fast'.

Signed-off-by: Bill Erickson <berickxx@gmail.com>

JBAS-1437 BS search mod bibs once

Signed-off-by: Bill Erickson <berickxx@gmail.com>

JBAS-1437 Backstage 010z replace records

Replace authority records whose 010a matches the 010z value for inbound
records.

Also avoid looking up deleted records when looking for matches.

Signed-off-by: Bill Erickson <berickxx@gmail.com>

JBAS-1437 Backstage quarterly export docs

Signed-off-by: Bill Erickson <berickxx@gmail.com>

JBAS-1437 Backstage process bib/auth file names update

Handle inconsistencies in BS results file naming.

* Process files ending in .UTF8 or .MRC
* Treat file with name AUTH as new/modified authority records.

Signed-off-by: Bill Erickson <berickxx@gmail.com>

JBAS-1437 Auth-2-auth linker --file option

Supports a --file option pointing to file of authority record IDs, one
ID per line.

Signed-off-by: Bill Erickson <berickxx@gmail.com>

JBAS-1437 Backstage record export/import scripts

* export-bibs.pl : export bib records to send to Backstage
* process-backstage-files.pl : process bib/auth records returned from BS
* README.adoc : howto docs

Signed-off-by: Bill Erickson <berickxx@gmail.com>

JBAS-1437 Auth2bib links honors authority 008 #14/#15

Only link 1XX and 7XX bib fields to authority records whose 008 #14
field is 'a'.

Only link 6XX bib fields to authority records whose 008 #15 field is 'a'.

https://www.loc.gov/marc/authority/ad008.html

Signed-off-by: Bill Erickson <berickxx@gmail.com>

JBAS-1437 link-new-auths new options, ID printing

--start-auth-id
--max-auth-count

Also, print-x-ids now prints IDs one per line without any logging
header.

Signed-off-by: Bill Erickson <berickxx@gmail.com>

JBAS-1437 Remove deprecated Backstage export functions

Signed-off-by: Bill Erickson <berickxx@gmail.com>

JBAS-1670 Tukwila teacher cards enabled

Signed-off-by: Bill Erickson <berickxx@gmail.com>

CAT-72 Claims return dialog sanity checks

Avoid the following error when editing hold shelf expire time:

timestamp.js, handle_apply(): TypeError:
    document.getElementById("newRadio1") is null

Signed-off-by: Bill Erickson <berickxx@gmail.com>

JBAS-1377 New headings report min date

Support applying a minimum report date in the new headings report to
prevent queries that report on (practically) all headings.  This happens
when the start date equals or precedes the deployment date for the
browse headings create_date column, which is set to NOW() by default.

Signed-off-by: Bill Erickson <berickxx@gmail.com>

JBAS-1377 date.getYMD() avoids clobbering date

Signed-off-by: Bill Erickson <berickxx@gmail.com>

JBAS-1605 Claims returned dialog label adjustments

Change "Use Default Date" to "Use Due Date".

Signed-off-by: Bill Erickson <berickxx@gmail.com>

JBAS-1459 Authority propagation updates bib editor/date

Cross-porting LP#1588948 - Authority record updates that propagate to
bib records also updates bib editor and edit_date columns.

Signed-off-by: Bill Erickson <berickxx@gmail.com>

JBAS-1638 Patron self-reg cleans spaces

Remove leading / trailing spaces from user-entered values in patron
self-registration app.  In addition to resulting in cleaner data, it
allows the patron editor's duplicate patron search code to work w/ like
values.

Signed-off-by: Bill Erickson <berickxx@gmail.com>

JBAS-1625 Pending patrons create date column

Signed-off-by: Bill Erickson <berickxx@gmail.com>

JBAS-1635 Physical -> Residence addr in patron summary

Changes the label for the address radio selector toggles in the patron
summary.

Signed-off-by: Bill Erickson <berickxx@gmail.com>

CAT-55 Save Archived Message Columns

Port previous commit's changes over to the
archived message's pane.

Signed-off-by: Kyle Huckins <khuckins@catalystdevworks.com>
 Changes to be committed:
modified:   Open-ILS/web/opac/locale/en-US/lang.dtd
modified:   Open-ILS/xul/staff_client/server/patron/standing_penalties.js
modified:   Open-ILS/xul/staff_client/server/patron/standing_penalties.xul

CAT-55 Save Patron Message Columns

Add Locale line for "Save Columns" in patron messages.
Add Save Column command to standing_penalties xul.
Add function to standing_penalties js to handle saving
columns.

Signed-off-by: Kyle Huckins <khuckins@catalystdevworks.com>
 Changes to be committed:
modified:   Open-ILS/web/opac/locale/en-US/lang.dtd
modified:   Open-ILS/xul/staff_client/server/patron/standing_penalties.js
modified:   Open-ILS/xul/staff_client/server/patron/standing_penalties.xul

KMAIN-1828 Full Patron Messages

Add new action on messages: "View Penalty/Message."

Add new xul window to display full note of message in
larger font than displayed on main messages page.

Add localization for new content.

Signed-off-by: Kyle Huckins <khuckins@catalystdevworks.com>
modified:   Open-ILS/web/opac/locale/en-US/lang.dtd
modified:   Open-ILS/xul/staff_client/chrome/content/main/constants.js
modified:   Open-ILS/xul/staff_client/server/patron/standing_penalties.js
modified:   Open-ILS/xul/staff_client/server/patron/standing_penalties.xul
new file:   Open-ILS/xul/staff_client/server/patron/view_standing_penalty.js
new file:   Open-ILS/xul/staff_client/server/patron/view_standing_penalty.xul

CAT-76 Remove author subfield 'e' from TPAC display/links

Avoid displaying 1XX or 7XX subfield 'e' values for authors and prevent
said values from populating auto-generated author search links.

Commit msg modified by Bill for clarity.

Signed-off-by: Billy Horn <bhorn@catalystdevworks.com>

CAT-50 Tighten stop sign page display

* Resize stop sign image to 80x85 from the original 80x113.
* Decrease margin between stop_sign.png and text.
* Remove blank space from bottom of stop_sign.png

Signed-off-by: Kyle Huckins <khuckins@catalystdevworks.com>

CAT-72 Claims returned default date option

Gives staff an option to use a default date for claims returned.

Minor cleanup and doc/comment additions by Bill.

Signed-off-by: Billy Horn <bhorn@catalystdevworks.com>
Signed-off-by: Bill Erickson <berickxx@gmail.com>

JBAS-1604 Vandelay queue delete speedup indexes

Add indexes to make deleting vandelay queue data faster, specifically
when vandelay records link to import items and acq lineitems.

Signed-off-by: Bill Erickson <berickxx@gmail.com>

CAT-77 Renewal override cancel error message repair.

Add check to make sure payload circ has a value.

-- Minor cleanup by Bill.

Signed-off-by: Billy Horn <bhorn@catalystdevworks.com>
Signed-off-by: Bill Erickson <berickxx@gmail.com>

CAT-79 Hide place-hold-for-me staff action

Add visibility: hidden to span containing the target radio

Signed-off-by: Billy Horn <bhorn@catalystdevworks.com>

JBAS-1443 Added content uses EAN / Misc.

Consider values from MARC 024 $a with indicator 1 values of 3 (EAN) or 8
(Misc/Unknown) as valid added content lookup keys.

Signed-off-by: Bill Erickson <berickxx@gmail.com>

JBAS-1415 Apply net_access_level in self-reg

During self-registration, if a patron is less than 17 years old, apply
the "Under 17 Plus" net_access_level to the staged user.  If the patron
is 17 or older, apply "17 and Up Only".

It shouldn't happen in practice since we require a DoB, but if no DoB is
present, no net_access_level value is applied.

Signed-off-by: Bill Erickson <berickxx@gmail.com>

JBAS-1415 Staged users get net_access_level

Signed-off-by: Bill Erickson <berickxx@gmail.com>

JBAS-1150 Record bucket 'deleted' column

Includes new 'deleted' column on 'mvr' class.

Signed-off-by: Bill Erickson <berickxx@gmail.com>

JBAS-905 Reduce copy status checkout alerts

Avoid copy status checkout alerts for these:

4    -- Missing
104  -- Evaluations
107  -- Display
112  -- Cleared
114  -- Workroom
115  -- For Opening
103  -- Story Time

Signed-off-by: Bill Erickson <berickxx@gmail.com>

CAT-69 Volume merge bug

Create function in update_items to handle
find_or_create_volume api call.

Create helper function to match acns and get
the amount of copies in supplied volume in PO.

Create helper function to get number copies from
current volume.

Clean up gather_copies().

Remove instances of direct attempts at modifying
IDL properties, replace with the proper method
calls.

Add helper function to set the acn for the items in the
current PO.

Add FM_ACN_RETRIEVE.authoritative call to find_or_create
helper function.

Set temp_acn's label class, prefix, and suffix
to their object's id within g.find_or_create_volume().

Remove unnecessary loop from g.render_batch_button()

Fix incorrect iteration style to
array-friendly one.

Signed-off-by: Kyle Huckins <khuckins@catalystdevworks.com>
 Changes to be committed:
modified:   Open-ILS/xul/staff_client/server/cat/update_items.js

JBAS-1377 New headings report UI / report view.

Adds a new reporter.cataloged_browse_entry DB view for finding browse
headings based on when their bibs were cataloged.

Adds a UI for searching and displayin such headings.

Signed-off-by: Bill Erickson <berickxx@gmail.com>

JBAS-1377 Date handling utility JS functions

Signed-off-by: Bill Erickson <berickxx@gmail.com>
Conflicts:
Open-ILS/web/js/dojo/openils/Util.js

JBAS-1640 Riverview school/classroom cards

Signed-off-by: Bill Erickson <berickxx@gmail.com>

JBAS-1598 Fed. Way school cards

Signed-off-by: Bill Erickson <berickxx@gmail.com>

JBAS-1643 Teacher districts must be enabled

Before the SFTP client will process a teacher account file, the district
code must be added to a new TEACHER_DISTRICTS configuration list at the
top of the file.

Also, the SFTP client not skips any files not conforming to the correct
csv file name convention.

Signed-off-by: Bill Erickson <berickxx@gmail.com>

JBAS-1660 School teacher cards use upper-case ID's

Signed-off-by: Bill Erickson <berickxx@gmail.com>

JBAS-1643 Teacher cards pad ID to 4 chars min.

Signed-off-by: Bill Erickson <berickxx@gmail.com>

JBAS-1598 Student importer exits on CSV parse error

An unterminated quote, for example, now causes the student generator to
stop processing entirely.

Signed-off-by: Bill Erickson <berickxx@gmail.com>

SQL demo server insert protection: payflow org settings

Signed-off-by: Bill Erickson <berickxx@gmail.com>

OS updates apply autoremove first

Experiment running 'autoremove' before 'dist-upgrade' to avoid occaisonal
problems with botched grub installs leading to un-bootable servers.

Signed-off-by: Bill Erickson <berickxx@gmail.com>

JBAS-1504 Mercer Island classroom card file rename

Signed-off-by: Bill Erickson <berickxx@gmail.com>

JBAS-1504 Mercer Island student/classroom cards

Signed-off-by: Bill Erickson <berickxx@gmail.com>

JBAS-1586 Teacher Ecard uses profile 903

Signed-off-by: Bill Erickson <berickxx@gmail.com>

JBAS-1586 Teacher accounts avoid activity extensions

Avoid applying activity-based expire_date extensions on teacher ecard
accounts, just like student and classroom cards.

Signed-off-by: Bill Erickson <berickxx@gmail.com>

JBAS-1586 Student account gen scripts support --teacher

* Using new Teacher Ecard profile
* 10-year expire
* Default to 1900-01-01 DoB.
* Slightly modified alert message
* Insert 't' between district code and ID in the barcode.

sftp-client-agent.sh treats all CSV files with 'teacher' in the filename
as teacher files.

Signed-off-by: Bill Erickson <berickxx@gmail.com>

JBAS-1586 Teach Ecard new profile

* Entry for permission.grp_tree
* Hold policy configuration to prevent holds for new profile
* Entry in patron editor UI for sorting the new profile in the group selector.

Signed-off-by: Bill Erickson <berickxx@gmail.com>

Copy auditor purge runs 30 mins

Avoid triggering Zabbix alerts about backed replication (and, indeed,
limit clogged replication) by running the copy audit purge for 30 mins
total.  This results in ~3.5M rows deleted per night while still within
the cleanup phase.

Signed-off-by: Bill Erickson <berickxx@gmail.com>

JBAS-1590 Northshore school cards

Signed-off-by: Bill Erickson <berickxx@gmail.com>

JBAS-1544 Payflow xact balance comparison repair

Signed-off-by: Bill Erickson <berickxx@gmail.com>

JBAS-1544 Paypal skip silent post URL option

Org unit setting to avoid sending the Paypal Silent Post URL during
secure token generation.  This is useful if you want to use a specific
server for handling silent posts that may not match the hostname of the
catalog paged accessed by the browser.

Signed-off-by: Bill Erickson <berickxx@gmail.com>

JBAS-1588 CC payment balances match active data

Ensure that the total amount to be tracked as payments in EG after a
successfull PayPal payment match the PP payment amount.  Exit early and
return a non-success to PP when the totals don't match.

This imbalance can occur when a transaction is paid by some other client
(secondary PP instance, staff client) after the PP transaction has been
initiated with the original transaction balance owed amount.

IOW, don't accept payments for transactions that were paid off through
some other mechanism while the PP transaction was in process.

Signed-off-by: Bill Erickson <berickxx@gmail.com>

JBAS-1587 CC always avoids negative transactions

At the middle layer, prevent the credit card transaction retrieval code
from ever accessing negative transactions.  This fixes a bug where
patrons could accidentally attempt to pay against negative transactions
by navigating directly to /eg/opac/biblio/main_payment_form with no
transactions provided in the URL, defaulting to all open (including
negative) transactions.

Signed-off-by: Bill Erickson <berickxx@gmail.com>

EDI retrieval sanity check in undef value

Avoid referencing potentially undef value as an array in the EDI
retrieva code.  Otherwise, all EDI retrieval stopped once the undef
value was encountered.

Signed-off-by: Bill Erickson <berickxx@gmail.com>

JBAS-1593 "Book Club" copy location BC SQL mods

Please create a new shelving location named "Book Club" that will be
non-visible to patrons (as discussed at the ILS Team meeting on 9/15/16).

JBAS-1550 Tukwila school/classroom cards

Signed-off-by: Bill Erickson <berickxx@gmail.com>

JBAS-1501 Audit purge for CRONTAB; minor rearranging

Signed-off-by: Bill Erickson <berickxx@gmail.com>

JBAS-1501 Copy audit table maintenance scripts

New DB function for deleting rows from auditor.asset_copy_history plus a
script we can run from CRON for nightly batches.

Signed-off-by: Bill Erickson <berickxx@gmail.com>

JBAS-1497 Remove CC number cleaner from CRONTAB

We no longer accept any credit card data from patrons, so there's no
data to regularly scrub.

Signed-off-by: Bill Erickson <berickxx@gmail.com>

Payflow org settings SQL insert-on-deploy wrapper

Avoid inserting payflow settings on new dev DB builds -- the data will
come later from an export.

Signed-off-by: Bill Erickson <berickxx@gmail.com>

JBAS-1472 Vashon student / classroom cards

Signed-off-by: Bill Erickson <berickxx@gmail.com>

JBAS-1541 Freegalsip user activity type

Signed-off-by: Bill Erickson <berickxx@gmail.com>

JBAS-1543, add Lucky Day to list of item circ modifiers and shelving locations that should not show in BC

JBAS-1503 Avoid setting SILENTPOSTURL (TLS issues)

Force PP to use evgmobile via PP manager since it is not requiring
the use of TLS 1.2.  In other words, avoid setting the SILENTPOSTURL
from the apache host.

Signed-off-by: Bill Erickson <berickxx@gmail.com>

JBAS-1503 PP fetch in-db user for last_xact_id

During Silent POST handling, fetch the user from the db to pick up the
most recent last_xact_id value for fines payment.

Signed-off-by: Bill Erickson <berickxx@gmail.com>

JBAS-1503 PayPal Layout-A plus Silent POST

Avoid iframe by sending patrons to PP to pay.  PP posts results back to
EG via Silent POST.  Silent POSTs that fail result in a voided payment
on the PP side.

Signed-off-by: Bill Erickson <berickxx@gmail.com>

Revert "JBAS-1503 E-com alert message, stage 1"

This reverts commit 62a68bbfc73c6415f3a975b2bcaabd536754d582.

Revert "JBAS-1503 Hide pay button, update warning"

This reverts commit 6e5f4b39204024e4dc9b16519697254dd6f579a7.

JBAS-1503 Hide pay button, update warning

Signed-off-by: Bill Erickson <berickxx@gmail.com>

JBAS-1503 E-com alert message, stage 1

Signed-off-by: Bill Erickson <berickxx@gmail.com>

JBAS-1494 Updated E-com FAQs link

Signed-off-by: Bill Erickson <berickxx@gmail.com>

JBAS-1494 PayFlow hosted page comment1=barcode

Send the patron's barcode as the transaction comment1 so it will appear
within the PayPal Manager for the business office.

Signed-off-by: Bill Erickson <berickxx@gmail.com>

JBAS-1494 PayFlow Hosted Pages for CC payments

https://developer.paypal.com/docs/classic/payflow/gs_ppa_hosted_pages/

* Library settings to configure and activate.
* New TPAC templates and WWW perl for processing payments.

Signed-off-by: Bill Erickson <berickxx@gmail.com>

JBAS-1413  marc-stream-importer learns authority queue/profile ID's

Signed-off-by: Bill Erickson <berickxx@gmail.com>

JBAS-1390 Connexion authority SQL insert check

Avoid inserts on initial build of sample databases.

Signed-off-by: Bill Erickson <berickxx@gmail.com>

Remove browser warning from Ecom payment form

Signed-off-by: Bill Erickson <berickxx@gmail.com>

JBAS-546 Patron summary shows 'Primary Phone'

... instead of 'Day Phone'

Signed-off-by: Bill Erickson <berickxx@gmail.com>

JBAS-544 Prevent manually setting 'Lost & Paid' status

Disable manual access to the copy Lost & Paid status.  When used, it is
applied by the system, not the staff.

Signed-off-by: Bill Erickson <berickxx@gmail.com>

JBAS-1438 Bib-to-auth linking ignores  (date of work)

Signed-off-by: Bill Erickson <berickxx@gmail.com>

JBAS-1417 Vand. authority import set edit date SQL

Signed-off-by: Bill Erickson <berickxx@gmail.com>

LP#1587639 Vandelay authority update editor/edit_date

Update the editor and edit_date of authority records when modified by
Vandelay merge/overlay.

Signed-off-by: Bill Erickson <berickxx@gmail.com>

JBAS-1417 New authority linker --modified-since

--modified-since implicitly includes --new-since because
edit_date=create_date on new authority records.  However, if both are
specified, the script OR's the queries.

If --modified-since and --new-since are used to gether, it creates an OR
of the 2 clauses.

Added a help() function.  Added CRONTAB entry.

Signed-off-by: Bill Erickson <berickxx@gmail.com>

os-update.sh gets a reboot option

Signed-off-by: Bill Erickson <berickxx@gmail.com>

eg-updater.sh: Apache control repairs/additions

1. If apache is already running when start-apache is called, stop it
first.  This is useful when calling start-apache (or start-all) directly
after a reboot, where Apache may have started automatically at boot.

2. Force a call to 'detach' when stop-apache is called directly as a
safety measure.

Signed-off-by: Bill Erickson <berickxx@gmail.com>