Bib record browser with 'see also', etc from linked authority headings
authorLebbeous Fogle-Weekley <lebbeous@esilibrary.com>
Wed, 24 Apr 2013 14:46:27 +0000 (10:46 -0400)
committerDan Wells <dbw2@calvin.edu>
Fri, 9 Aug 2013 19:02:02 +0000 (15:02 -0400)
This feature provides a patron-oriented OPAC interface for browsing
bibliographic records.

Users choose to browse by Author, Title, Subject, or Series. They then
enter a browse term, and the nearest match from a left-anchored search
on the headings extracted for browse purposes will be displayed in a
typical backwards/forwards paging display. Headings link to search
results pages showing the related records. If the browse heading is
linked to any authority records, and if any *other* authority records
point to those with "See also" or other non-main entry headings, those
alternative headings are displayed a linked to a search results page
showing related bib records related to the alternate heading.

The counts of holdings displayed next to headings from bibliographic
records are subject to the same visiibility tests as search. This means
that the org unit (and copy location group) dropdown on the browse
interface affects counds, and it further means that whether or not
you're looking at the browse interface through the staff client makes a
difference.

This builds on the two previous commits that provide inter-authority
linking and the linking of metabib.browse_entry rows to authority
records.

This also contains, in squashed form, these commits that resulted from
collaboration on LP #1177810:

    Two bugfixes to OPAC Browse: non-filing indicators, leading-article warning
    Fix paste-o encountered in Bib browse upgrade script
    OPAC Browse: fix 0-9 link in paging shortcuts; padding issues
    OPAC Browse: Improve authority code to show more headings
    OPAC Browse: Fix authority counting
    Extensions to our MODS32 stylesheet to capture all possible NFI in fields
    OPAC Browse: i18n improvement for short terms
    OPAC Browser: Display Public General Notes from authorities when possible
    OPAC Browse: Build browse entry sort_value column separately from value
    OPAC Browse: We don't want role/relator info in browse headings
    OPAC Browse: Better display of tracings from authorities
    OPAC Browse: Pick up authority links from 650 fields
    OPAC Browse: Show authority tracings only when inter-authority links exist
    OPAC Browse: use  superpage concept for performance; fix other counting bug
    OPAC Browse: Fix broken authority reference link

Signed-off-by: Lebbeous Fogle-Weekley <lebbeous@esilibrary.com>
Signed-off-by: Kathy Lussier <klussier@masslnc.org>
Signed-off-by: Jason Stephenson <jstephenson@mvlc.org>
22 files changed:
Open-ILS/examples/fm_IDL.xml
Open-ILS/src/perlmods/lib/OpenILS/Application/AppUtils.pm
Open-ILS/src/perlmods/lib/OpenILS/Application/Storage/Driver/Pg/QueryParser.pm
Open-ILS/src/perlmods/lib/OpenILS/WWW/EGCatLoader.pm
Open-ILS/src/perlmods/lib/OpenILS/WWW/EGCatLoader/Browse.pm [new file with mode: 0644]
Open-ILS/src/perlmods/lib/OpenILS/WWW/EGCatLoader/Util.pm
Open-ILS/src/sql/Pg/002.schema.config.sql
Open-ILS/src/sql/Pg/011.schema.authority.sql
Open-ILS/src/sql/Pg/030.schema.metabib.sql
Open-ILS/src/sql/Pg/950.data.seed-values.sql
Open-ILS/src/sql/Pg/953.data.MODS32-xsl.sql
Open-ILS/src/sql/Pg/999.functions.global.sql
Open-ILS/src/sql/Pg/upgrade/XXXX.schema.config-metabib-interauthority.sql
Open-ILS/src/sql/Pg/upgrade/YYYY.schema.bib-auth-browse.sql
Open-ILS/src/templates/opac/advanced.tt2
Open-ILS/src/templates/opac/browse.tt2 [new file with mode: 0644]
Open-ILS/src/templates/opac/css/style.css.tt2
Open-ILS/src/templates/opac/parts/config.tt2
Open-ILS/src/templates/opac/parts/qtype_selector.tt2
Open-ILS/src/templates/opac/parts/searchbar.tt2
Open-ILS/web/css/skin/default/opac/semiauto.css
docs/RELEASE_NOTES_NEXT/OPAC/BibAuthBrowse.txt [new file with mode: 0644]

index a439d6d..f7843ba 100644 (file)
@@ -2116,6 +2116,7 @@ Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA  02110-1301, USA
                        <field reporter:label="Control Set" name="control_set" reporter:datatype="link"/>
                        <field reporter:label="Tag" name="tag" reporter:datatype="text" oils_obj:required="true" oils_obj:validate="^.{3}$"/>
                        <field reporter:label="Subfield List" name="sf_list" reporter:datatype="text" />
+                       <field reporter:label="Subfield List for Display" name="display_sf_list" reporter:datatype="text" />
                        <field reporter:label="Non-filing Indicator" name="nfi" reporter:datatype="text" />
                        <field reporter:label="Name" name="name" reporter:datatype="text" oils_persist:i18n="true" oils_obj:required="true" />
                        <field reporter:label="Description" name="description" reporter:datatype="text" oils_persist:i18n="true" />
@@ -5852,6 +5853,19 @@ SELECT  usr,
                        <link field="record" reltype="has_a" key="id" map="" class="are"/>
                </links>
        </class>
+       <class id="aalink" controller="open-ils.cstore" oils_obj:fieldmapper="authority::authority_linking" oils_persist:tablename="authority.authority_linking" reporter:label="Authority to Authority Linking">
+               <fields oils_persist:primary="id" oils_persist:sequence="authority.authority_linking_id_seq">
+                       <field name="id" reporter:label="ID" reporter:datatype="id" />
+                       <field name="source" reporter:label="Source Record" reporter:datatype="link" />
+                       <field name="target" reporter:label="Target Record" reporter:datatype="link" />
+                       <field name="field" reporter:label="Authority Field" reporter:datatype="link" />
+               </fields>
+               <links>
+                       <link field="source" reltype="has_a" key="id" map="" class="are"/>
+                       <link field="target" reltype="has_a" key="id" map="" class="are"/>
+                       <link field="field" reltype="has_a" key="id" map="" class="acsaf"/>
+               </links>
+       </class>
        <class id="cnct" controller="open-ils.cstore" oils_obj:fieldmapper="config::non_cataloged_type" oils_persist:tablename="config.non_cataloged_type" reporter:label="Non-cataloged Type">
                <fields oils_persist:primary="id" oils_persist:sequence="config.non_cataloged_type_id_seq">
                        <field reporter:label="Circulation Duration" name="circ_duration" reporter:datatype="interval"/>
index 82644f9..92f1daf 100644 (file)
@@ -17,6 +17,7 @@ use UUID::Tiny;
 use Encode;
 use DateTime;
 use DateTime::Format::ISO8601;
+use List::MoreUtils qw/uniq/;
 
 # ---------------------------------------------------------------------------
 # Pile of utilty methods used accross applications.
@@ -2120,6 +2121,22 @@ sub strip_marc_fields {
     return $class->entityize($marcdoc->documentElement->toString);
 }
 
+# Given a list of PostgreSQL arrays of numbers,
+# unnest the numbers and return a unique set, skipping any list elements
+# that are just '{NULL}'.
+sub unique_unnested_numbers {
+    my $class = shift;
+
+    no warnings 'numeric';
+
+    return uniq(
+        map(
+            int,
+            map { $_ eq 'NULL' ? undef : (split /,/, $_) }
+                map { substr($_, 1, -1) } @_
+        )
+    );
+}
 
 1;
 
index 431e26f..544dba9 100644 (file)
@@ -666,6 +666,8 @@ __PACKAGE__->add_search_filter( 'container' );
 # Start from a list of record ids, either bre or metarecords, depending on the #metabib modifier
 __PACKAGE__->add_search_filter( 'record_list' );
 
+__PACKAGE__->add_search_filter( 'has_browse_entry' );
+
 # used internally, but generally not user-settable
 __PACKAGE__->add_search_filter( 'preferred_language' );
 __PACKAGE__->add_search_filter( 'preferred_language_weight' );
@@ -1110,6 +1112,12 @@ sub flatten {
                     $where .= "$key ${NOT}IN (" . join(',', map { $self->QueryParser->quote_value($_) } @{$filter->args}) . ')';
                 }
 
+            } elsif ($filter->name eq 'has_browse_entry') {
+                if (@{$filter->args} >= 2) {
+                    my $entry = int(shift @{$filter->args});
+                    my $fields = join(",", map(int, @{$filter->args}));
+                    $from .= "\n" . $spc x 3 . sprintf("INNER JOIN metabib.browse_entry_def_map mbedm ON (mbedm.source = m.source AND mbedm.entry = %d AND mbedm.def IN (%s))", $entry, $fields);
+                }
             } elsif ($filter->name eq 'edit_date' or $filter->name eq 'create_date') {
                 # bre.create_date and bre.edit_date filtering
                 my $datefilter = $filter->name;
index 0838a29..244afa2 100644 (file)
@@ -19,6 +19,7 @@ use Time::HiRes;
 # EGCatLoader sub-modules 
 use OpenILS::WWW::EGCatLoader::Util;
 use OpenILS::WWW::EGCatLoader::Account;
+use OpenILS::WWW::EGCatLoader::Browse;
 use OpenILS::WWW::EGCatLoader::Search;
 use OpenILS::WWW::EGCatLoader::Record;
 use OpenILS::WWW::EGCatLoader::Container;
@@ -123,6 +124,7 @@ sub load {
     return $self->load_print_record if $path =~ m|opac/record/print|;
     return $self->load_record if $path =~ m|opac/record/\d|;
     return $self->load_cnbrowse if $path =~ m|opac/cnbrowse|;
+    return $self->load_browse if $path =~ m|opac/browse|;
 
     return $self->load_mylist_add if $path =~ m|opac/mylist/add|;
     return $self->load_mylist_delete if $path =~ m|opac/mylist/delete|;
diff --git a/Open-ILS/src/perlmods/lib/OpenILS/WWW/EGCatLoader/Browse.pm b/Open-ILS/src/perlmods/lib/OpenILS/WWW/EGCatLoader/Browse.pm
new file mode 100644 (file)
index 0000000..cc9d73c
--- /dev/null
@@ -0,0 +1,424 @@
+package OpenILS::WWW::EGCatLoader;
+
+use strict;
+use warnings;
+
+use OpenSRF::Utils::Logger qw/$logger/;
+use OpenILS::Utils::CStoreEditor qw/:funcs/;
+use OpenILS::Utils::Fieldmapper;
+use OpenILS::Utils::Normalize qw/search_normalize/;
+use OpenILS::Application::AppUtils;
+use OpenSRF::Utils::JSON;
+use OpenSRF::Utils::Cache;
+use OpenSRF::Utils::SettingsClient;
+
+use Digest::MD5 qw/md5_hex/;
+use Apache2::Const -compile => qw/OK/;
+use MARC::Record;
+#use Data::Dumper;
+#$Data::Dumper::Indent = 0;
+
+my $U = 'OpenILS::Application::AppUtils';
+my $browse_cache;
+my $browse_timeout;
+
+# Plain procedural functions start here.
+#
+sub _init_browse_cache {
+    if (not defined $browse_cache) {
+        my $conf = new OpenSRF::Utils::SettingsClient;
+
+        $browse_timeout = $conf->config_value(
+            "apps", "open-ils.search", "app_settings", "cache_timeout"
+        ) || 300;
+        $browse_cache = new OpenSRF::Utils::Cache("global");
+    }
+}
+
+sub _get_authority_heading {
+    my ($field, $sf_lookup) = @_;
+
+    return join(
+        " ",
+        map { $_->[1] } grep { $sf_lookup->{$_->[0]} } $field->subfields
+    );
+}
+
+# Object methods start here.
+#
+
+# Returns cache key and a list of parameters for DB proc metabib.browse().
+sub prepare_browse_parameters {
+    my ($self) = @_;
+
+    no warnings 'uninitialized';
+
+    # XXX TODO add config.global_flag rows for browse limit-limit and
+    # browse offset-limit?
+
+    my $limit = int($self->cgi->param('blimit') || 10);
+    my $offset = int($self->cgi->param('boffset') || 0);
+    my $force_backward = scalar($self->cgi->param('bback'));
+
+    my @params = (
+        scalar($self->cgi->param('qtype')),
+        scalar($self->cgi->param('bterm')),
+        $self->ctx->{copy_location_group_org} ||
+            $self->ctx->{aou_tree}->()->id,
+        $self->ctx->{copy_location_group},
+        $self->ctx->{is_staff} ? 't' : 'f',
+        scalar($self->cgi->param('bpivot')),
+        $force_backward ? 't' : 'f'
+    );
+
+    # We do need $limit, $offset, and $force_backward as part of the
+    # cache key, but we also need to keep them separate from other
+    # parameters for purposes of paging link generation.
+    return (
+        "oils_browse_" . md5_hex(
+            OpenSRF::Utils::JSON->perl2JSON(
+                [@params, $limit, $offset, $force_backward]
+            )
+        ),
+        $limit, $offset, $force_backward, @params
+    );
+}
+
+# Break out any Public General Notes (field 680) for display and
+# hyperlinking. These are sometimes (erroneously?) called "scope notes."
+# I say erroneously, tentatively, because LoC doesn't seem to document
+# a "scope notes" field for authority records, while it does so for
+# classification records, which are something else. But I am not a
+# librarian.
+sub extract_public_general_notes {
+    my ($self, $record, $row) = @_;
+
+    my @notes;
+    foreach my $note ($record->field('680')) {
+        my @note;
+        my $last_heading;
+
+        foreach my $subfield ($note->subfields) {
+            my ($code, $value) = @$subfield;
+
+            if ($code eq 'i') {
+                push @note, $value;
+            } elsif ($code eq '5') {
+                if ($last_heading) {
+                    my $org = $self->ctx->{get_aou_by_shortname}->($value);
+                    $last_heading->{org_id} = $org->id if $org;
+                }
+                push @note, { institution => $value };
+            } elsif ($code eq 'a') {
+                $last_heading = {
+                    heading => $value, bterm => search_normalize($value)
+                };
+                push @note, $last_heading;
+            }
+        }
+
+        push @notes, \@note if @note;
+    }
+
+    $row->{notes} = \@notes;
+}
+
+sub find_authority_headings_and_notes {
+    my ($self, $row) = @_;
+
+    my $acsaf_table =
+        $self->ctx->{get_authority_fields}->($row->{control_set});
+
+    $row->{headings} = [];
+
+    my $record;
+    eval {
+        $record = new_from_xml MARC::Record($row->{marc});
+    };
+    if ($@) {
+        $logger->warn("Problem with MARC from authority record #" .
+            $row->{id} . ": $@");
+        return $row;    # We're called in map(), so we must move on without
+                        # a fuss.
+    }
+
+    $self->extract_public_general_notes($record, $row);
+
+    # By applying grep in this way, we get acsaf objects that *have* and
+    # therefore *aren't* main entries, which is what we want.
+    foreach my $acsaf (grep { $_->main_entry } values(%$acsaf_table)) {
+        my @fields = $record->field($acsaf->tag);
+        my %sf_lookup = map { $_ => 1 } split("", $acsaf->display_sf_list);
+        my @headings;
+
+        foreach my $field (@fields) {
+            my $h = { heading => _get_authority_heading($field, \%sf_lookup) };
+
+            # XXX I was getting "target" from authority.authority_linking, but
+            # that makes no sense: that table can only tell you that one
+            # authority record as a whole points at another record.  It does
+            # not record when a specific *field* in one authority record
+            # points to another record (not that it makes much sense for
+            # one authority record to have links to multiple others, but I can't
+            # say there definitely aren't cases for that).
+            $h->{target} = $2
+                if ($field->subfield('0') || "") =~ /(^|\))(\d+)$/;
+
+            push @headings, $h;
+        }
+
+        push @{$row->{headings}}, {$acsaf->id => \@headings} if @headings;
+    }
+
+    return $row;
+}
+
+sub map_authority_headings_to_results {
+    my ($self, $linked, $results, $auth_ids) = @_;
+
+    # Use the linked authority records' control sets to find and pick
+    # out non-main-entry headings. Build the headings and make a
+    # combined data structure for the template's use.
+    my %linked_headings_by_auth_id = map {
+        $_->{id} => $self->find_authority_headings_and_notes($_)
+    } @$linked;
+
+    # Graft this authority heading data onto our main result set at the
+    # "authorities" column.
+    foreach my $row (@$results) {
+        $row->{authorities} = [
+            map { $linked_headings_by_auth_id{$_} } @{$row->{authorities}}
+        ];
+    }
+
+    # Get linked-bib counts for each of those authorities, and put THAT
+    # information into place in the data structure.
+    my $counts = $self->editor->json_query({
+        select => {
+            abl => [
+                {column => "id", transform => "count",
+                    alias => "count", aggregate => 1},
+                "authority"
+            ]
+        },
+        from => {abl => {}},
+        where => {
+            "+abl" => {
+                authority => [
+                    @$auth_ids,
+                    $U->unique_unnested_numbers(map { $_->{target} } @$linked)
+                ]
+            }
+        }
+    }) or return;
+
+    my %auth_counts = map { $_->{authority} => $_->{count} } @$counts;
+
+    # Soooo nesty!  We look for places where we'll need a count of bibs
+    # linked to an authority record, and put it there for the template to find.
+    for my $row (@$results) {
+        for my $auth (@{$row->{authorities}}) {
+            if ($auth->{headings}) {
+                for my $outer_heading (@{$auth->{headings}}) {
+                    for my $heading_blob (@{(values %$outer_heading)[0]}) {
+                        if ($heading_blob->{target}) {
+                            $heading_blob->{target_count} =
+                                $auth_counts{$heading_blob->{target}};
+                        }
+                    }
+                }
+            }
+        }
+    }
+}
+
+# flesh_browse_results() attaches data from authority records. It
+# changes $results and returns 1 for success, undef for failure (in which
+# case $self->editor->event should always point to the reason for failure).
+# $results must be an arrayref of result rows from the DB's metabib.browse()
+sub flesh_browse_results {
+    my ($self, $results) = @_;
+
+    # Turn comma-seprated strings of numbers in "authorities" column
+    # into arrays.
+    $_->{authorities} = [split /,/, $_->{authorities}] foreach @$results;
+
+    # Group them in one arrray, not worrying about dupes because we're about
+    # to use them in an IN () comparison in a SQL query.
+    my @auth_ids = map { @{$_->{authorities}} } @$results;
+
+    if (@auth_ids) {
+        # Get all linked authority records themselves
+        my $linked = $self->editor->json_query({
+            select => {
+                are => [qw/id marc control_set/],
+                aalink => [{column => "target", transform => "array_agg",
+                    aggregate => 1}]
+            },
+            from => {
+                are => {
+                    aalink => {
+                        type => "left",
+                        fkey => "id", field => "source"
+                    }
+                }
+            },
+            where => {"+are" => {id => \@auth_ids}}
+        }) or return;
+
+        $self->map_authority_headings_to_results($linked, $results, \@auth_ids);
+    }
+
+    return 1;
+}
+
+sub load_browse_impl {
+    my ($self, $limit, $offset, $force_backward, @params) = @_;
+
+    my $inner_limit = ($offset >= 0 and not $force_backward) ?
+        $limit + 1 : $limit;
+
+    my $results = $self->editor->json_query({
+        from => [
+            "metabib.browse", (@params, $inner_limit, $offset)
+        ]
+    });
+
+    if (not $results) {  # DB error, not empty result set.
+        $logger->warn(
+            "error in browse (direct): " . $self->editor->event->{textcode}
+        );
+        $self->ctx->{browse_error} = 1;
+
+        return;
+    } elsif (not $self->flesh_browse_results($results)) {
+        $logger->warn(
+            "error in browse (flesh): " . $self->editor->event->{textcode}
+        );
+        $self->ctx->{browse_error} = 1;
+
+        return;
+    }
+
+    return $results;
+}
+
+# $results can be modified by this function.  This would be simpler
+# but for the moving pivot concept that helps us avoid paging with
+# large offsets (slow).
+sub infer_browse_paging {
+    my ($self, $results, $limit, $offset, $force_backward) = @_;
+
+    # (All these comments assume a default limit of 10).  For typical
+    # not-backwards requests not at the end of the result set, we
+    # should have an eleventh result that tells us what's next.
+    while (scalar @$results > $limit) {
+        $self->ctx->{forward_pivot} = (pop @$results)->{browse_entry};
+        $self->ctx->{more_forward} = 1;
+    }
+
+    # If we're going backwards by pivot id, we don't have an eleventh
+    # result to tell us we can page forward, but we can assume we can
+    # go forward because duh, we followed a link backward to get here.
+    if ($force_backward and $self->cgi->param('bpivot')) {
+        $self->ctx->{forward_pivot} = scalar($self->cgi->param('bpivot'));
+        $self->ctx->{more_forward} = 1;
+    }
+
+    # The pivot that the user can use for going backwards is the first
+    # of the result set.
+    if (@$results) {
+        $self->ctx->{back_pivot} = $results->[0]->{browse_entry};
+    }
+
+    # The result of these tests relate to basic limit/offset paging.
+
+    # This comparison for setting more_forward does not fold into
+    # those for setting more_back.
+    if ($offset < 0 || $force_backward) {
+        $self->ctx->{more_forward} = 1;
+    }
+
+    if ($offset > 0 or (!$force_backward and $self->cgi->param('bpivot'))) {
+        $self->ctx->{more_back} = 1;
+    } elsif (scalar @$results < $limit) {
+        $self->ctx->{more_back} = 0;
+    } elsif (not($self->cgi->param('bterm') eq '0' and
+        not defined $self->cgi->param('bpivot'))) {   # paging links
+        $self->ctx->{more_back} = 1;
+    }
+}
+
+sub leading_article_test {
+    my ($self, $qtype, $bterm) = @_;
+
+    my $flag_name = "opac.browse.warnable_regexp_per_class";
+    my $flag = $self->ctx->{get_cgf}->($flag_name);
+
+    return unless $flag->enabled eq 't';
+
+    my $map;
+
+    eval { $map = OpenSRF::Utils::JSON->JSON2perl($flag->value); };
+    if ($@) {
+        $logger->warn("cgf '$flag_name' enabled but value is invalid JSON? $@");
+        return;
+    }
+
+    # Don't crash over any of the things that could go wrong in here:
+    eval {
+        if ($map->{$qtype}) {
+            if ($bterm =~ qr/$map->{$qtype}/i) {
+                $self->ctx->{browse_leading_article_warning} = 1;
+            }
+        }
+    };
+    if ($@) {
+        $logger->warn("cgf '$flag_name' has valid JSON in value, but: $@");
+    }
+}
+
+sub load_browse {
+    my ($self) = @_;
+
+    _init_browse_cache();
+
+    $self->ctx->{more_forward} = 0;
+    $self->ctx->{more_back} = 0;
+
+    if ($self->cgi->param('qtype') and defined $self->cgi->param('bterm')) {
+
+        $self->leading_article_test(
+            $self->cgi->param('qtype'),
+            $self->cgi->param('bterm')
+        );
+
+        my ($cache_key, $limit, $offset, $force_backward, @params) =
+            $self->prepare_browse_parameters;
+
+        my $results = $browse_cache->get_cache($cache_key);
+        if (not $results) {
+            $results = $self->load_browse_impl(
+                $limit, $offset, $force_backward, @params
+            );
+            if ($results) {
+                $browse_cache->put_cache($cache_key, $results, $browse_timeout);
+            }
+        }
+
+        if ($results) {
+            $self->infer_browse_paging(
+                $results, $limit, $offset, $force_backward
+            );
+            $self->ctx->{browse_results} = $results;
+        }
+
+        # We don't need an else clause to send the user a 5XX error or
+        # anything. Errors will have been logged, and $ctx will be
+        # prepared so a template can show a nicer error to the user.
+    }
+
+    return Apache2::Const::OK;
+}
+
+1;
index b8dd4f6..6382210 100644 (file)
@@ -21,7 +21,8 @@ our %cache = ( # cached data
     search_filter_groups => {en_us => {}},
     aou_tree => {en_us => undef},
     aouct_tree => {},
-    eg_cache_hash => undef
+    eg_cache_hash => undef,
+    authority_fields => {en_us => {}}
 );
 
 sub init_ro_object_cache {
@@ -230,6 +231,21 @@ sub init_ro_object_cache {
         return $cache{org_settings}{$ctx->{locale}}{$org_id}{$setting};
     };
 
+    # retrieve and cache acsaf values
+    $ro_object_subs->{get_authority_fields} = sub {
+        my ($control_set) = @_;
+
+        if (not exists $cache{authority_fields}{$ctx->{locale}}{$control_set}) {
+            my $acs = $e->search_authority_control_set_authority_field(
+                {control_set => $control_set}
+            ) or return;
+            $cache{authority_fields}{$ctx->{locale}}{$control_set} =
+                +{ map { $_->id => $_ } @$acs };
+        }
+
+        return $cache{authority_fields}{$ctx->{locale}}{$control_set};
+    };
+
     $ctx->{$_} = $ro_object_subs->{$_} for keys %$ro_object_subs;
 }
 
index 0b6cd8b..5c1e0b1 100644 (file)
@@ -193,7 +193,9 @@ CREATE TABLE config.metabib_field (
        facet_field     BOOL    NOT NULL DEFAULT FALSE,
        browse_field    BOOL    NOT NULL DEFAULT TRUE,
        browse_xpath   TEXT,
+       browse_sort_xpath TEXT,
        facet_xpath     TEXT,
+       authority_xpath TEXT,
        restrict        BOOL    DEFAULT FALSE NOT NULL
 );
 COMMENT ON TABLE config.metabib_field IS $$
index 6d4fb50..3277555 100644 (file)
@@ -35,6 +35,7 @@ CREATE TABLE authority.control_set_authority_field (
     tag         CHAR(3) NOT NULL,
     nfi         CHAR(1),          -- non-filing indicator
     sf_list     TEXT    NOT NULL,
+    display_sf_list     TEXT NOT NULL,
     name        TEXT    NOT NULL, -- i18n
     description TEXT,             -- i18n
     linking_subfield CHAR(1)
index 1cd740d..ae74fba 100644 (file)
@@ -185,9 +185,16 @@ CREATE INDEX metabib_facet_entry_source_idx ON metabib.facet_entry (source);
 
 CREATE TABLE metabib.browse_entry (
     id BIGSERIAL PRIMARY KEY,
-    value TEXT unique,
-    index_vector tsvector
+    value TEXT,
+    index_vector tsvector,
+    sort_value  TEXT NOT NULL,
+    UNIQUE(value, sort_value)
 );
+
+
+CREATE INDEX browse_entry_sort_value_idx
+    ON metabib.browse_entry USING BTREE (sort_value);
+
 CREATE INDEX metabib_browse_entry_index_vector_idx ON metabib.browse_entry USING GIN (index_vector);
 CREATE TRIGGER metabib_browse_entry_fti_trigger
     BEFORE INSERT OR UPDATE ON metabib.browse_entry
@@ -198,7 +205,8 @@ CREATE TABLE metabib.browse_entry_def_map (
     id BIGSERIAL PRIMARY KEY,
     entry BIGINT REFERENCES metabib.browse_entry (id),
     def INT REFERENCES config.metabib_field (id),
-    source BIGINT REFERENCES biblio.record_entry (id)
+    source BIGINT REFERENCES biblio.record_entry (id),
+    authority BIGINT REFERENCES authority.record_entry (id) ON DELETE SET NULL
 );
 CREATE INDEX browse_entry_def_map_def_idx ON metabib.browse_entry_def_map (def);
 CREATE INDEX browse_entry_def_map_entry_idx ON metabib.browse_entry_def_map (entry);
@@ -384,14 +392,15 @@ CREATE INDEX metabib_metarecord_source_map_metarecord_idx ON metabib.metarecord_
 CREATE INDEX metabib_metarecord_source_map_source_record_idx ON metabib.metarecord_source_map (source);
 
 CREATE TYPE metabib.field_entry_template AS (
-        field_class     TEXT,
-        field           INT,
-        facet_field     BOOL,
-        search_field    BOOL,
-        browse_field   BOOL,
-        source          BIGINT,
-        value           TEXT,
-        authority       BIGINT
+    field_class         TEXT,
+    field               INT,
+    facet_field         BOOL,
+    search_field        BOOL,
+    browse_field        BOOL,
+    source              BIGINT,
+    value               TEXT,
+    authority           BIGINT,
+    sort_value          TEXT
 );
 
 
@@ -406,6 +415,7 @@ DECLARE
     xml_node_list   TEXT[];
     facet_text  TEXT;
     browse_text TEXT;
+    sort_value  TEXT;
     raw_text    TEXT;
     curr_text   TEXT;
     joiner      TEXT := default_joiner; -- XXX will index defs supply a joiner?
@@ -479,10 +489,25 @@ BEGIN
                     browse_text := curr_text;
                 END IF;
 
+                IF idx.browse_sort_xpath IS NOT NULL AND
+                    idx.browse_sort_xpath <> '' THEN
+
+                    sort_value := oils_xpath_string(
+                        idx.browse_sort_xpath, xml_node, joiner,
+                        ARRAY[ARRAY[xfrm.prefix, xfrm.namespace_uri]]
+                    );
+                ELSE
+                    sort_value := browse_text;
+                END IF;
+
                 output_row.field_class = idx.field_class;
                 output_row.field = idx.id;
                 output_row.source = rid;
                 output_row.value = BTRIM(REGEXP_REPLACE(browse_text, E'\\s+', ' ', 'g'));
+                output_row.sort_value :=
+                    public.search_normalize(sort_value);
+
+                output_row.authority := NULL;
 
                 IF idx.authority_xpath IS NOT NULL AND idx.authority_xpath <> '' THEN
                     authority_text := oils_xpath_string(
@@ -506,6 +531,7 @@ BEGIN
                 output_row.browse_field = TRUE;
                 RETURN NEXT output_row;
                 output_row.browse_field = FALSE;
+                output_row.sort_value := NULL;
             END IF;
 
             -- insert raw node text for faceting
@@ -611,6 +637,7 @@ DECLARE
     b_skip_facet    BOOL;
     b_skip_browse   BOOL;
     b_skip_search   BOOL;
+    value_prepped   TEXT;
 BEGIN
 
     SELECT COALESCE(NULLIF(skip_facet, FALSE), EXISTS (SELECT enabled FROM config.internal_flag WHERE name =  'ingest.skip_facet_indexing' AND enabled)) INTO b_skip_facet;
@@ -650,12 +677,18 @@ BEGIN
             -- evergreen.oils_tsearch2()) changes.  It may or may not be
             -- expensive to add a comparison of index_vector to index_vector
             -- to the WHERE clause below.
-            SELECT INTO mbe_row * FROM metabib.browse_entry WHERE value = ind_data.value;
+
+            value_prepped := metabib.browse_normalize(ind_data.value, ind_data.field);
+            SELECT INTO mbe_row * FROM metabib.browse_entry
+                WHERE value = value_prepped AND sort_value = ind_data.sort_value;
+
             IF FOUND THEN
                 mbe_id := mbe_row.id;
             ELSE
-                INSERT INTO metabib.browse_entry (value) VALUES
-                    (metabib.browse_normalize(ind_data.value, ind_data.field));
+                INSERT INTO metabib.browse_entry
+                    ( value, sort_value ) VALUES
+                    ( value_prepped, ind_data.sort_value );
+
                 mbe_id := CURRVAL('metabib.browse_entry_id_seq'::REGCLASS);
             END IF;
 
@@ -1740,4 +1773,243 @@ BEGIN
 END;
 $$ LANGUAGE PLPGSQL;
 
+
+CREATE TYPE metabib.flat_browse_entry_appearance AS (
+    browse_entry    BIGINT,
+    value           TEXT,
+    fields          TEXT,
+    authorities     TEXT,
+    sources         INT,        -- visible ones, that is
+    row_number      INT,        -- internal use, sort of
+    accurate        BOOL        -- Count in sources field is accurate? Not
+                                -- if we had more than a browse superpage
+                                -- of records to look at.
+);
+
+
+CREATE OR REPLACE FUNCTION metabib.browse_pivot(
+    search_field        INT[],
+    browse_term         TEXT
+) RETURNS BIGINT AS $p$
+DECLARE
+    id                  BIGINT;
+BEGIN
+    SELECT INTO id mbe.id FROM metabib.browse_entry mbe
+        JOIN metabib.browse_entry_def_map mbedm ON (
+            mbedm.entry = mbe.id AND
+            mbedm.def = ANY(search_field)
+        )
+        WHERE mbe.sort_value >= public.search_normalize(browse_term)
+        ORDER BY mbe.sort_value, mbe.value LIMIT 1;
+
+    RETURN id;
+END;
+$p$ LANGUAGE PLPGSQL;
+
+CREATE OR REPLACE FUNCTION metabib.staged_browse(
+    core_query              TEXT,
+    context_org             INT,
+    context_locations       INT[],
+    staff                   BOOL,
+    browse_superpage_size   INT,
+    result_limit            INT,
+    use_offset              INT
+) RETURNS SETOF metabib.flat_browse_entry_appearance AS $p$
+DECLARE
+    core_cursor             REFCURSOR;
+    core_record             RECORD;
+    qpfts_query             TEXT;
+    result_row              metabib.flat_browse_entry_appearance%ROWTYPE;
+    results_skipped         INT := 0;
+    results_returned        INT := 0;
+    slice_start             INT;
+    slice_end               INT;
+    full_end                INT;
+    superpage_of_records    BIGINT[];
+    superpage_size          INT;
+BEGIN
+    OPEN core_cursor FOR EXECUTE core_query;
+
+    LOOP
+        FETCH core_cursor INTO core_record;
+        EXIT WHEN NOT FOUND;
+
+        result_row.sources := 0;
+
+        full_end := ARRAY_LENGTH(core_record.records, 1);
+        superpage_size := COALESCE(browse_superpage_size, full_end);
+        slice_start := 1;
+        slice_end := superpage_size;
+
+        WHILE result_row.sources = 0 AND slice_start <= full_end LOOP
+            superpage_of_records := core_record.records[slice_start:slice_end];
+            qpfts_query :=
+                'SELECT NULL::BIGINT AS id, ARRAY[r] AS records, ' ||
+                '1::INT AS rel FROM (SELECT UNNEST(' ||
+                quote_literal(superpage_of_records) || '::BIGINT[]) AS r) rr';
+
+            -- We use search.query_parser_fts() for visibility testing.
+            -- We're calling it once per browse-superpage worth of records
+            -- out of the set of records related to a given mbe, until we've
+            -- either exhausted that set of records or found at least 1
+            -- visible record.
+
+            SELECT INTO result_row.sources visible
+                FROM search.query_parser_fts(
+                    context_org, NULL, qpfts_query, NULL,
+                    context_locations, 0, NULL, NULL, FALSE, staff, FALSE
+                ) qpfts
+                WHERE qpfts.rel IS NULL;
+
+            slice_start := slice_start + superpage_size;
+            slice_end := slice_end + superpage_size;
+        END LOOP;
+
+        -- Accurate?  Well, probably.
+        result_row.accurate := browse_superpage_size IS NULL OR
+            browse_superpage_size >= full_end;
+
+        IF result_row.sources > 0 THEN
+            IF results_skipped < use_offset THEN
+                results_skipped := results_skipped + 1;
+                CONTINUE;
+            END IF;
+
+            result_row.browse_entry := core_record.id;
+            result_row.authorities := core_record.authorities;
+            result_row.fields := core_record.fields;
+            result_row.value := core_record.value;
+
+            -- This is needed so our caller can flip it and reverse it.
+            result_row.row_number := results_returned;
+
+            RETURN NEXT result_row;
+
+            results_returned := results_returned + 1;
+
+            EXIT WHEN results_returned >= result_limit;
+        END IF;
+    END LOOP;
+END;
+$p$ LANGUAGE PLPGSQL;
+
+-- This is optimized to be fast for values of result_offset near zero.
+CREATE OR REPLACE FUNCTION metabib.browse(
+    search_field            INT[],
+    browse_term             TEXT,
+    context_org             INT DEFAULT NULL,
+    context_loc_group       INT DEFAULT NULL,
+    staff                   BOOL DEFAULT FALSE,
+    pivot_id                BIGINT DEFAULT NULL,
+    force_backward          BOOL DEFAULT FALSE,
+    result_limit            INT DEFAULT 10,
+    result_offset           INT DEFAULT 0   -- Can be negative!
+) RETURNS SETOF metabib.flat_browse_entry_appearance AS $p$
+DECLARE
+    core_query              TEXT;
+    whole_query             TEXT;
+    pivot_sort_value        TEXT;
+    pivot_sort_fallback     TEXT;
+    context_locations       INT[];
+    use_offset              INT;
+    browse_superpage_size   INT;
+    results_skipped         INT := 0;
+BEGIN
+    IF pivot_id IS NULL THEN
+        pivot_id := metabib.browse_pivot(search_field, browse_term);
+    END IF;
+
+    SELECT INTO pivot_sort_value, pivot_sort_fallback
+        sort_value, value FROM metabib.browse_entry WHERE id = pivot_id;
+
+    IF pivot_sort_value IS NULL THEN
+        RETURN;
+    END IF;
+
+    IF context_loc_group IS NOT NULL THEN
+        SELECT INTO context_locations ARRAY_AGG(location)
+            FROM asset.copy_location_group_map
+            WHERE lgroup = context_loc_group;
+    END IF;
+
+    SELECT INTO browse_superpage_size value     -- NULL ok
+        FROM config.global_flag
+        WHERE enabled AND name = 'opac.browse.holdings_visibility_test_limit';
+
+    core_query := '
+    SELECT
+        mbe.id,
+        mbe.value,
+        mbe.sort_value,
+        (SELECT ARRAY_AGG(src) FROM (
+            SELECT DISTINCT UNNEST(ARRAY_AGG(mbedm.source)) AS src
+        ) ss) AS records,
+        (SELECT ARRAY_TO_STRING(ARRAY_AGG(authority), $$,$$) FROM (
+            SELECT DISTINCT UNNEST(ARRAY_AGG(mbedm.authority)) AS authority
+        ) au) AS authorities,
+        (SELECT ARRAY_TO_STRING(ARRAY_AGG(field), $$,$$) FROM (
+            SELECT DISTINCT UNNEST(ARRAY_AGG(mbedm.def)) AS field
+        ) fi) AS fields
+    FROM metabib.browse_entry mbe
+    JOIN metabib.browse_entry_def_map mbedm ON (
+        mbedm.entry = mbe.id AND
+        mbedm.def = ANY(' || quote_literal(search_field) || ')
+    )
+    WHERE ';
+
+    -- PostgreSQL is not magic. We can't actually pass a negative offset.
+    IF result_offset >= 0 AND NOT force_backward THEN
+        use_offset := result_offset;
+        core_query := core_query ||
+            ' mbe.sort_value >= ' || quote_literal(pivot_sort_value) ||
+        ' GROUP BY 1,2,3 ORDER BY mbe.sort_value, mbe.value ';
+
+        RETURN QUERY SELECT * FROM metabib.staged_browse(
+            core_query, context_org, context_locations,
+            staff, browse_superpage_size, result_limit, use_offset
+        );
+    ELSE
+        -- Part 1 of 2 to deliver what the user wants with a negative offset:
+        core_query := core_query ||
+            ' mbe.sort_value < ' || quote_literal(pivot_sort_value) ||
+        ' GROUP BY 1,2,3 ORDER BY mbe.sort_value DESC, mbe.value DESC ';
+
+        -- Part 2 of 2 to deliver what the user wants with a negative offset:
+        RETURN QUERY SELECT * FROM (SELECT * FROM metabib.staged_browse(
+            core_query, context_org, context_locations,
+            staff, browse_superpage_size, result_limit, use_offset
+        )) sb ORDER BY row_number DESC;
+
+    END IF;
+END;
+$p$ LANGUAGE PLPGSQL;
+
+CREATE OR REPLACE FUNCTION metabib.browse(
+    search_class        TEXT,
+    browse_term         TEXT,
+    context_org         INT DEFAULT NULL,
+    context_loc_group   INT DEFAULT NULL,
+    staff               BOOL DEFAULT FALSE,
+    pivot_id            BIGINT DEFAULT NULL,
+    force_backward      BOOL DEFAULT FALSE,
+    result_limit        INT DEFAULT 10,
+    result_offset       INT DEFAULT 0   -- Can be negative, implying backward!
+) RETURNS SETOF metabib.flat_browse_entry_appearance AS $p$
+BEGIN
+    RETURN QUERY SELECT * FROM metabib.browse(
+        (SELECT COALESCE(ARRAY_AGG(id), ARRAY[]::INT[])
+            FROM config.metabib_field WHERE field_class = search_class),
+        browse_term,
+        context_org,
+        context_loc_group,
+        staff,
+        pivot_id,
+        force_backward,
+        result_limit,
+        result_offset
+    );
+END;
+$p$ LANGUAGE PLPGSQL;
+
+
 COMMIT;
index 1aa746c..2b6772d 100644 (file)
@@ -110,37 +110,37 @@ INSERT INTO config.xml_transform VALUES ( 'mods33', 'http://www.loc.gov/mods/v3'
 INSERT INTO config.xml_transform VALUES ( 'marc21expand880', 'http://www.loc.gov/MARC21/slim', 'marc', '' );
 
 -- Index Definitions
-INSERT INTO config.metabib_field ( id, field_class, name, label, format, xpath, facet_field ) VALUES 
-    (1, 'series', 'seriestitle', oils_i18n_gettext(1, 'Series Title', 'cmf', 'label'), 'mods32', $$//mods32:mods/mods32:relatedItem[@type="series"]/mods32:titleInfo$$, TRUE );
-
-INSERT INTO config.metabib_field ( id, field_class, name, label, format, xpath ) VALUES 
-    (2, 'title', 'abbreviated', oils_i18n_gettext(2, 'Abbreviated Title', 'cmf', 'label'), 'mods32', $$//mods32:mods/mods32:titleInfo[mods32:title and (@type='abbreviated')]$$ );
-INSERT INTO config.metabib_field ( id, field_class, name, label, format, xpath ) VALUES 
-    (3, 'title', 'translated', oils_i18n_gettext(3, 'Translated Title', 'cmf', 'label'), 'mods32', $$//mods32:mods/mods32:titleInfo[mods32:title and (@type='translated')]$$ );
-INSERT INTO config.metabib_field ( id, field_class, name, label, format, xpath ) VALUES 
-    (4, 'title', 'alternative', oils_i18n_gettext(4, 'Alternate Title', 'cmf', 'label'), 'mods32', $$//mods32:mods/mods32:titleInfo[mods32:title and (@type='alternative')]$$ );
-INSERT INTO config.metabib_field ( id, field_class, name, label, format, xpath ) VALUES 
-    (5, 'title', 'uniform', oils_i18n_gettext(5, 'Uniform Title', 'cmf', 'label'), 'mods32', $$//mods32:mods/mods32:titleInfo[mods32:title and (@type='uniform')]$$ );
-INSERT INTO config.metabib_field ( id, field_class, name, label, format, xpath ) VALUES 
-    (6, 'title', 'proper', oils_i18n_gettext(6, 'Title Proper', 'cmf', 'label'), 'mods32', $$//mods32:mods/mods32:titleNonfiling[mods32:title and not (@type)]$$ );
-
-INSERT INTO config.metabib_field ( id, field_class, name, label, format, xpath, facet_xpath, facet_field ) VALUES 
-    (7, 'author', 'corporate', oils_i18n_gettext(7, 'Corporate Author', 'cmf', 'label'), 'mods32', $$//mods32:mods/mods32:name[@type='corporate' and (mods32:role/mods32:roleTerm[text()='creator'] or mods32:role/mods32:roleTerm[text()='aut'] or mods32:role/mods32:roleTerm[text()='cre'])]$$, $$//*[local-name()='namePart']$$, TRUE ); -- /* to fool vim */;
-INSERT INTO config.metabib_field ( id, field_class, name, label, format, xpath, facet_xpath, facet_field ) VALUES 
-    (8, 'author', 'personal', oils_i18n_gettext(8, 'Personal Author', 'cmf', 'label'), 'mods32', $$//mods32:mods/mods32:name[@type='personal' and mods32:role/mods32:roleTerm[text()='creator']]$$, $$//*[local-name()='namePart']$$, TRUE ); -- /* to fool vim */;
-INSERT INTO config.metabib_field ( id, field_class, name, label, format, xpath, facet_xpath, facet_field ) VALUES 
-    (9, 'author', 'conference', oils_i18n_gettext(9, 'Conference Author', 'cmf', 'label'), 'mods32', $$//mods32:mods/mods32:name[@type='conference' and mods32:role/mods32:roleTerm[text()='creator']]$$, $$//*[local-name()='namePart']$$, TRUE ); -- /* to fool vim */;
-INSERT INTO config.metabib_field ( id, field_class, name, label, format, xpath, facet_xpath, facet_field ) VALUES 
-    (10, 'author', 'other', oils_i18n_gettext(10, 'Other Author', 'cmf', 'label'), 'mods32', $$//mods32:mods/mods32:name[@type='personal' and not(mods32:role/mods32:roleTerm[text()='creator'])]$$, $$//*[local-name()='namePart']$$, TRUE ); -- /* to fool vim */;
-
-INSERT INTO config.metabib_field ( id, field_class, name, label, format, xpath, facet_field ) VALUES 
-    (11, 'subject', 'geographic', oils_i18n_gettext(11, 'Geographic Subject', 'cmf', 'label'), 'mods32', $$//mods32:mods/mods32:subject/mods32:geographic$$, TRUE );
-INSERT INTO config.metabib_field ( id, field_class, name, label, format, xpath, facet_xpath, facet_field ) VALUES 
-    (12, 'subject', 'name', oils_i18n_gettext(12, 'Name Subject', 'cmf', 'label'), 'mods32', $$//mods32:mods/mods32:subject/mods32:name$$, $$//*[local-name()='namePart']$$, TRUE ); -- /* to fool vim */;
-INSERT INTO config.metabib_field ( id, field_class, name, label, format, xpath, facet_field ) VALUES 
-    (13, 'subject', 'temporal', oils_i18n_gettext(13, 'Temporal Subject', 'cmf', 'label'), 'mods32', $$//mods32:mods/mods32:subject/mods32:temporal$$, TRUE );
-INSERT INTO config.metabib_field ( id, field_class, name, label, format, xpath, facet_field ) VALUES 
-    (14, 'subject', 'topic', oils_i18n_gettext(14, 'Topic Subject', 'cmf', 'label'), 'mods32', $$//mods32:mods/mods32:subject/mods32:topic$$, TRUE );
+INSERT INTO config.metabib_field ( id, field_class, name, label, format, xpath, facet_field, authority_xpath, browse_sort_xpath ) VALUES 
+    (1, 'series', 'seriestitle', oils_i18n_gettext(1, 'Series Title', 'cmf', 'label'), 'mods32', $$//mods32:mods/mods32:relatedItem[@type="series"]/mods32:titleInfo[@type="nfi"]$$, TRUE, '//@xlink:href', $$*[local-name() != "nonSort"]$$ );
+
+INSERT INTO config.metabib_field ( id, field_class, name, label, format, xpath, authority_xpath ) VALUES 
+    (2, 'title', 'abbreviated', oils_i18n_gettext(2, 'Abbreviated Title', 'cmf', 'label'), 'mods32', $$//mods32:mods/mods32:titleInfo[mods32:title and (@type='abbreviated')]$$, '//@xlink:href' );
+INSERT INTO config.metabib_field ( id, field_class, name, label, format, xpath, authority_xpath, browse_sort_xpath ) VALUES 
+    (3, 'title', 'translated', oils_i18n_gettext(3, 'Translated Title', 'cmf', 'label'), 'mods32', $$//mods32:mods/mods32:titleInfo[mods32:title and (@type='translated-nfi')]$$, '//@xlink:href', $$*[local-name() != "nonSort"]$$ );
+INSERT INTO config.metabib_field ( id, field_class, name, label, format, xpath, authority_xpath, browse_sort_xpath ) VALUES 
+    (4, 'title', 'alternative', oils_i18n_gettext(4, 'Alternate Title', 'cmf', 'label'), 'mods32', $$//mods32:mods/mods32:titleInfo[mods32:title and (@type='alternative-nfi')]$$, '//@xlink:href', $$*[local-name() != "nonSort"]$$ );
+INSERT INTO config.metabib_field ( id, field_class, name, label, format, xpath, authority_xpath, browse_sort_xpath ) VALUES 
+    (5, 'title', 'uniform', oils_i18n_gettext(5, 'Uniform Title', 'cmf', 'label'), 'mods32', $$//mods32:mods/mods32:titleInfo[mods32:title and (@type='uniform-nfi')]$$, '//@xlink:href', $$*[local-name() != "nonSort"]$$ );
+INSERT INTO config.metabib_field ( id, field_class, name, label, format, xpath, authority_xpath, browse_field, browse_sort_xpath ) VALUES 
+    (6, 'title', 'proper', oils_i18n_gettext(6, 'Title Proper', 'cmf', 'label'), 'mods32', $$//mods32:mods/mods32:titleNonfiling[mods32:title and not (@type)]$$, '//@xlink:href', TRUE, $$*[local-name() != "nonSort"]$$ );
+
+INSERT INTO config.metabib_field ( id, field_class, name, label, format, xpath, facet_xpath, facet_field , authority_xpath, browse_xpath) VALUES 
+    (7, 'author', 'corporate', oils_i18n_gettext(7, 'Corporate Author', 'cmf', 'label'), 'mods32', $$//mods32:mods/mods32:name[@type='corporate' and (mods32:role/mods32:roleTerm[text()='creator'] or mods32:role/mods32:roleTerm[text()='aut'] or mods32:role/mods32:roleTerm[text()='cre'])]$$, $$//*[local-name()='namePart']$$, TRUE, '//@xlink:href',$$//*[local-name()='namePart']$$ ); -- /* to fool vim */;
+INSERT INTO config.metabib_field ( id, field_class, name, label, format, xpath, facet_xpath, facet_field, authority_xpath, browse_xpath ) VALUES 
+    (8, 'author', 'personal', oils_i18n_gettext(8, 'Personal Author', 'cmf', 'label'), 'mods32', $$//mods32:mods/mods32:name[@type='personal' and mods32:role/mods32:roleTerm[text()='creator']]$$, $$//*[local-name()='namePart']$$, TRUE, '//@xlink:href',$$//*[local-name()='namePart']$$ ); -- /* to fool vim */;
+INSERT INTO config.metabib_field ( id, field_class, name, label, format, xpath, facet_xpath, facet_field, authority_xpath, browse_xpath ) VALUES 
+    (9, 'author', 'conference', oils_i18n_gettext(9, 'Conference Author', 'cmf', 'label'), 'mods32', $$//mods32:mods/mods32:name[@type='conference' and mods32:role/mods32:roleTerm[text()='creator']]$$, $$//*[local-name()='namePart']$$, TRUE, '//@xlink:href',$$//*[local-name()='namePart']$$ ); -- /* to fool vim */;
+INSERT INTO config.metabib_field ( id, field_class, name, label, format, xpath, facet_xpath, facet_field, authority_xpath, browse_xpath ) VALUES 
+    (10, 'author', 'other', oils_i18n_gettext(10, 'Other Author', 'cmf', 'label'), 'mods32', $$//mods32:mods/mods32:name[@type='personal' and not(mods32:role/mods32:roleTerm[text()='creator'])]$$, $$//*[local-name()='namePart']$$, TRUE, '//@xlink:href',$$//*[local-name()='namePart']$$ ); -- /* to fool vim */;
+
+INSERT INTO config.metabib_field ( id, field_class, name, label, format, xpath, facet_field, authority_xpath ) VALUES 
+    (11, 'subject', 'geographic', oils_i18n_gettext(11, 'Geographic Subject', 'cmf', 'label'), 'mods32', $$//mods32:mods/mods32:subject/mods32:geographic$$, TRUE, '//@xlink:href' );
+INSERT INTO config.metabib_field ( id, field_class, name, label, format, xpath, facet_xpath, facet_field, authority_xpath ) VALUES 
+    (12, 'subject', 'name', oils_i18n_gettext(12, 'Name Subject', 'cmf', 'label'), 'mods32', $$//mods32:mods/mods32:subject/mods32:name$$, $$//*[local-name()='namePart']$$, TRUE, '//@xlink:href' ); -- /* to fool vim */;
+INSERT INTO config.metabib_field ( id, field_class, name, label, format, xpath, facet_field, authority_xpath ) VALUES 
+    (13, 'subject', 'temporal', oils_i18n_gettext(13, 'Temporal Subject', 'cmf', 'label'), 'mods32', $$//mods32:mods/mods32:subject/mods32:temporal$$, TRUE, '//@xlink:href' );
+INSERT INTO config.metabib_field ( id, field_class, name, label, format, xpath, facet_field, authority_xpath ) VALUES 
+    (14, 'subject', 'topic', oils_i18n_gettext(14, 'Topic Subject', 'cmf', 'label'), 'mods32', $$//mods32:mods/mods32:subject/mods32:topic$$, TRUE, '//@xlink:href' );
 --INSERT INTO config.metabib_field ( id, field_class, name, format, xpath ) VALUES 
 --  ( id, field_class, name, xpath ) VALUES ( 'subject', 'genre', 'mods32', $$//mods32:mods/mods32:genre$$ );
 INSERT INTO config.metabib_field ( id, field_class, name, label, format, xpath, browse_field ) VALUES 
@@ -176,6 +176,9 @@ INSERT INTO config.metabib_field ( id, field_class, name, label, format, xpath,
     (29, 'identifier', 'scn', oils_i18n_gettext(29, 'System Control Number', 'cmf', 'label'), 'marcxml', $$//marc:datafield[@tag='035']/marc:subfield[@code="a"]$$, FALSE);
 INSERT INTO config.metabib_field ( id, field_class, name, label, format, xpath, browse_field) VALUES
     (30, 'identifier', 'lccn', oils_i18n_gettext(30, 'LC Control Number', 'cmf', 'label'), 'marcxml', $$//marc:datafield[@tag='010']/marc:subfield[@code="a" or @code='z']$$, FALSE);
+INSERT INTO config.metabib_field ( id, field_class, name, label, xpath, format, search_field, facet_field, browse_field) VALUES
+    (31, 'title', 'browse', oils_i18n_gettext(31, 'Title Proper (Browse)', 'cmf', 'label'), $$//mods32:mods/mods32:titleInfo[not (@type)]/mods32:title$$, 'mods32', FALSE, FALSE, TRUE);
+
 
 SELECT SETVAL('config.metabib_field_id_seq'::TEXT, (SELECT MAX(id) FROM config.metabib_field), TRUE);
 
@@ -9180,6 +9183,30 @@ INSERT INTO config.global_flag (name, label)
         )
     );
 
+INSERT INTO config.global_flag (name, value, enabled, label)
+VALUES (
+    'opac.browse.warnable_regexp_per_class',
+    '{"title": "^(a|the|an)\\s"}',
+    FALSE,
+    oils_i18n_gettext(
+        'opac.browse.warnable_regexp_per_class',
+        'Map of search classes to regular expressions to warn user about leading articles.',
+        'cgf',
+        'label'
+    )
+),
+(
+    'opac.browse.holdings_visibility_test_limit',
+    '100',
+    TRUE,
+    oils_i18n_gettext(
+        'opac.browse.holdings_visibility_test_limit',
+        'Don''t look for more than this number of records with holdings when displaying browse headings with visible record counts.',
+        'cgf',
+        'label'
+    )
+);
+
 INSERT INTO config.usr_setting_type (name,opac_visible,label,description,datatype)
     VALUES (
         'history.circ.retention_age',
@@ -10332,65 +10359,76 @@ INSERT INTO authority.control_set (id, name, description) VALUES (
 );
 
 -- Entries that need to respect an NFI
-INSERT INTO authority.control_set_authority_field (id, control_set, main_entry, tag, sf_list, name, nfi) VALUES
-    (4, 1, NULL, '130', 'adfgklmnoprstvxyz', oils_i18n_gettext('4','Heading -- Uniform Title','acsaf','name'), '2'),
-    (24, 1, 4, '530', 'adfgiklmnoprstvwxyz4', oils_i18n_gettext('24','See Also From Tracing -- Uniform Title','acsaf','name'), '2'),
-    (44, 1, 4, '730', 'adfghklmnoprstvwxyz25', oils_i18n_gettext('44','Established Heading Linking Entry -- Uniform Title','acsaf','name'), '2'),
-    (64, 1, 4, '430', 'adfgiklmnoprstvwxyz4', oils_i18n_gettext('64','See From Tracing -- Uniform Title','acsaf','name'), '2');
+INSERT INTO authority.control_set_authority_field (id, control_set, main_entry, tag, sf_list, display_sf_list, name, nfi) VALUES
+    (4, 1, NULL, '130', 'adfgklmnoprstvxyz', 'adfgklmnoprstvxyz', oils_i18n_gettext('4','Heading -- Uniform Title','acsaf','name'), '2'),
+    (24, 1, 4, '530', 'adfgiklmnoprstvwxyz4', 'adfgiklmnoprstvxyz', oils_i18n_gettext('24','See Also From Tracing -- Uniform Title','acsaf','name'), '2'),
+    (44, 1, 4, '730', 'adfghklmnoprstvwxyz25', 'adfghklmnoprstvxyz', oils_i18n_gettext('44','Established Heading Linking Entry -- Uniform Title','acsaf','name'), '2'),
+    (64, 1, 4, '430', 'adfgiklmnoprstvwxyz4', 'adfgiklmnoprstvxyz', oils_i18n_gettext('64','See From Tracing -- Uniform Title','acsaf','name'), '2');
 
-INSERT INTO authority.control_set_authority_field (id, control_set, main_entry, tag, sf_list, name) VALUES
+INSERT INTO authority.control_set_authority_field (id, control_set, main_entry, tag, sf_list, display_sf_list, name) VALUES
 
 -- Main entries
-    (1, 1, NULL, '100', 'abcdefklmnopqrstvxyz', oils_i18n_gettext('1','Heading -- Personal Name','acsaf','name')),
-    (2, 1, NULL, '110', 'abcdefgklmnoprstvxyz', oils_i18n_gettext('2','Heading -- Corporate Name','acsaf','name')),
-    (3, 1, NULL, '111', 'acdefgklnpqstvxyz', oils_i18n_gettext('3','Heading -- Meeting Name','acsaf','name')),
-    (5, 1, NULL, '150', 'abvxyz', oils_i18n_gettext('5','Heading -- Topical Term','acsaf','name')),
-    (6, 1, NULL, '151', 'avxyz', oils_i18n_gettext('6','Heading -- Geographic Name','acsaf','name')),
-    (7, 1, NULL, '155', 'avxyz', oils_i18n_gettext('7','Heading -- Genre/Form Term','acsaf','name')),
-    (8, 1, NULL, '180', 'vxyz', oils_i18n_gettext('8','Heading -- General Subdivision','acsaf','name')),
-    (9, 1, NULL, '181', 'vxyz', oils_i18n_gettext('9','Heading -- Geographic Subdivision','acsaf','name')),
-    (10, 1, NULL, '182', 'vxyz', oils_i18n_gettext('10','Heading -- Chronological Subdivision','acsaf','name')),
-    (11, 1, NULL, '185', 'vxyz', oils_i18n_gettext('11','Heading -- Form Subdivision','acsaf','name')),
-    (12, 1, NULL, '148', 'avxyz', oils_i18n_gettext('12','Heading -- Chronological Term','acsaf','name')),
+    (1, 1, NULL, '100', 'abcdefklmnopqrstvxyz', 'abcdefklmnopqrstvxyz',
+        oils_i18n_gettext('1','Heading -- Personal Name','acsaf','name')),
+    (2, 1, NULL, '110', 'abcdefgklmnoprstvxyz', 'abcdefgklmnoprstvxyz',
+        oils_i18n_gettext('2','Heading -- Corporate Name','acsaf','name')),
+    (3, 1, NULL, '111', 'acdefgklnpqstvxyz', 'acdefgklnpqstvxyz',
+        oils_i18n_gettext('3','Heading -- Meeting Name','acsaf','name')),
+    (5, 1, NULL, '150', 'abvxyz', 'abvxyz',
+        oils_i18n_gettext('5','Heading -- Topical Term','acsaf','name')),
+    (6, 1, NULL, '151', 'avxyz', 'avxyz',
+        oils_i18n_gettext('6','Heading -- Geographic Name','acsaf','name')),
+    (7, 1, NULL, '155', 'avxyz', 'avxyz',
+        oils_i18n_gettext('7','Heading -- Genre/Form Term','acsaf','name')),
+    (8, 1, NULL, '180', 'vxyz', 'vxyz',
+        oils_i18n_gettext('8','Heading -- General Subdivision','acsaf','name')),
+    (9, 1, NULL, '181', 'vxyz', 'vxyz',
+        oils_i18n_gettext('9','Heading -- Geographic Subdivision','acsaf','name')),
+    (10, 1, NULL, '182', 'vxyz', 'vxyz',
+        oils_i18n_gettext('10','Heading -- Chronological Subdivision','acsaf','name')),
+    (11, 1, NULL, '185', 'vxyz', 'vxyz',
+        oils_i18n_gettext('11','Heading -- Form Subdivision','acsaf','name')),
+    (12, 1, NULL, '148', 'avxyz', 'avxyz',
+        oils_i18n_gettext('12','Heading -- Chronological Term','acsaf','name')),
 
 -- See Also From tracings
-    (21, 1, 1, '500', 'abcdefiklmnopqrstvwxyz4', oils_i18n_gettext('21','See Also From Tracing -- Personal Name','acsaf','name')),
-    (22, 1, 2, '510', 'abcdefgiklmnoprstvwxyz4', oils_i18n_gettext('22','See Also From Tracing -- Corporate Name','acsaf','name')),
-    (23, 1, 3, '511', 'acdefgiklnpqstvwxyz4', oils_i18n_gettext('23','See Also From Tracing -- Meeting Name','acsaf','name')),
-    (25, 1, 5, '550', 'abivwxyz4', oils_i18n_gettext('25','See Also From Tracing -- Topical Term','acsaf','name')),
-    (26, 1, 6, '551', 'aivwxyz4', oils_i18n_gettext('26','See Also From Tracing -- Geographic Name','acsaf','name')),
-    (27, 1, 7, '555', 'aivwxyz4', oils_i18n_gettext('27','See Also From Tracing -- Genre/Form Term','acsaf','name')),
-    (28, 1, 8, '580', 'ivwxyz4', oils_i18n_gettext('28','See Also From Tracing -- General Subdivision','acsaf','name')),
-    (29, 1, 9, '581', 'ivwxyz4', oils_i18n_gettext('29','See Also From Tracing -- Geographic Subdivision','acsaf','name')),
-    (30, 1, 10, '582', 'ivwxyz4', oils_i18n_gettext('30','See Also From Tracing -- Chronological Subdivision','acsaf','name')),
-    (31, 1, 11, '585', 'ivwxyz4', oils_i18n_gettext('31','See Also From Tracing -- Form Subdivision','acsaf','name')),
-    (32, 1, 12, '548', 'aivwxyz4', oils_i18n_gettext('32','See Also From Tracing -- Chronological Term','acsaf','name')),
+    (21, 1, 1, '500', 'abcdefiklmnopqrstvwxyz4', 'abcdefiklmnopqrstvxyz', oils_i18n_gettext('21','See Also From Tracing -- Personal Name','acsaf','name')),
+    (22, 1, 2, '510', 'abcdefgiklmnoprstvwxyz4', 'abcdefgiklmnoprstvxyz', oils_i18n_gettext('22','See Also From Tracing -- Corporate Name','acsaf','name')),
+    (23, 1, 3, '511', 'acdefgiklnpqstvwxyz4', 'acdefgiklnpqstvxyz', oils_i18n_gettext('23','See Also From Tracing -- Meeting Name','acsaf','name')),
+    (25, 1, 5, '550', 'abivwxyz4', 'abivxyz', oils_i18n_gettext('25','See Also From Tracing -- Topical Term','acsaf','name')),
+    (26, 1, 6, '551', 'aivwxyz4', 'aivxyz', oils_i18n_gettext('26','See Also From Tracing -- Geographic Name','acsaf','name')),
+    (27, 1, 7, '555', 'aivwxyz4', 'aivxyz', oils_i18n_gettext('27','See Also From Tracing -- Genre/Form Term','acsaf','name')),
+    (28, 1, 8, '580', 'ivwxyz4', 'ivxyz', oils_i18n_gettext('28','See Also From Tracing -- General Subdivision','acsaf','name')),
+    (29, 1, 9, '581', 'ivwxyz4', 'ivxyz', oils_i18n_gettext('29','See Also From Tracing -- Geographic Subdivision','acsaf','name')),
+    (30, 1, 10, '582', 'ivwxyz4', 'ivxyz', oils_i18n_gettext('30','See Also From Tracing -- Chronological Subdivision','acsaf','name')),
+    (31, 1, 11, '585', 'ivwxyz4', 'ivxyz', oils_i18n_gettext('31','See Also From Tracing -- Form Subdivision','acsaf','name')),
+    (32, 1, 12, '548', 'aivwxyz4', 'aivxyz', oils_i18n_gettext('32','See Also From Tracing -- Chronological Term','acsaf','name')),
 
 -- Linking entries
-    (41, 1, 1, '700', 'abcdefghjklmnopqrstvwxyz25', oils_i18n_gettext('41','Established Heading Linking Entry -- Personal Name','acsaf','name')),
-    (42, 1, 2, '710', 'abcdefghklmnoprstvwxyz25', oils_i18n_gettext('42','Established Heading Linking Entry -- Corporate Name','acsaf','name')),
-    (43, 1, 3, '711', 'acdefghklnpqstvwxyz25', oils_i18n_gettext('43','Established Heading Linking Entry -- Meeting Name','acsaf','name')),
-    (45, 1, 5, '750', 'abvwxyz25', oils_i18n_gettext('45','Established Heading Linking Entry -- Topical Term','acsaf','name')),
-    (46, 1, 6, '751', 'avwxyz25', oils_i18n_gettext('46','Established Heading Linking Entry -- Geographic Name','acsaf','name')),
-    (47, 1, 7, '755', 'avwxyz25', oils_i18n_gettext('47','Established Heading Linking Entry -- Genre/Form Term','acsaf','name')),
-    (48, 1, 8, '780', 'vwxyz25', oils_i18n_gettext('48','Subdivision Linking Entry -- General Subdivision','acsaf','name')),
-    (49, 1, 9, '781', 'vwxyz25', oils_i18n_gettext('49','Subdivision Linking Entry -- Geographic Subdivision','acsaf','name')),
-    (50, 1, 10, '782', 'vwxyz25', oils_i18n_gettext('50','Subdivision Linking Entry -- Chronological Subdivision','acsaf','name')),
-    (51, 1, 11, '785', 'vwxyz25', oils_i18n_gettext('51','Subdivision Linking Entry -- Form Subdivision','acsaf','name')),
-    (52, 1, 12, '748', 'avwxyz25', oils_i18n_gettext('52','Established Heading Linking Entry -- Chronological Term','acsaf','name')),
+    (41, 1, 1, '700', 'abcdefghjklmnopqrstvwxyz25', 'abcdefghjklmnopqrstvxyz', oils_i18n_gettext('41','Established Heading Linking Entry -- Personal Name','acsaf','name')),
+    (42, 1, 2, '710', 'abcdefghklmnoprstvwxyz25', 'abcdefghklmnoprstvxyz', oils_i18n_gettext('42','Established Heading Linking Entry -- Corporate Name','acsaf','name')),
+    (43, 1, 3, '711', 'acdefghklnpqstvwxyz25', 'acdefghklnpqstvxyz', oils_i18n_gettext('43','Established Heading Linking Entry -- Meeting Name','acsaf','name')),
+    (45, 1, 5, '750', 'abvwxyz25', 'abvxyz', oils_i18n_gettext('45','Established Heading Linking Entry -- Topical Term','acsaf','name')),
+    (46, 1, 6, '751', 'avwxyz25', 'avxyz', oils_i18n_gettext('46','Established Heading Linking Entry -- Geographic Name','acsaf','name')),
+    (47, 1, 7, '755', 'avwxyz25', 'avxyz', oils_i18n_gettext('47','Established Heading Linking Entry -- Genre/Form Term','acsaf','name')),
+    (48, 1, 8, '780', 'vwxyz25', 'vxyz', oils_i18n_gettext('48','Subdivision Linking Entry -- General Subdivision','acsaf','name')),
+    (49, 1, 9, '781', 'vwxyz25', 'vxyz', oils_i18n_gettext('49','Subdivision Linking Entry -- Geographic Subdivision','acsaf','name')),
+    (50, 1, 10, '782', 'vwxyz25', 'vxyz', oils_i18n_gettext('50','Subdivision Linking Entry -- Chronological Subdivision','acsaf','name')),
+    (51, 1, 11, '785', 'vwxyz25', 'vxyz', oils_i18n_gettext('51','Subdivision Linking Entry -- Form Subdivision','acsaf','name')),
+    (52, 1, 12, '748', 'avwxyz25', 'avxyz', oils_i18n_gettext('52','Established Heading Linking Entry -- Chronological Term','acsaf','name')),
 
 -- See From tracings
-    (61, 1, 1, '400', 'abcdefiklmnopqrstvwxyz4', oils_i18n_gettext('61','See From Tracing -- Personal Name','acsaf','name')),
-    (62, 1, 2, '410', 'abcdefgiklmnoprstvwxyz4', oils_i18n_gettext('62','See From Tracing -- Corporate Name','acsaf','name')),
-    (63, 1, 3, '411', 'acdefgiklnpqstvwxyz4', oils_i18n_gettext('63','See From Tracing -- Meeting Name','acsaf','name')),
-    (65, 1, 5, '450', 'abivwxyz4', oils_i18n_gettext('65','See From Tracing -- Topical Term','acsaf','name')),
-    (66, 1, 6, '451', 'aivwxyz4', oils_i18n_gettext('66','See From Tracing -- Geographic Name','acsaf','name')),
-    (67, 1, 7, '455', 'aivwxyz4', oils_i18n_gettext('67','See From Tracing -- Genre/Form Term','acsaf','name')),
-    (68, 1, 8, '480', 'ivwxyz4', oils_i18n_gettext('68','See From Tracing -- General Subdivision','acsaf','name')),
-    (69, 1, 9, '481', 'ivwxyz4', oils_i18n_gettext('69','See From Tracing -- Geographic Subdivision','acsaf','name')),
-    (70, 1, 10, '482', 'ivwxyz4', oils_i18n_gettext('70','See From Tracing -- Chronological Subdivision','acsaf','name')),
-    (71, 1, 11, '485', 'ivwxyz4', oils_i18n_gettext('71','See From Tracing -- Form Subdivision','acsaf','name')),
-    (72, 1, 12, '448', 'aivwxyz4', oils_i18n_gettext('72','See From Tracing -- Chronological Term','acsaf','name'));
+    (61, 1, 1, '400', 'abcdefiklmnopqrstvwxyz4', 'abcdefiklmnopqrstvxyz', oils_i18n_gettext('61','See From Tracing -- Personal Name','acsaf','name')),
+    (62, 1, 2, '410', 'abcdefgiklmnoprstvwxyz4', 'abcdefgiklmnoprstvxyz', oils_i18n_gettext('62','See From Tracing -- Corporate Name','acsaf','name')),
+    (63, 1, 3, '411', 'acdefgiklnpqstvwxyz4', 'acdefgiklnpqstvxyz', oils_i18n_gettext('63','See From Tracing -- Meeting Name','acsaf','name')),
+    (65, 1, 5, '450', 'abivwxyz4', 'abivxyz', oils_i18n_gettext('65','See From Tracing -- Topical Term','acsaf','name')),
+    (66, 1, 6, '451', 'aivwxyz4', 'aivxyz', oils_i18n_gettext('66','See From Tracing -- Geographic Name','acsaf','name')),
+    (67, 1, 7, '455', 'aivwxyz4', 'aivxyz', oils_i18n_gettext('67','See From Tracing -- Genre/Form Term','acsaf','name')),
+    (68, 1, 8, '480', 'ivwxyz4', 'ivxyz', oils_i18n_gettext('68','See From Tracing -- General Subdivision','acsaf','name')),
+    (69, 1, 9, '481', 'ivwxyz4', 'ivxyz', oils_i18n_gettext('69','See From Tracing -- Geographic Subdivision','acsaf','name')),
+    (70, 1, 10, '482', 'ivwxyz4', 'ivxyz', oils_i18n_gettext('70','See From Tracing -- Chronological Subdivision','acsaf','name')),
+    (71, 1, 11, '485', 'ivwxyz4', 'ivxyz', oils_i18n_gettext('71','See From Tracing -- Form Subdivision','acsaf','name')),
+    (72, 1, 12, '448', 'aivwxyz4', 'aivxyz', oils_i18n_gettext('72','See From Tracing -- Chronological Term','acsaf','name'));
 
 UPDATE authority.control_set_authority_field
     SET linking_subfield = '0' WHERE main_entry IS NOT NULL;
index 542440a..64da844 100644 (file)
@@ -182,6 +182,16 @@ Added Log Comment
                        </titleInfo>
                </xsl:for-each>
                <xsl:for-each select="marc:datafield[@tag='242']">
+                       <xsl:variable name="titleChop">
+                               <xsl:call-template name="chopPunctuation">
+                                       <xsl:with-param name="chopString">
+                                               <xsl:call-template name="subfieldSelect">
+                                                       <!-- 1/04 removed $h, b -->
+                                                       <xsl:with-param name="codes">a</xsl:with-param>
+                                               </xsl:call-template>
+                                       </xsl:with-param>
+                               </xsl:call-template>
+                       </xsl:variable>
                        <titleInfo type="translated">
                                <!--09/01/04 Added subfield $y-->
                                <xsl:for-each select="marc:subfield[@code='y']">
@@ -190,19 +200,36 @@ Added Log Comment
                                        </xsl:attribute>
                                </xsl:for-each>
                                <title>
-                                       <xsl:call-template name="chopPunctuation">
-                                               <xsl:with-param name="chopString">
-                                                       <xsl:call-template name="subfieldSelect">
-                                                               <!-- 1/04 removed $h, b -->
-                                                               <xsl:with-param name="codes">a</xsl:with-param>
-                                                       </xsl:call-template>
-                                               </xsl:with-param>
-                                       </xsl:call-template>
+                                       <xsl:value-of select="$titleChop" />
                                </title>
                                <!-- 1/04 fix -->
                                <xsl:call-template name="subtitle"/>
                                <xsl:call-template name="part"/>
                        </titleInfo>
+                       <titleInfo type="translated-nfi">
+                               <xsl:for-each select="marc:subfield[@code='y']">
+                                       <xsl:attribute name="lang">
+                                               <xsl:value-of select="text()"/>
+                                       </xsl:attribute>
+                               </xsl:for-each>
+                               <xsl:choose>
+                                       <xsl:when test="@ind2>0">
+                                               <nonSort>
+                                                       <xsl:value-of select="substring($titleChop,1,@ind2)"/>
+                                               </nonSort>
+                                               <title>
+                                                       <xsl:value-of select="substring($titleChop,@ind2+1)"/>
+                                               </title>
+                                       </xsl:when>
+                                       <xsl:otherwise>
+                                               <title>
+                                                       <xsl:value-of select="$titleChop" />
+                                               </title>
+                                       </xsl:otherwise>
+                               </xsl:choose>
+                               <xsl:call-template name="subtitle"/>
+                               <xsl:call-template name="part"/>
+                       </titleInfo>
                </xsl:for-each>
                <xsl:for-each select="marc:datafield[@tag='246']">
                        <titleInfo type="alternative">
@@ -226,39 +253,91 @@ Added Log Comment
                        </titleInfo>
                </xsl:for-each>
                <xsl:for-each select="marc:datafield[@tag='130']|marc:datafield[@tag='240']|marc:datafield[@tag='730'][@ind2!='2']">
+                       <xsl:variable name="nfi">
+                               <xsl:choose>
+                                       <xsl:when test="@tag='240'">
+                                               <xsl:value-of select="@ind2"/>
+                                       </xsl:when>
+                                       <xsl:otherwise>
+                                               <xsl:value-of select="@ind1"/>
+                                       </xsl:otherwise>
+                               </xsl:choose>
+                       </xsl:variable>
+                       <xsl:variable name="titleChop">
+                               <xsl:call-template name="uri" />
+                               <xsl:variable name="str">
+                                       <xsl:for-each select="marc:subfield">
+                                               <xsl:if test="(contains('adfklmor',@code) and (not(../marc:subfield[@code='n' or @code='p']) or (following-sibling::marc:subfield[@code='n' or @code='p'])))">
+                                                       <xsl:value-of select="text()"/>
+                                                       <xsl:text> </xsl:text>
+                                               </xsl:if>
+                                       </xsl:for-each>
+                               </xsl:variable>
+                               <xsl:call-template name="chopPunctuation">
+                                       <xsl:with-param name="chopString">
+                                               <xsl:value-of select="substring($str,1,string-length($str)-1)"/>
+                                       </xsl:with-param>
+                               </xsl:call-template>
+                       </xsl:variable>
                        <titleInfo type="uniform">
                                <title>
-                                       <xsl:call-template name="uri" />
-                                       <xsl:variable name="str">
-                                               <xsl:for-each select="marc:subfield">
-                                                       <xsl:if test="(contains('adfklmor',@code) and (not(../marc:subfield[@code='n' or @code='p']) or (following-sibling::marc:subfield[@code='n' or @code='p'])))">
-                                                               <xsl:value-of select="text()"/>
-                                                               <xsl:text> </xsl:text>
-                                                       </xsl:if>
-                                               </xsl:for-each>
-                                       </xsl:variable>
-                                       <xsl:call-template name="chopPunctuation">
-                                               <xsl:with-param name="chopString">
-                                                       <xsl:value-of select="substring($str,1,string-length($str)-1)"/>
-                                               </xsl:with-param>
-                                       </xsl:call-template>
+                                       <xsl:value-of select="$titleChop"/>
                                </title>
                                <xsl:call-template name="part"/>
                        </titleInfo>
+                       <titleInfo type="uniform-nfi">
+                               <xsl:choose>
+                                       <xsl:when test="$nfi>0">
+                                               <nonSort>
+                                                       <xsl:value-of select="substring($titleChop,1,$nfi)"/>
+                                               </nonSort>
+                                               <title>
+                                                       <xsl:value-of select="substring($titleChop,$nfi+1)"/>
+                                               </title>
+                                       </xsl:when>
+                                       <xsl:otherwise>
+                                               <title>
+                                                       <xsl:value-of select="$titleChop"/>
+                                               </title>
+                                       </xsl:otherwise>
+                               </xsl:choose>
+                               <xsl:call-template name="part"/>
+                       </titleInfo>
                </xsl:for-each>
                <xsl:for-each select="marc:datafield[@tag='740'][@ind2!='2']">
+                       <xsl:variable name="titleChop">
+                               <xsl:call-template name="chopPunctuation">
+                                       <xsl:with-param name="chopString">
+                                               <xsl:call-template name="subfieldSelect">
+                                                       <xsl:with-param name="codes">ah</xsl:with-param>
+                                               </xsl:call-template>
+                                       </xsl:with-param>
+                               </xsl:call-template>
+                       </xsl:variable>
                        <titleInfo type="alternative">
                                <title>
-                                       <xsl:call-template name="chopPunctuation">
-                                               <xsl:with-param name="chopString">
-                                                       <xsl:call-template name="subfieldSelect">
-                                                               <xsl:with-param name="codes">ah</xsl:with-param>
-                                                       </xsl:call-template>
-                                               </xsl:with-param>
-                                       </xsl:call-template>
+                                       <xsl:value-of select="$titleChop" />
                                </title>
                                <xsl:call-template name="part"/>
                        </titleInfo>
+                       <titleInfo type="alternative-nfi">
+                               <xsl:choose>
+                                       <xsl:when test="@ind1>0">
+                                               <nonSort>
+                                                       <xsl:value-of select="substring($titleChop,1,@ind1)"/>
+                                               </nonSort>
+                                               <title>
+                                                       <xsl:value-of select="substring($titleChop,@ind1+1)"/>
+                                               </title>
+                                       </xsl:when>
+                                       <xsl:otherwise>
+                                               <title>
+                                                       <xsl:value-of select="$titleChop" />
+                                               </title>
+                                       </xsl:otherwise>
+                               </xsl:choose>
+                               <xsl:call-template name="part"/>
+                       </titleInfo>
                </xsl:for-each>
                <xsl:for-each select="marc:datafield[@tag='100']">
                        <name type="personal">
@@ -1591,18 +1670,40 @@ Added Log Comment
                </xsl:for-each>
                <xsl:for-each select="marc:datafield[@tag=440]">
                        <relatedItem type="series">
+                               <xsl:variable name="titleChop">
+                                       <xsl:call-template name="chopPunctuation">
+                                               <xsl:with-param name="chopString">
+                                                       <xsl:call-template name="subfieldSelect">
+                                                               <xsl:with-param name="codes">av</xsl:with-param>
+                                                       </xsl:call-template>
+                                               </xsl:with-param>
+                                       </xsl:call-template>
+                               </xsl:variable>
                                <titleInfo>
                                        <title>
-                                               <xsl:call-template name="chopPunctuation">
-                                                       <xsl:with-param name="chopString">
-                                                               <xsl:call-template name="subfieldSelect">
-                                                                       <xsl:with-param name="codes">av</xsl:with-param>
-                                                               </xsl:call-template>
-                                                       </xsl:with-param>
-                                               </xsl:call-template>
+                                               <xsl:value-of select="$titleChop" />
                                        </title>
                                        <xsl:call-template name="part"></xsl:call-template>
                                </titleInfo>
+                               <titleInfo type="nfi">
+                                       <xsl:choose>
+                                               <xsl:when test="@ind2>0">
+                                                       <nonSort>
+                                                               <xsl:value-of select="substring($titleChop,1,@ind2)"/>
+                                                       </nonSort>
+                                                       <title>
+                                                               <xsl:value-of select="substring($titleChop,@ind2+1)"/>
+                                                       </title>
+                                                       <xsl:call-template name="part"/>
+                                               </xsl:when>
+                                               <xsl:otherwise>
+                                                       <title>
+                                                               <xsl:value-of select="$titleChop" />
+                                                       </title>
+                                               </xsl:otherwise>
+                                       </xsl:choose>
+                                       <xsl:call-template name="part"></xsl:call-template>
+                               </titleInfo>
                        </relatedItem>
                </xsl:for-each>
                <xsl:for-each select="marc:datafield[@tag=490][@ind1=0]">
@@ -1788,16 +1889,37 @@ Added Log Comment
                <xsl:for-each select="marc:datafield[@tag=740][@ind2=2]">
                        <relatedItem>
                                <xsl:call-template name="constituentOrRelatedType"></xsl:call-template>
+                               <xsl:variable name="titleChop">
+                                       <xsl:call-template name="chopPunctuation">
+                                               <xsl:with-param name="chopString">
+                                                       <xsl:value-of select="marc:subfield[@code='a']"></xsl:value-of>
+                                               </xsl:with-param>
+                                       </xsl:call-template>
+                               </xsl:variable>
                                <titleInfo>
                                        <title>
-                                               <xsl:call-template name="chopPunctuation">
-                                                       <xsl:with-param name="chopString">
-                                                               <xsl:value-of select="marc:subfield[@code='a']"></xsl:value-of>
-                                                       </xsl:with-param>
-                                               </xsl:call-template>
+                                               <xsl:value-of select="$titleChop" />
                                        </title>
                                        <xsl:call-template name="part"></xsl:call-template>
                                </titleInfo>
+                               <titleInfo type="nfi">
+                                       <xsl:choose>
+                                               <xsl:when test="@ind1>0">
+                                                       <nonSort>
+                                                               <xsl:value-of select="substring($titleChop,1,@ind1)"/>
+                                                       </nonSort>
+                                                       <title>
+                                                               <xsl:value-of select="substring($titleChop,@ind1+1)"/>
+                                                       </title>
+                                               </xsl:when>
+                                               <xsl:otherwise>
+                                                       <title>
+                                                               <xsl:value-of select="$titleChop" />
+                                                       </title>
+                                               </xsl:otherwise>
+                                       </xsl:choose>
+                                       <xsl:call-template name="part"></xsl:call-template>
+                               </titleInfo>
                                <xsl:call-template name="relatedForm"></xsl:call-template>
                        </relatedItem>
                </xsl:for-each>
@@ -1951,18 +2073,39 @@ Added Log Comment
                </xsl:for-each>
                <xsl:for-each select="marc:datafield[@tag='830']">
                        <relatedItem type="series">
+                               <xsl:variable name="titleChop">
+                                       <xsl:call-template name="chopPunctuation">
+                                               <xsl:with-param name="chopString">
+                                                       <xsl:call-template name="subfieldSelect">
+                                                               <xsl:with-param name="codes">adfgklmorsv</xsl:with-param>
+                                                       </xsl:call-template>
+                                               </xsl:with-param>
+                                       </xsl:call-template>
+                               </xsl:variable>
                                <titleInfo>
                                        <title>
-                                               <xsl:call-template name="chopPunctuation">
-                                                       <xsl:with-param name="chopString">
-                                                               <xsl:call-template name="subfieldSelect">
-                                                                       <xsl:with-param name="codes">adfgklmorsv</xsl:with-param>
-                                                               </xsl:call-template>
-                                                       </xsl:with-param>
-                                               </xsl:call-template>
+                                               <xsl:value-of select="$titleChop" />
                                        </title>
                                        <xsl:call-template name="part"/>
                                </titleInfo>
+                               <titleInfo type="nfi">
+                                       <xsl:choose>
+                                               <xsl:when test="@ind2>0">
+                                                       <nonSort>
+                                                               <xsl:value-of select="substring($titleChop,1,@ind2)"/>
+                                                       </nonSort>
+                                                       <title>
+                                                               <xsl:value-of select="substring($titleChop,@ind2+1)"/>
+                                                       </title>
+                                               </xsl:when>
+                                               <xsl:otherwise>
+                                                       <title>
+                                                               <xsl:value-of select="$titleChop" />
+                                                       </title>
+                                               </xsl:otherwise>
+                                       </xsl:choose>
+                                       <xsl:call-template name="part"/>
+                               </titleInfo>
                                <xsl:call-template name="relatedForm"/>
                        </relatedItem>
                </xsl:for-each>
@@ -2629,6 +2772,7 @@ Added Log Comment
                <subject>
                        <xsl:call-template name="subjectAuthority"></xsl:call-template>
                        <name type="personal">
+                               <xsl:call-template name="uri" />
                                <xsl:call-template name="termsOfAddress"></xsl:call-template>
                                <namePart>
                                        <xsl:call-template name="chopPunctuation">
@@ -2650,6 +2794,7 @@ Added Log Comment
                <subject>
                        <xsl:call-template name="subjectAuthority"></xsl:call-template>
                        <name type="corporate">
+                               <xsl:call-template name="uri" />
                                <xsl:for-each select="marc:subfield[@code='a']">
                                        <namePart>
                                                <xsl:value-of select="."></xsl:value-of>
@@ -2676,6 +2821,7 @@ Added Log Comment
                <subject>
                        <xsl:call-template name="subjectAuthority"></xsl:call-template>
                        <name type="conference">
+                               <xsl:call-template name="uri" />
                                <namePart>
                                        <xsl:call-template name="subfieldSelect">
                                                <xsl:with-param name="codes">abcdeqnp</xsl:with-param>
@@ -2695,17 +2841,39 @@ Added Log Comment
        <xsl:template match="marc:datafield[@tag=630]">
                <subject>
                        <xsl:call-template name="subjectAuthority"></xsl:call-template>
+                       <xsl:variable name="titleChop">
+                               <xsl:call-template name="chopPunctuation">
+                                       <xsl:with-param name="chopString">
+                                               <xsl:call-template name="subfieldSelect">
+                                                       <xsl:with-param name="codes">adfhklor</xsl:with-param>
+                                               </xsl:call-template>
+                                       </xsl:with-param>
+                               </xsl:call-template>
+                       </xsl:variable>
                        <titleInfo>
                                <title>
-                                       <xsl:call-template name="chopPunctuation">
-                                               <xsl:with-param name="chopString">
-                                                       <xsl:call-template name="subfieldSelect">
-                                                               <xsl:with-param name="codes">adfhklor</xsl:with-param>
-                                                       </xsl:call-template>
-                                               </xsl:with-param>
-                                       </xsl:call-template>
-                                       <xsl:call-template name="part"></xsl:call-template>
+                                       <xsl:value-of select="$titleChop" />
                                </title>
+                               <xsl:call-template name="part"></xsl:call-template>
+                       </titleInfo>
+                       <titleInfo type="nfi">
+                               <xsl:choose>
+                                       <xsl:when test="@ind1>0">
+                                               <nonSort>
+                                                       <xsl:value-of select="substring($titleChop,1,@ind1)"/>
+                                               </nonSort>
+                                               <title>
+                                                       <xsl:value-of select="substring($titleChop,@ind1+1)"/>
+                                               </title>
+                                               <xsl:call-template name="part"/>
+                                       </xsl:when>
+                                       <xsl:otherwise>
+                                               <title>
+                                                       <xsl:value-of select="$titleChop" />
+                                               </title>
+                                       </xsl:otherwise>
+                               </xsl:choose>
+                               <xsl:call-template name="part"></xsl:call-template>
                        </titleInfo>
                        <xsl:call-template name="subjectAnyOrder"></xsl:call-template>
                </subject>
@@ -2714,6 +2882,7 @@ Added Log Comment
                <subject>
                        <xsl:call-template name="subjectAuthority"></xsl:call-template>
                        <topic>
+                               <xsl:call-template name="uri" />
                                <xsl:call-template name="chopPunctuation">
                                        <xsl:with-param name="chopString">
                                                <xsl:call-template name="subfieldSelect">
@@ -2730,6 +2899,7 @@ Added Log Comment
                        <xsl:call-template name="subjectAuthority"></xsl:call-template>
                        <xsl:for-each select="marc:subfield[@code='a']">
                                <geographic>
+                                       <xsl:call-template name="uri" />
                                        <xsl:call-template name="chopPunctuation">
                                                <xsl:with-param name="chopString" select="."></xsl:with-param>
                                        </xsl:call-template>
@@ -2742,6 +2912,7 @@ Added Log Comment
                <subject>
                        <xsl:for-each select="marc:subfield[@code='a']">
                                <topic>
+                                       <xsl:call-template name="uri" />
                                        <xsl:value-of select="."></xsl:value-of>
                                </topic>
                        </xsl:for-each>
@@ -2754,8 +2925,8 @@ Added Log Comment
                                        <xsl:value-of select="marc:subfield[@code=2]"></xsl:value-of>
                                </xsl:attribute>
                        </xsl:if>
-                       <xsl:call-template name="uri" />
                        <occupation>
+                               <xsl:call-template name="uri" />
                                <xsl:call-template name="chopPunctuation">
                                        <xsl:with-param name="chopString">
                                                <xsl:value-of select="marc:subfield[@code='a']"></xsl:value-of>
index 404480f..e419dae 100644 (file)
@@ -1537,7 +1537,7 @@ BEGIN
             FROM authority.control_set_authority_field
             WHERE tag IN (
                 SELECT UNNEST(
-                    XPATH('//*[starts-with(@tag,"1")]/@tag',rec_marc::XML)::TEXT[]
+                    XPATH('//*[starts-with(@tag,"1")]/@tag',rec_marc_xml)::TEXT[]
                 )
             ) LIMIT 1;
 
index 0424a77..7901876 100644 (file)
@@ -33,7 +33,7 @@ BEGIN
             FROM authority.control_set_authority_field
             WHERE tag IN (
                 SELECT UNNEST(
-                    XPATH('//*[starts-with(@tag,"1")]/@tag',rec_marc::XML)::TEXT[]
+                    XPATH('//*[starts-with(@tag,"1")]/@tag',rec_marc_xml::XML)::TEXT[]
                 )
             ) LIMIT 1;
 
index ba30e46..66c8fcb 100644 (file)
@@ -3,11 +3,21 @@ BEGIN;
 -- check whether patch can be applied
 -- SELECT evergreen.upgrade_deps_block_check('YYYY', :eg_version);
 
+ALTER TABLE authority.control_set_authority_field
+    ADD COLUMN display_sf_list TEXT;
+
+UPDATE authority.control_set_authority_field
+    SET display_sf_list = REGEXP_REPLACE(sf_list, '[w254]', '', 'g');
+
+ALTER TABLE authority.control_set_authority_field
+    ALTER COLUMN display_sf_list SET NOT NULL;
+
 ALTER TABLE metabib.browse_entry_def_map
     ADD COLUMN authority BIGINT REFERENCES authority.record_entry (id)
         ON DELETE SET NULL;
 
 ALTER TABLE config.metabib_field ADD COLUMN authority_xpath TEXT;
+ALTER TABLE config.metabib_field ADD COLUMN browse_sort_xpath TEXT;
 
 UPDATE config.metabib_field
     SET authority_xpath = '//@xlink:href'
@@ -17,7 +27,7 @@ UPDATE config.metabib_field
         browse_field IS TRUE;
 
 ALTER TYPE metabib.field_entry_template ADD ATTRIBUTE authority BIGINT;
-
+ALTER TYPE metabib.field_entry_template ADD ATTRIBUTE sort_value TEXT;
 
 CREATE OR REPLACE FUNCTION metabib.reingest_metabib_field_entries( bib_id BIGINT, skip_facet BOOL DEFAULT FALSE, skip_browse BOOL DEFAULT FALSE, skip_search BOOL DEFAULT FALSE ) RETURNS VOID AS $func$
 DECLARE
@@ -28,6 +38,7 @@ DECLARE
     b_skip_facet    BOOL;
     b_skip_browse   BOOL;
     b_skip_search   BOOL;
+    value_prepped   TEXT;
 BEGIN
 
     SELECT COALESCE(NULLIF(skip_facet, FALSE), EXISTS (SELECT enabled FROM config.internal_flag WHERE name =  'ingest.skip_facet_indexing' AND enabled)) INTO b_skip_facet;
@@ -67,12 +78,18 @@ BEGIN
             -- evergreen.oils_tsearch2()) changes.  It may or may not be
             -- expensive to add a comparison of index_vector to index_vector
             -- to the WHERE clause below.
-            SELECT INTO mbe_row * FROM metabib.browse_entry WHERE value = ind_data.value;
+
+            value_prepped := metabib.browse_normalize(ind_data.value, ind_data.field);
+            SELECT INTO mbe_row * FROM metabib.browse_entry
+                WHERE value = value_prepped AND sort_value = ind_data.sort_value;
+
             IF FOUND THEN
                 mbe_id := mbe_row.id;
             ELSE
-                INSERT INTO metabib.browse_entry (value) VALUES
-                    (metabib.browse_normalize(ind_data.value, ind_data.field));
+                INSERT INTO metabib.browse_entry
+                    ( value, sort_value ) VALUES
+                    ( value_prepped, ind_data.sort_value );
+
                 mbe_id := CURRVAL('metabib.browse_entry_id_seq'::REGCLASS);
             END IF;
 
@@ -112,6 +129,7 @@ DECLARE
     xml_node_list   TEXT[];
     facet_text  TEXT;
     browse_text TEXT;
+    sort_value  TEXT;
     raw_text    TEXT;
     curr_text   TEXT;
     joiner      TEXT := default_joiner; -- XXX will index defs supply a joiner?
@@ -180,10 +198,25 @@ BEGIN
                     browse_text := curr_text;
                 END IF;
 
+                IF idx.browse_sort_xpath IS NOT NULL AND
+                    idx.browse_sort_xpath <> '' THEN
+
+                    sort_value := oils_xpath_string(
+                        idx.browse_sort_xpath, xml_node, joiner,
+                        ARRAY[ARRAY[xfrm.prefix, xfrm.namespace_uri]]
+                    );
+                ELSE
+                    sort_value := browse_text;
+                END IF;
+
                 output_row.field_class = idx.field_class;
                 output_row.field = idx.id;
                 output_row.source = rid;
                 output_row.value = BTRIM(REGEXP_REPLACE(browse_text, E'\\s+', ' ', 'g'));
+                output_row.sort_value :=
+                    public.search_normalize(sort_value);
+
+                output_row.authority := NULL;
 
                 IF idx.authority_xpath IS NOT NULL AND idx.authority_xpath <> '' THEN
                     authority_text := oils_xpath_string(
@@ -207,6 +240,7 @@ BEGIN
                 output_row.browse_field = TRUE;
                 RETURN NEXT output_row;
                 output_row.browse_field = FALSE;
+                output_row.sort_value := NULL;
             END IF;
 
             -- insert raw node text for faceting
@@ -241,6 +275,7 @@ BEGIN
 
             output_row.search_field = TRUE;
             RETURN NEXT output_row;
+            output_row.search_field = FALSE;
         END IF;
 
     END LOOP;
@@ -435,6 +470,16 @@ Added Log Comment
                        </titleInfo>
                </xsl:for-each>
                <xsl:for-each select="marc:datafield[@tag='242']">
+                       <xsl:variable name="titleChop">
+                               <xsl:call-template name="chopPunctuation">
+                                       <xsl:with-param name="chopString">
+                                               <xsl:call-template name="subfieldSelect">
+                                                       <!-- 1/04 removed $h, b -->
+                                                       <xsl:with-param name="codes">a</xsl:with-param>
+                                               </xsl:call-template>
+                                       </xsl:with-param>
+                               </xsl:call-template>
+                       </xsl:variable>
                        <titleInfo type="translated">
                                <!--09/01/04 Added subfield $y-->
                                <xsl:for-each select="marc:subfield[@code='y']">
@@ -443,19 +488,36 @@ Added Log Comment
                                        </xsl:attribute>
                                </xsl:for-each>
                                <title>
-                                       <xsl:call-template name="chopPunctuation">
-                                               <xsl:with-param name="chopString">
-                                                       <xsl:call-template name="subfieldSelect">
-                                                               <!-- 1/04 removed $h, b -->
-                                                               <xsl:with-param name="codes">a</xsl:with-param>
-                                                       </xsl:call-template>
-                                               </xsl:with-param>
-                                       </xsl:call-template>
+                                       <xsl:value-of select="$titleChop" />
                                </title>
                                <!-- 1/04 fix -->
                                <xsl:call-template name="subtitle"/>
                                <xsl:call-template name="part"/>
                        </titleInfo>
+                       <titleInfo type="translated-nfi">
+                               <xsl:for-each select="marc:subfield[@code='y']">
+                                       <xsl:attribute name="lang">
+                                               <xsl:value-of select="text()"/>
+                                       </xsl:attribute>
+                               </xsl:for-each>
+                               <xsl:choose>
+                                       <xsl:when test="@ind2>0">
+                                               <nonSort>
+                                                       <xsl:value-of select="substring($titleChop,1,@ind2)"/>
+                                               </nonSort>
+                                               <title>
+                                                       <xsl:value-of select="substring($titleChop,@ind2+1)"/>
+                                               </title>
+                                       </xsl:when>
+                                       <xsl:otherwise>
+                                               <title>
+                                                       <xsl:value-of select="$titleChop" />
+                                               </title>
+                                       </xsl:otherwise>
+                               </xsl:choose>
+                               <xsl:call-template name="subtitle"/>
+                               <xsl:call-template name="part"/>
+                       </titleInfo>
                </xsl:for-each>
                <xsl:for-each select="marc:datafield[@tag='246']">
                        <titleInfo type="alternative">
@@ -479,39 +541,91 @@ Added Log Comment
                        </titleInfo>
                </xsl:for-each>
                <xsl:for-each select="marc:datafield[@tag='130']|marc:datafield[@tag='240']|marc:datafield[@tag='730'][@ind2!='2']">
+                       <xsl:variable name="nfi">
+                               <xsl:choose>
+                                       <xsl:when test="@tag='240'">
+                                               <xsl:value-of select="@ind2"/>
+                                       </xsl:when>
+                                       <xsl:otherwise>
+                                               <xsl:value-of select="@ind1"/>
+                                       </xsl:otherwise>
+                               </xsl:choose>
+                       </xsl:variable>
+                       <xsl:variable name="titleChop">
+                               <xsl:call-template name="uri" />
+                               <xsl:variable name="str">
+                                       <xsl:for-each select="marc:subfield">
+                                               <xsl:if test="(contains('adfklmor',@code) and (not(../marc:subfield[@code='n' or @code='p']) or (following-sibling::marc:subfield[@code='n' or @code='p'])))">
+                                                       <xsl:value-of select="text()"/>
+                                                       <xsl:text> </xsl:text>
+                                               </xsl:if>
+                                       </xsl:for-each>
+                               </xsl:variable>
+                               <xsl:call-template name="chopPunctuation">
+                                       <xsl:with-param name="chopString">
+                                               <xsl:value-of select="substring($str,1,string-length($str)-1)"/>
+                                       </xsl:with-param>
+                               </xsl:call-template>
+                       </xsl:variable>
                        <titleInfo type="uniform">
                                <title>
-                                       <xsl:call-template name="uri" />
-                                       <xsl:variable name="str">
-                                               <xsl:for-each select="marc:subfield">
-                                                       <xsl:if test="(contains('adfklmor',@code) and (not(../marc:subfield[@code='n' or @code='p']) or (following-sibling::marc:subfield[@code='n' or @code='p'])))">
-                                                               <xsl:value-of select="text()"/>
-                                                               <xsl:text> </xsl:text>
-                                                       </xsl:if>
-                                               </xsl:for-each>
-                                       </xsl:variable>
-                                       <xsl:call-template name="chopPunctuation">
-                                               <xsl:with-param name="chopString">
-                                                       <xsl:value-of select="substring($str,1,string-length($str)-1)"/>
-                                               </xsl:with-param>
-                                       </xsl:call-template>
+                                       <xsl:value-of select="$titleChop"/>
                                </title>
                                <xsl:call-template name="part"/>
                        </titleInfo>
+                       <titleInfo type="uniform-nfi">
+                               <xsl:choose>
+                                       <xsl:when test="$nfi>0">
+                                               <nonSort>
+                                                       <xsl:value-of select="substring($titleChop,1,$nfi)"/>
+                                               </nonSort>
+                                               <title>
+                                                       <xsl:value-of select="substring($titleChop,$nfi+1)"/>
+                                               </title>
+                                       </xsl:when>
+                                       <xsl:otherwise>
+                                               <title>
+                                                       <xsl:value-of select="$titleChop"/>
+                                               </title>
+                                       </xsl:otherwise>
+                               </xsl:choose>
+                               <xsl:call-template name="part"/>
+                       </titleInfo>
                </xsl:for-each>
                <xsl:for-each select="marc:datafield[@tag='740'][@ind2!='2']">
+                       <xsl:variable name="titleChop">
+                               <xsl:call-template name="chopPunctuation">
+                                       <xsl:with-param name="chopString">
+                                               <xsl:call-template name="subfieldSelect">
+                                                       <xsl:with-param name="codes">ah</xsl:with-param>
+                                               </xsl:call-template>
+                                       </xsl:with-param>
+                               </xsl:call-template>
+                       </xsl:variable>
                        <titleInfo type="alternative">
                                <title>
-                                       <xsl:call-template name="chopPunctuation">
-                                               <xsl:with-param name="chopString">
-                                                       <xsl:call-template name="subfieldSelect">
-                                                               <xsl:with-param name="codes">ah</xsl:with-param>
-                                                       </xsl:call-template>
-                                               </xsl:with-param>
-                                       </xsl:call-template>
+                                       <xsl:value-of select="$titleChop" />
                                </title>
                                <xsl:call-template name="part"/>
                        </titleInfo>
+                       <titleInfo type="alternative-nfi">
+                               <xsl:choose>
+                                       <xsl:when test="@ind1>0">
+                                               <nonSort>
+                                                       <xsl:value-of select="substring($titleChop,1,@ind1)"/>
+                                               </nonSort>
+                                               <title>
+                                                       <xsl:value-of select="substring($titleChop,@ind1+1)"/>
+                                               </title>
+                                       </xsl:when>
+                                       <xsl:otherwise>
+                                               <title>
+                                                       <xsl:value-of select="$titleChop" />
+                                               </title>
+                                       </xsl:otherwise>
+                               </xsl:choose>
+                               <xsl:call-template name="part"/>
+                       </titleInfo>
                </xsl:for-each>
                <xsl:for-each select="marc:datafield[@tag='100']">
                        <name type="personal">
@@ -1844,18 +1958,40 @@ Added Log Comment
                </xsl:for-each>
                <xsl:for-each select="marc:datafield[@tag=440]">
                        <relatedItem type="series">
+                               <xsl:variable name="titleChop">
+                                       <xsl:call-template name="chopPunctuation">
+                                               <xsl:with-param name="chopString">
+                                                       <xsl:call-template name="subfieldSelect">
+                                                               <xsl:with-param name="codes">av</xsl:with-param>
+                                                       </xsl:call-template>
+                                               </xsl:with-param>
+                                       </xsl:call-template>
+                               </xsl:variable>
                                <titleInfo>
                                        <title>
-                                               <xsl:call-template name="chopPunctuation">
-                                                       <xsl:with-param name="chopString">
-                                                               <xsl:call-template name="subfieldSelect">
-                                                                       <xsl:with-param name="codes">av</xsl:with-param>
-                                                               </xsl:call-template>
-                                                       </xsl:with-param>
-                                               </xsl:call-template>
+                                               <xsl:value-of select="$titleChop" />
                                        </title>
                                        <xsl:call-template name="part"></xsl:call-template>
                                </titleInfo>
+                               <titleInfo type="nfi">
+                                       <xsl:choose>
+                                               <xsl:when test="@ind2>0">
+                                                       <nonSort>
+                                                               <xsl:value-of select="substring($titleChop,1,@ind2)"/>
+                                                       </nonSort>
+                                                       <title>
+                                                               <xsl:value-of select="substring($titleChop,@ind2+1)"/>
+                                                       </title>
+                                                       <xsl:call-template name="part"/>
+                                               </xsl:when>
+                                               <xsl:otherwise>
+                                                       <title>
+                                                               <xsl:value-of select="$titleChop" />
+                                                       </title>
+                                               </xsl:otherwise>
+                                       </xsl:choose>
+                                       <xsl:call-template name="part"></xsl:call-template>
+                               </titleInfo>
                        </relatedItem>
                </xsl:for-each>
                <xsl:for-each select="marc:datafield[@tag=490][@ind1=0]">
@@ -2041,16 +2177,37 @@ Added Log Comment
                <xsl:for-each select="marc:datafield[@tag=740][@ind2=2]">
                        <relatedItem>
                                <xsl:call-template name="constituentOrRelatedType"></xsl:call-template>
+                               <xsl:variable name="titleChop">
+                                       <xsl:call-template name="chopPunctuation">
+                                               <xsl:with-param name="chopString">
+                                                       <xsl:value-of select="marc:subfield[@code='a']"></xsl:value-of>
+                                               </xsl:with-param>
+                                       </xsl:call-template>
+                               </xsl:variable>
                                <titleInfo>
                                        <title>
-                                               <xsl:call-template name="chopPunctuation">
-                                                       <xsl:with-param name="chopString">
-                                                               <xsl:value-of select="marc:subfield[@code='a']"></xsl:value-of>
-                                                       </xsl:with-param>
-                                               </xsl:call-template>
+                                               <xsl:value-of select="$titleChop" />
                                        </title>
                                        <xsl:call-template name="part"></xsl:call-template>
                                </titleInfo>
+                               <titleInfo type="nfi">
+                                       <xsl:choose>
+                                               <xsl:when test="@ind1>0">
+                                                       <nonSort>
+                                                               <xsl:value-of select="substring($titleChop,1,@ind1)"/>
+                                                       </nonSort>
+                                                       <title>
+                                                               <xsl:value-of select="substring($titleChop,@ind1+1)"/>
+                                                       </title>
+                                               </xsl:when>
+                                               <xsl:otherwise>
+                                                       <title>
+                                                               <xsl:value-of select="$titleChop" />
+                                                       </title>
+                                               </xsl:otherwise>
+                                       </xsl:choose>
+                                       <xsl:call-template name="part"></xsl:call-template>
+                               </titleInfo>
                                <xsl:call-template name="relatedForm"></xsl:call-template>
                        </relatedItem>
                </xsl:for-each>
@@ -2204,18 +2361,39 @@ Added Log Comment
                </xsl:for-each>
                <xsl:for-each select="marc:datafield[@tag='830']">
                        <relatedItem type="series">
+                               <xsl:variable name="titleChop">
+                                       <xsl:call-template name="chopPunctuation">
+                                               <xsl:with-param name="chopString">
+                                                       <xsl:call-template name="subfieldSelect">
+                                                               <xsl:with-param name="codes">adfgklmorsv</xsl:with-param>
+                                                       </xsl:call-template>
+                                               </xsl:with-param>
+                                       </xsl:call-template>
+                               </xsl:variable>
                                <titleInfo>
                                        <title>
-                                               <xsl:call-template name="chopPunctuation">
-                                                       <xsl:with-param name="chopString">
-                                                               <xsl:call-template name="subfieldSelect">
-                                                                       <xsl:with-param name="codes">adfgklmorsv</xsl:with-param>
-                                                               </xsl:call-template>
-                                                       </xsl:with-param>
-                                               </xsl:call-template>
+                                               <xsl:value-of select="$titleChop" />
                                        </title>
                                        <xsl:call-template name="part"/>
                                </titleInfo>
+                               <titleInfo type="nfi">
+                                       <xsl:choose>
+                                               <xsl:when test="@ind2>0">
+                                                       <nonSort>
+                                                               <xsl:value-of select="substring($titleChop,1,@ind2)"/>
+                                                       </nonSort>
+                                                       <title>
+                                                               <xsl:value-of select="substring($titleChop,@ind2+1)"/>
+                                                       </title>
+                                               </xsl:when>
+                                               <xsl:otherwise>
+                                                       <title>
+                                                               <xsl:value-of select="$titleChop" />
+                                                       </title>
+                                               </xsl:otherwise>
+                                       </xsl:choose>
+                                       <xsl:call-template name="part"/>
+                               </titleInfo>
                                <xsl:call-template name="relatedForm"/>
                        </relatedItem>
                </xsl:for-each>
@@ -2882,6 +3060,7 @@ Added Log Comment
                <subject>
                        <xsl:call-template name="subjectAuthority"></xsl:call-template>
                        <name type="personal">
+                               <xsl:call-template name="uri" />
                                <xsl:call-template name="termsOfAddress"></xsl:call-template>
                                <namePart>
                                        <xsl:call-template name="chopPunctuation">
@@ -2903,6 +3082,7 @@ Added Log Comment
                <subject>
                        <xsl:call-template name="subjectAuthority"></xsl:call-template>
                        <name type="corporate">
+                               <xsl:call-template name="uri" />
                                <xsl:for-each select="marc:subfield[@code='a']">
                                        <namePart>
                                                <xsl:value-of select="."></xsl:value-of>
@@ -2929,6 +3109,7 @@ Added Log Comment
                <subject>
                        <xsl:call-template name="subjectAuthority"></xsl:call-template>
                        <name type="conference">
+                               <xsl:call-template name="uri" />
                                <namePart>
                                        <xsl:call-template name="subfieldSelect">
                                                <xsl:with-param name="codes">abcdeqnp</xsl:with-param>
@@ -2948,17 +3129,39 @@ Added Log Comment
        <xsl:template match="marc:datafield[@tag=630]">
                <subject>
                        <xsl:call-template name="subjectAuthority"></xsl:call-template>
+                       <xsl:variable name="titleChop">
+                               <xsl:call-template name="chopPunctuation">
+                                       <xsl:with-param name="chopString">
+                                               <xsl:call-template name="subfieldSelect">
+                                                       <xsl:with-param name="codes">adfhklor</xsl:with-param>
+                                               </xsl:call-template>
+                                       </xsl:with-param>
+                               </xsl:call-template>
+                       </xsl:variable>
                        <titleInfo>
                                <title>
-                                       <xsl:call-template name="chopPunctuation">
-                                               <xsl:with-param name="chopString">
-                                                       <xsl:call-template name="subfieldSelect">
-                                                               <xsl:with-param name="codes">adfhklor</xsl:with-param>
-                                                       </xsl:call-template>
-                                               </xsl:with-param>
-                                       </xsl:call-template>
-                                       <xsl:call-template name="part"></xsl:call-template>
+                                       <xsl:value-of select="$titleChop" />
                                </title>
+                               <xsl:call-template name="part"></xsl:call-template>
+                       </titleInfo>
+                       <titleInfo type="nfi">
+                               <xsl:choose>
+                                       <xsl:when test="@ind1>0">
+                                               <nonSort>
+                                                       <xsl:value-of select="substring($titleChop,1,@ind1)"/>
+                                               </nonSort>
+                                               <title>
+                                                       <xsl:value-of select="substring($titleChop,@ind1+1)"/>
+                                               </title>
+                                               <xsl:call-template name="part"/>
+                                       </xsl:when>
+                                       <xsl:otherwise>
+                                               <title>
+                                                       <xsl:value-of select="$titleChop" />
+                                               </title>
+                                       </xsl:otherwise>
+                               </xsl:choose>
+                               <xsl:call-template name="part"></xsl:call-template>
                        </titleInfo>
                        <xsl:call-template name="subjectAnyOrder"></xsl:call-template>
                </subject>
@@ -2967,6 +3170,7 @@ Added Log Comment
                <subject>
                        <xsl:call-template name="subjectAuthority"></xsl:call-template>
                        <topic>
+                               <xsl:call-template name="uri" />
                                <xsl:call-template name="chopPunctuation">
                                        <xsl:with-param name="chopString">
                                                <xsl:call-template name="subfieldSelect">
@@ -2983,6 +3187,7 @@ Added Log Comment
                        <xsl:call-template name="subjectAuthority"></xsl:call-template>
                        <xsl:for-each select="marc:subfield[@code='a']">
                                <geographic>
+                                       <xsl:call-template name="uri" />
                                        <xsl:call-template name="chopPunctuation">
                                                <xsl:with-param name="chopString" select="."></xsl:with-param>
                                        </xsl:call-template>
@@ -2995,6 +3200,7 @@ Added Log Comment
                <subject>
                        <xsl:for-each select="marc:subfield[@code='a']">
                                <topic>
+                                       <xsl:call-template name="uri" />
                                        <xsl:value-of select="."></xsl:value-of>
                                </topic>
                        </xsl:for-each>
@@ -3007,8 +3213,8 @@ Added Log Comment
                                        <xsl:value-of select="marc:subfield[@code=2]"></xsl:value-of>
                                </xsl:attribute>
                        </xsl:if>
-                       <xsl:call-template name="uri" />
                        <occupation>
+                               <xsl:call-template name="uri" />
                                <xsl:call-template name="chopPunctuation">
                                        <xsl:with-param name="chopString">
                                                <xsl:value-of select="marc:subfield[@code='a']"></xsl:value-of>
@@ -6882,4 +7088,342 @@ Revision 1.2 - Added Log Comment  2003/03/24 19:37:42  ckeith
        </xsl:template>
 </xsl:stylesheet>$$ WHERE name = 'mods33';
 
+
+INSERT INTO config.global_flag (name, value, enabled, label) VALUES
+(
+    'opac.browse.warnable_regexp_per_class',
+    '{"title": "^(a|the|an)\\s"}',
+    FALSE,
+    oils_i18n_gettext(
+        'opac.browse.warnable_regexp_per_class',
+        'Map of search classes to regular expressions to warn user about leading articles.',
+        'cgf',
+        'label'
+    )
+),
+(
+    'opac.browse.holdings_visibility_test_limit',
+    '100',
+    TRUE,
+    oils_i18n_gettext(
+        'opac.browse.holdings_visibility_test_limit',
+        'Don''t look for more than this number of records with holdings when displaying browse headings with visible record counts.',
+        'cgf',
+        'label'
+    )
+);
+
+ALTER TABLE metabib.browse_entry DROP CONSTRAINT browse_entry_value_key;
+ALTER TABLE metabib.browse_entry ADD COLUMN sort_value TEXT;
+DELETE FROM metabib.browse_entry_def_map; -- Yeah.
+DELETE FROM metabib.browse_entry WHERE sort_value IS NULL;
+ALTER TABLE metabib.browse_entry ALTER COLUMN sort_value SET NOT NULL;
+ALTER TABLE metabib.browse_entry ADD UNIQUE (value, sort_value);
+DROP TRIGGER IF EXISTS mbe_sort_value ON metabib.browse_entry;
+
+CREATE INDEX browse_entry_sort_value_idx
+    ON metabib.browse_entry USING BTREE (sort_value);
+
+-- NOTE If I understand ordered indices correctly, an index on sort_value DESC
+-- is not actually needed, even though we do have a query that does ORDER BY
+-- on this column in that direction.  The previous index serves for both
+-- directions, and ordering in an index is only helpful for multi-column
+-- indices, I think. See http://www.postgresql.org/docs/9.1/static/indexes-ordering.html
+
+-- CREATE INDEX CONCURRENTLY browse_entry_sort_value_idx_desc
+--     ON metabib.browse_entry USING BTREE (sort_value DESC);
+
+CREATE TYPE metabib.flat_browse_entry_appearance AS (
+    browse_entry    BIGINT,
+    value           TEXT,
+    fields          TEXT,
+    authorities     TEXT,
+    sources         INT,        -- visible ones, that is
+    row_number      INT,        -- internal use, sort of
+    accurate        BOOL        -- Count in sources field is accurate? Not
+                                -- if we had more than a browse superpage
+                                -- of records to look at.
+);
+
+
+CREATE OR REPLACE FUNCTION metabib.browse_pivot(
+    search_field        INT[],
+    browse_term         TEXT
+) RETURNS BIGINT AS $p$
+DECLARE
+    id                  BIGINT;
+BEGIN
+    SELECT INTO id mbe.id FROM metabib.browse_entry mbe
+        JOIN metabib.browse_entry_def_map mbedm ON (
+            mbedm.entry = mbe.id AND
+            mbedm.def = ANY(search_field)
+        )
+        WHERE mbe.sort_value >= public.search_normalize(browse_term)
+        ORDER BY mbe.sort_value, mbe.value LIMIT 1;
+
+    RETURN id;
+END;
+$p$ LANGUAGE PLPGSQL;
+
+CREATE OR REPLACE FUNCTION metabib.staged_browse(
+    core_query              TEXT,
+    context_org             INT,
+    context_locations       INT[],
+    staff                   BOOL,
+    browse_superpage_size   INT,
+    result_limit            INT,
+    use_offset              INT
+) RETURNS SETOF metabib.flat_browse_entry_appearance AS $p$
+DECLARE
+    core_cursor             REFCURSOR;
+    core_record             RECORD;
+    qpfts_query             TEXT;
+    result_row              metabib.flat_browse_entry_appearance%ROWTYPE;
+    results_skipped         INT := 0;
+    results_returned        INT := 0;
+    slice_start             INT;
+    slice_end               INT;
+    full_end                INT;
+    superpage_of_records    BIGINT[];
+    superpage_size          INT;
+BEGIN
+    OPEN core_cursor FOR EXECUTE core_query;
+
+    LOOP
+        FETCH core_cursor INTO core_record;
+        EXIT WHEN NOT FOUND;
+
+        result_row.sources := 0;
+
+        full_end := ARRAY_LENGTH(core_record.records, 1);
+        superpage_size := COALESCE(browse_superpage_size, full_end);
+        slice_start := 1;
+        slice_end := superpage_size;
+
+        WHILE result_row.sources = 0 AND slice_start <= full_end LOOP
+            superpage_of_records := core_record.records[slice_start:slice_end];
+            qpfts_query :=
+                'SELECT NULL::BIGINT AS id, ARRAY[r] AS records, ' ||
+                '1::INT AS rel FROM (SELECT UNNEST(' ||
+                quote_literal(superpage_of_records) || '::BIGINT[]) AS r) rr';
+
+            -- We use search.query_parser_fts() for visibility testing.
+            -- We're calling it once per browse-superpage worth of records
+            -- out of the set of records related to a given mbe, until we've
+            -- either exhausted that set of records or found at least 1
+            -- visible record.
+
+            SELECT INTO result_row.sources visible
+                FROM search.query_parser_fts(
+                    context_org, NULL, qpfts_query, NULL,
+                    context_locations, 0, NULL, NULL, FALSE, staff, FALSE
+                ) qpfts
+                WHERE qpfts.rel IS NULL;
+
+            slice_start := slice_start + superpage_size;
+            slice_end := slice_end + superpage_size;
+        END LOOP;
+
+        -- Accurate?  Well, probably.
+        result_row.accurate := browse_superpage_size IS NULL OR
+            browse_superpage_size >= full_end;
+
+        IF result_row.sources > 0 THEN
+            IF results_skipped < use_offset THEN
+                results_skipped := results_skipped + 1;
+                CONTINUE;
+            END IF;
+
+            result_row.browse_entry := core_record.id;
+            result_row.authorities := core_record.authorities;
+            result_row.fields := core_record.fields;
+            result_row.value := core_record.value;
+
+            -- This is needed so our caller can flip it and reverse it.
+            result_row.row_number := results_returned;
+
+            RETURN NEXT result_row;
+
+            results_returned := results_returned + 1;
+
+            EXIT WHEN results_returned >= result_limit;
+        END IF;
+    END LOOP;
+END;
+$p$ LANGUAGE PLPGSQL;
+
+-- This is optimized to be fast for values of result_offset near zero.
+CREATE OR REPLACE FUNCTION metabib.browse(
+    search_field            INT[],
+    browse_term             TEXT,
+    context_org             INT DEFAULT NULL,
+    context_loc_group       INT DEFAULT NULL,
+    staff                   BOOL DEFAULT FALSE,
+    pivot_id                BIGINT DEFAULT NULL,
+    force_backward          BOOL DEFAULT FALSE,
+    result_limit            INT DEFAULT 10,
+    result_offset           INT DEFAULT 0   -- Can be negative!
+) RETURNS SETOF metabib.flat_browse_entry_appearance AS $p$
+DECLARE
+    core_query              TEXT;
+    whole_query             TEXT;
+    pivot_sort_value        TEXT;
+    pivot_sort_fallback     TEXT;
+    context_locations       INT[];
+    use_offset              INT;
+    browse_superpage_size   INT;
+    results_skipped         INT := 0;
+BEGIN
+    IF pivot_id IS NULL THEN
+        pivot_id := metabib.browse_pivot(search_field, browse_term);
+    END IF;
+
+    SELECT INTO pivot_sort_value, pivot_sort_fallback
+        sort_value, value FROM metabib.browse_entry WHERE id = pivot_id;
+
+    IF pivot_sort_value IS NULL THEN
+        RETURN;
+    END IF;
+
+    IF context_loc_group IS NOT NULL THEN
+        SELECT INTO context_locations ARRAY_AGG(location)
+            FROM asset.copy_location_group_map
+            WHERE lgroup = context_loc_group;
+    END IF;
+
+    SELECT INTO browse_superpage_size value     -- NULL ok
+        FROM config.global_flag
+        WHERE enabled AND name = 'opac.browse.holdings_visibility_test_limit';
+
+    core_query := '
+    SELECT
+        mbe.id,
+        mbe.value,
+        mbe.sort_value,
+        (SELECT ARRAY_AGG(src) FROM (
+            SELECT DISTINCT UNNEST(ARRAY_AGG(mbedm.source)) AS src
+        ) ss) AS records,
+        (SELECT ARRAY_TO_STRING(ARRAY_AGG(authority), $$,$$) FROM (
+            SELECT DISTINCT UNNEST(ARRAY_AGG(mbedm.authority)) AS authority
+        ) au) AS authorities,
+        (SELECT ARRAY_TO_STRING(ARRAY_AGG(field), $$,$$) FROM (
+            SELECT DISTINCT UNNEST(ARRAY_AGG(mbedm.def)) AS field
+        ) fi) AS fields
+    FROM metabib.browse_entry mbe
+    JOIN metabib.browse_entry_def_map mbedm ON (
+        mbedm.entry = mbe.id AND
+        mbedm.def = ANY(' || quote_literal(search_field) || ')
+    )
+    WHERE ';
+
+    -- PostgreSQL is not magic. We can't actually pass a negative offset.
+    IF result_offset >= 0 AND NOT force_backward THEN
+        use_offset := result_offset;
+        core_query := core_query ||
+            ' mbe.sort_value >= ' || quote_literal(pivot_sort_value) ||
+        ' GROUP BY 1,2,3 ORDER BY mbe.sort_value, mbe.value ';
+
+        RETURN QUERY SELECT * FROM metabib.staged_browse(
+            core_query, context_org, context_locations,
+            staff, browse_superpage_size, result_limit, use_offset
+        );
+    ELSE
+        -- Part 1 of 2 to deliver what the user wants with a negative offset:
+        core_query := core_query ||
+            ' mbe.sort_value < ' || quote_literal(pivot_sort_value) ||
+        ' GROUP BY 1,2,3 ORDER BY mbe.sort_value DESC, mbe.value DESC ';
+
+        -- Part 2 of 2 to deliver what the user wants with a negative offset:
+        RETURN QUERY SELECT * FROM (SELECT * FROM metabib.staged_browse(
+            core_query, context_org, context_locations,
+            staff, browse_superpage_size, result_limit, use_offset
+        )) sb ORDER BY row_number DESC;
+
+    END IF;
+END;
+$p$ LANGUAGE PLPGSQL;
+
+CREATE OR REPLACE FUNCTION metabib.browse(
+    search_class        TEXT,
+    browse_term         TEXT,
+    context_org         INT DEFAULT NULL,
+    context_loc_group   INT DEFAULT NULL,
+    staff               BOOL DEFAULT FALSE,
+    pivot_id            BIGINT DEFAULT NULL,
+    force_backward      BOOL DEFAULT FALSE,
+    result_limit        INT DEFAULT 10,
+    result_offset       INT DEFAULT 0   -- Can be negative, implying backward!
+) RETURNS SETOF metabib.flat_browse_entry_appearance AS $p$
+BEGIN
+    RETURN QUERY SELECT * FROM metabib.browse(
+        (SELECT COALESCE(ARRAY_AGG(id), ARRAY[]::INT[])
+            FROM config.metabib_field WHERE field_class = search_class),
+        browse_term,
+        context_org,
+        context_loc_group,
+        staff,
+        pivot_id,
+        force_backward,
+        result_limit,
+        result_offset
+    );
+END;
+$p$ LANGUAGE PLPGSQL;
+
+UPDATE config.metabib_field
+SET
+    xpath = $$//mods32:mods/mods32:relatedItem[@type="series"]/mods32:titleInfo[@type="nfi"]$$,
+    browse_sort_xpath = $$*[local-name() != "nonSort"]$$,
+    browse_xpath = NULL
+WHERE
+    field_class = 'series' AND name = 'seriestitle' ;
+
+UPDATE config.metabib_field
+SET
+    xpath = $$//mods32:mods/mods32:titleInfo[mods32:title and not (@type)]$$,
+    browse_sort_xpath = $$*[local-name() != "nonSort"]$$,
+    browse_xpath = NULL,
+    browse_field = TRUE
+WHERE
+    field_class = 'title' AND name = 'proper' ;
+
+UPDATE config.metabib_field
+SET
+    xpath = $$//mods32:mods/mods32:titleInfo[mods32:title and (@type='alternative-nfi')]$$,
+    browse_sort_xpath = $$*[local-name() != "nonSort"]$$,
+    browse_xpath = NULL
+WHERE
+    field_class = 'title' AND name = 'alternative' ;
+
+UPDATE config.metabib_field
+SET
+    xpath = $$//mods32:mods/mods32:titleInfo[mods32:title and (@type='uniform-nfi')]$$,
+    browse_sort_xpath = $$*[local-name() != "nonSort"]$$,
+    browse_xpath = NULL
+WHERE
+    field_class = 'title' AND name = 'uniform' ;
+
+UPDATE config.metabib_field
+SET
+    xpath = $$//mods32:mods/mods32:titleInfo[mods32:title and (@type='translated-nfi')]$$,
+    browse_sort_xpath = $$*[local-name() != "nonSort"]$$,
+    browse_xpath = NULL
+WHERE
+    field_class = 'title' AND name = 'translated' ;
+
+-- This keeps extra terms like "creator" out of browse headings.
+UPDATE config.metabib_field
+    SET browse_xpath = $$//*[local-name()='namePart']$$     -- vim */
+    WHERE
+        browse_field AND
+        browse_xpath IS NULL AND
+        field_class = 'author';
+
 COMMIT;
+
+\qecho This is a browse-only reingest of your bib records. It may take a while.
+\qecho You may cancel now without losing the effect of the rest of the
+\qecho upgrade script, and arrange the reingest later.
+\qecho .
+SELECT metabib.reingest_metabib_field_entries(id, TRUE, FALSE, TRUE)
+    FROM biblio.record_entry;
index 220c56f..38f01fb 100644 (file)
@@ -8,9 +8,11 @@
     <div id="search-wrapper">
         <div id="search-box">
             <span class="search_catalog_lbl">[% l('Search the Catalog') %]</span>
-            <a href="[% mkurl(ctx.opac_root _ '/home') %]"
-                id="home_adv_search_link"><span
-                class="adv_search_font">[%l('Basic Search')%]</span></a>
+            <span><a href="[% mkurl(ctx.opac_root _ '/home') %]"
+                    id="home_adv_search_link">[%l('Basic Search')%]</a></span>
+
+            <span><a href="[% mkurl(ctx.opac_root _ '/browse') %]">[%
+                    l('Browse the Catalog')%]</a></span>
         </div>
         <div id="adv_search_parent">
             <div id="adv_search_tabs">
diff --git a/Open-ILS/src/templates/opac/browse.tt2 b/Open-ILS/src/templates/opac/browse.tt2
new file mode 100644 (file)
index 0000000..35a9d5a
--- /dev/null
@@ -0,0 +1,174 @@
+[%- # This is the bib and authority combined record browser.
+
+    PROCESS "opac/parts/header.tt2";
+    PROCESS "opac/parts/misc_util.tt2";
+    PROCESS "opac/parts/org_selector.tt2";
+    WRAPPER "opac/parts/base.tt2";
+    INCLUDE "opac/parts/topnav.tt2";
+
+    ctx.page_title = l("Browse the Catalog");
+    blimit = CGI.param('blimit') || 10;
+    boffset = CGI.param('boffset') || 0;
+
+    depart_list = ['blimit', 'bterm', 'boffset', 'bpivot', 'bback'];
+%]
+
+    <div id="search-wrapper">
+        [%# XXX TODO Give searchbar.tt2 more smarts so we can just do:
+          # INCLUDE "opac/parts/searchbar.tt2" %]
+        <div id="search-box">
+            <span class="search_catalog_lbl"><a href="[% mkurl(ctx.opac_root _ '/home', {}, depart_list) %]">[% l('Search the Catalog') %]</a></span>
+            <span><a href="[% mkurl(ctx.opac_root _ '/advanced', {}, depart_list) %]"
+                    id="home_adv_search_link">[%l('Advanced Search')%]</a></span>
+            <span>[% l('Browse the Catalog') %]</span>
+        </div>
+    </div>
+    <div id="content-wrapper">
+        <div id="main-content">
+            <div id="browse-the-catalog">
+                <div id="browse-controls">
+                    <form method="get">
+                        <input type="hidden" name="blimit"
+                            value="[% blimit %]" />
+
+                        [% control_qtype = INCLUDE "opac/parts/qtype_selector.tt2"
+                            id="browse-search-class" browse_only=1 %]
+
+                        [% control_bterm = BLOCK %]<input type="text" name="bterm" id="browse-term"
+                            value="[% CGI.param('bterm') | html %]" />[% END %]
+                        [% control_locg = INCLUDE build_org_selector id='browse-context'
+                            show_loc_groups=1
+                            arialabel=l('Select holding library') %]
+                        [% l('Browse by [_1] for [_2] held under [_3]', control_qtype, control_bterm, control_locg) %]
+
+                        <input type="submit" value="[% l('Go') %]" />
+                    </form>
+                </div>
+
+                [% BLOCK browse_pager %]
+                <div class="browse-pager">
+                    [% IF ctx.more_back %]
+                    <a class="opac-button" href="[% mkurl('', {bpivot => ctx.back_pivot, bback => 1}) %]">&larr; [%l ('Back') %]</a>
+                    [% END %]
+                    [% IF browse.english_pager; # XXX how to apply i18n here?
+                        current_qtype = CGI.param('qtype') || 'title' %]
+                    <span class="browse-shortcuts">
+                        <a href="[% mkurl('', {qtype => current_qtype, bterm => '0'}, ['boffset','bpivot','bback']) %]">0-9</a>
+                        [% FOR letter IN ['A'..'Z'] %]
+                            <a href="[% mkurl('', {qtype => current_qtype, bterm => letter}, ['boffset','bpivot','bback']) %]">[% letter %]</a>
+                        [% END %]
+                    </span>
+                    [% END %]
+
+                    [% IF ctx.more_forward %]
+                    <a class="opac-button" href="[% mkurl('', {bpivot => ctx.forward_pivot}, ['bback']) %]">[%l ('Forward') %] &rarr;</a>
+                    [% END %]
+                </div>
+                [% END %]
+
+                [% PROCESS browse_pager %]
+
+                <div id="browse-results">
+                [% IF ctx.browse_error %]
+                    <span class="browse-error">
+                        [% l("An error occurred browsing records. " _
+                        "Please try again in a moment or report the issue " _
+                        "to library staff.") %]
+                    </span>
+                [% ELSE %]
+                    [% IF ctx.browse_leading_article_warning %]
+                    <div class="browse-leading-article-warning">
+                            [% l("Your browse term seems to begin with an article. You might get better results by omitting the article.") %]
+                    </div>
+                    [% END %]
+                    <ul class="browse-result-list">
+                    [% FOR result IN ctx.browse_results %]
+                        <li class="browse-result">
+                            <span class="browse-result-value">
+                                <a href="[% mkurl(
+                                    ctx.opac_root _ '/results', {
+                                        'fi:has_browse_entry' => (result.browse_entry _ ',' _ result.fields)
+                                    }) %]">[% result.value | html %]</a>
+                            </span>
+                            <span class="browse-result-sources">([%
+                                IF result.accurate == 'f';
+                                    l("At least"); " ";
+                                END;
+                                result.sources %])</span>
+                            [% IF result.authorities.size %]
+                            <ul class="browse-result-authority-headings">
+                                [% FOR a IN result.authorities;
+                                    PROCESS authority_notes authority=a;
+
+                                    # Other than displaying public general
+                                    # notes, we can go no further sans
+                                    # control_set.
+                                    NEXT UNLESS a.control_set;
+
+                                    # get_authority_fields is fast and cache-y.
+                                    acs = ctx.get_authority_fields(a.control_set);
+                                    FOR field_group IN a.headings;
+                                        field_id = field_group.keys.0;
+                                        field = acs.$field_id;
+                                        headings = field_group.values.0;
+                                        FOR h IN headings;
+                                            # We could display headings without
+                                            # links here when h.target is
+                                            # undef, if we wanted to, but note
+                                            # that h.target_count is only
+                                            # defined when h.target is.
+
+                                            IF h.target %]
+                                            <li><span class="browse-result-authority-field-name">[% field.name %]</span>
+                                            <a href="[% mkurl(ctx.opac_root _ '/results', {query => 'identifier|authority_id[' _ h.target _ ']'}) %]">[% h.heading | html %]</a>
+                                            <span class="browse-result-authority-bib-links">([% h.target_count %])</span>
+                                            </li>
+                                            [% END %]
+                                        [% END %]
+                                    [% END %]
+                                [% END %]
+                            </ul>
+                            [% END %]
+                        </li>
+                    [% END %]
+                    </ul>
+                [% END %]
+                </div>
+
+                [% PROCESS browse_pager %]
+            </div>
+
+            <div class="common-full-pad"></div>        
+        </div>
+    </div>
+
+    [% BLOCK authority_notes;
+        # Displays public general notes (sometimes called "scope notes" ?)
+        FOR note IN authority.notes %]
+            <div class="browse-public-general-note">
+                <span class="browse-public-general-note-label">
+                    [% l("Note:") %]
+                </span>
+                <span class="browse-public-general-note-body">
+            [% FOR piece IN note;
+                IF piece.heading;
+                    mkurl_args = {bterm => piece.bterm};
+                    IF piece.org_id;
+                        mkurl_args.locg = piece.org_id;
+                    END;
+                %]
+                <a href="[% mkurl('', mkurl_args, ['boffset','bpivot','bback']) %]">[% piece.heading | html %]</a>
+                [% ELSIF piece.institution %]
+                <span class="browse-public-general-note-institution">
+                    [% piece.institution | html %]
+                </span>
+                [% ELSE %]
+                    [% piece | html %]
+                [% END;
+            END %]
+                </span>
+            </div>
+        [% END;
+    END;    # end of BLOCK authority_notes %]
+
+[% END %]
index 00d90d9..3efc864 100644 (file)
@@ -877,10 +877,6 @@ table.acct_notes th {
     padding-right: 5px;
 }
 
-.adv_search_font {
-    font-size: [% css_fonts.size_smaller %];
-}
-
 .search_catalog_lbl {
     font-size: [% css_fonts.size_bigger %];
 }
@@ -1536,3 +1532,42 @@ a.preflib_change {
     color: [% css_colors.text_invert %];
     text-align: center;
 }
+
+#search-box > span {
+    margin: 0 1em;
+}
+.browse-error {
+    font-weight: bold;
+    font-color: #c00;
+}
+.browse-result-sources, .browse-result-authority-bib-links {
+    margin-left: 1em;
+}
+.browse-pager {
+    margin: 2ex 0;
+}
+.browse-result-list {
+    list-style-type: square;
+}
+.browse-shortcuts {
+    font-size: 120%;
+}
+.browse-result-authority-field-name {
+    font-style: italic;
+    margin-right: 1em;
+}
+.browse-leading-article-warning {
+    font-style: italic;
+    font-size: 110%;
+}
+.browse-public-general-note {
+    font-size: 110%;
+}
+.browse-public-general-note-label { }
+.browse-public-general-note-institution {
+    font-style: normal;
+    font-weight: bold;
+}
+.browse-public-general-note-body {
+    font-style: italic;
+}
index 1ba05cc..a0be85b 100644 (file)
@@ -150,8 +150,17 @@ search.basic_config = {
 ctx.google_books_preview = 0;
 
 ##############################################################################
+
 # Set a maintenance message to display in the catalogue
 #
 # ctx.maintenance_message = "The system will not be available February 29, 2104.";
 
+# Browse settings
+# Set to 1 or 'true' to enable.  This controls whether or not the
+# "0-9 A B C D ..." links appear on the browse page.  We don't yet have a
+# serviceable way to internationalize these links, so sites must choose to
+# turn on this feature.
+
+browse.english_pager = 0;
+
 %]
index eda10a3..30edbc6 100644 (file)
@@ -1,16 +1,17 @@
 [%  query_types = [
     {value => "keyword", label => l("Keyword")},
-    {value => "title", label => l("Title")},
+    {value => "title", label => l("Title"), browse => 1},
     {value => "jtitle", label => l("Journal Title")},
-    {value => "author", label => l("Author")},
-    {value => "subject", label => l("Subject")},
-    {value => "series", label => l("Series")},
+    {value => "author", label => l("Author"), browse => 1},
+    {value => "subject", label => l("Subject"), browse => 1},
+    {value => "series", label => l("Series"), browse => 1},
     {value => "id|bibcn", label => l("Bib Call Number")}
 ] %]
-<select name="qtype"[% IF id; ' id="'; id ; '"' ; END -%]
+<select name="[% name || 'qtype' %]"[% IF id; ' id="'; id ; '"' ; END -%]
     aria-label="[% l('Select query type:') %]">
     [%  query_type = query_type || CGI.param('qtype') || search.default_qtypes.0;
-        FOR qt IN query_types -%]
+      FOR qt IN query_types;
+        NEXT IF browse_only AND NOT qt.browse -%]
     <option value='[% qt.value | html %]'[%
         query_type == qt.value ? ' selected="selected"' : ''
     %]>[% qt.label | html %]</option>
index 0359c83..84d6008 100644 (file)
@@ -1,13 +1,13 @@
 [% PROCESS "opac/parts/org_selector.tt2" %]
-<div id="search-box">    
+<div id="search-wrapper">
     [% UNLESS took_care_of_form -%]
     <form action="[% ctx.opac_root %]/results" method="get">
     [%- END %]
-    <div>
+    <div id="search-box">
         <span class="search_catalog_lbl">[% l('Search the Catalog') %]</span>
         <a href="[% mkurl(ctx.opac_root _ '/advanced') %]"
-            id="home_adv_search_link"><span
-            class="adv_search_font">[% l('Advanced Search') %]</span></a>
+            id="home_adv_search_link">[% l('Advanced Search') %]</a>
+        <span class="browse_the_catalog_lbl"><a href="[% mkurl(ctx.opac_root _ '/browse', {}, ['fi:has_browse_entry']) %]">[% l('Browse the Catalog') %]</a></span>
     </div>
     <div class="searchbar">[%- l('Search ');
         IF search.basic_config.type == 'attr';
index 88f3284..dfe8a6f 100644 (file)
@@ -19,7 +19,6 @@
 #new_cat_link_holder a { display: block; width: 675px; height: 213px; }
 .pos-rel { position: relative; }
 #search-box table { position: relative; left: -10px; }
-#home_adv_search_link { position: relative; top: -1px; left: 10px; }
 #util_back_btn { position: relative; top: 1px; left: 10px; }
 #util_help_btn { position: relative; top: 2px; left: 40px; }
 #util_forw_btn { position: relative; top: 2px; left: 50px; }
diff --git a/docs/RELEASE_NOTES_NEXT/OPAC/BibAuthBrowse.txt b/docs/RELEASE_NOTES_NEXT/OPAC/BibAuthBrowse.txt
new file mode 100644 (file)
index 0000000..279bfbd
--- /dev/null
@@ -0,0 +1,45 @@
+Bib record browser with linked authorities
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+This feature provides a patron-oriented OPAC interface for browsing
+bibliographic records.
+
+Users choose to browse by Author, Title, Subject, or Series. They then
+enter a browse term, and the nearest match from a left-anchored search
+on the headings extracted for browse purposes will be displayed in a
+typical backwards/forwards paging display. Headings link to search
+results pages showing the related records. If the browse heading is
+linked to any authority records, and if any *other* authority records
+point to those with "See also" or other non-main entry headings, those
+alternative headings are displayed a linked to a search results page
+showing related bib records related to the alternate heading.
+
+The counts of holdings displayed next to headings from bibliographic
+records are subject to the same visiibility tests as search. This means
+that the org unit (and copy location group) dropdown on the browse
+interface affects counds, and it further means that whether or not
+you're looking at the browse interface through the staff client makes a
+difference.
+
+Configuration considerations for site administrators
+++++++++++++++++++++++++++++++++++++++++++++++++++++
+There are two off-by-default features that site administrators may wish
+to enable.
+
+  * Quick paging links (English): By changing the
+    ''browse.english_pager'' setting to 1 in the
+    ''opac/parts/config.tt2'' file for a site's active OPAC templates,
+    you can make shortcut browsing links ''0-9 A B C D ...'' appear
+    between the Back and Forward buttons on the browse page. I haven't
+    figured out how to make this feature internationalizable, so it's
+    off by default.  You can turn it on if it works for your language,
+    or have a look at improving it if it doesn't.
+
+  * There is a global flag by the name
+    ''opac.browse.warnable_regexp_per_class'' to control what leading
+    articles in users' entered browse terms trigger a warning about how
+    it might be better to search for "Rolling Stones" instead of "The
+    Rolling Stones" (or whatever). This is off by default, but can be
+    enabled if it suits your catalog, and can even be customized per
+    search class (author, title, series, subject).
+