Link checker: user interface and supporting fixes (part 1)
authorLebbeous Fogle-Weekley <lebbeous@esilibrary.com>
Fri, 17 Aug 2012 16:17:00 +0000 (12:17 -0400)
committerMike Rylander <mrylander@gmail.com>
Thu, 14 Feb 2013 19:19:17 +0000 (14:19 -0500)
Add open-ils.url_verify service to example OpenSRF configs
ML methods to create sessions and do the searching/bucketing
    We can't use PCRUD to create url_verify.session objects because a) you
    couldn't trust the creator field if we allowed that, and b) the
    container foreign key has a not-null constraint, so you have to create
    that first, and you can't do that with PCRUD.
    I've removed the C, U and D perms for PCRUD for url_verify.session, but
    I left the R in case we wind up using that.
Beginnings for the big session kick-off UI.  Not yet functional.
Get all search results, not just first 10
Check for session ownership and for previous searchitude
Deal with moved publish_fieldmapper() method
    This is a companion commit to
    fac45ab9b1cb8924 / Move Fieldmapper API call to Application.pm
    Without it, Flattener and Action/Trigger stop working with errors like
    this:
    [Mon Aug 20 13:50:18 2012] [error] [client XXX.XXX.XXX.XXX] Exception:
    OpenSRF::EX::ERROR 2012-08-20T13:50:18 main -e:0 System ERROR:
    Exception: OpenSRF::DomainObject::oilsMethodException
    2012-08-20T13:50:18 OpenSRF::AppRequest
    /usr/local/share/perl/5.10.1/OpenSRF/AppSession.pm:1064 <500>   *** Call
    to [open-ils.fielder.flattened_search.execute.atomic] failed for session
    [1345485018.767884163.96534353976], thread trace [1]:\nNo field by the
    name publish_fieldmapper in Fieldmapper! at
    /usr/local/share/perl/5.10.1/OpenILS/Utils/Fieldmapper.pm line
    270.\n\n\n\n, referer:
    http://XXXXXXX/eg/conify/global/actor/search_filter_group
Use a perm that actually exists
More UI work. Saved search selector & search scope OU selector & cosmetics
Fix subtle Perl issue
    Not a syntax error that the compiler will catch, but see
    "perldoc -f do" which will lead you do "perldoc perlsyn"
Buckets and their items aren't designed to be PCRUD accessible,
    so we need a handy view to link URL Verify Sessions to the bib
    contained.  We can leverage this in flattener queries.
Pretty much finished session create UI but for cloning
Permisison fixing
whitespace
Fix previously nonfunctional stored procedure url_verify.extract_urls(INT,INT)
Call URL extraction phase from UI
Fix xpath generation to match what works
Various fixes, largely UI
Refactor create_session as dojo module.
Fix IDL permissions that require jumps
Essentials for URL selecting interface
Verification sorta works
A note about open-ils.url_verify.verify_url for future reference

Signed-off-by: Lebbeous Fogle-Weekley <lebbeous@esilibrary.com>
Signed-off-by: Mike Rylander <mrylander@gmail.com>
17 files changed:
Open-ILS/examples/fm_IDL.xml
Open-ILS/examples/opensrf.xml.example
Open-ILS/examples/opensrf_core.xml.example
Open-ILS/src/extras/ils_events.xml
Open-ILS/src/perlmods/lib/OpenILS/Application/Flattener.pm
Open-ILS/src/perlmods/lib/OpenILS/Application/Trigger.pm
Open-ILS/src/perlmods/lib/OpenILS/Application/Trigger/Event.pm
Open-ILS/src/perlmods/lib/OpenILS/Application/URLVerify.pm
Open-ILS/src/sql/Pg/076.functions.url_verify.sql
Open-ILS/src/sql/Pg/upgrade/YYYY.functions.url_verify.sql
Open-ILS/src/templates/url_verify/create_session.tt2 [new file with mode: 0644]
Open-ILS/src/templates/url_verify/select_urls.tt2 [new file with mode: 0644]
Open-ILS/web/js/dojo/openils/URLVerify/CreateSession.js [new file with mode: 0644]
Open-ILS/web/js/dojo/openils/URLVerify/SelectURLs.js [new file with mode: 0644]
Open-ILS/web/js/dojo/openils/URLVerify/nls/URLVerify.js [new file with mode: 0644]
Open-ILS/web/js/dojo/openils/widget/FilteringTreeSelect.js
Open-ILS/web/js/dojo/openils/widget/FlattenerGrid.js

index c774122..7726438 100644 (file)
@@ -9366,15 +9366,43 @@ SELECT  usr,
 
         <permacrud xmlns="http://open-ils.org/spec/opensrf/IDL/permacrud/v1">
             <actions>
-                <create permission="ADMIN_URL_VERIFY" context_field="owning_lib"/>
-                <retrieve permission="ADMIN_URL_VERIFY" context_field="owning_lib"/>
-                <update permission="ADMIN_URL_VERIFY" context_field="owning_lib"/>
-                <delete permission="ADMIN_URL_VERIFY" context_field="owning_lib"/>
+                <retrieve permission="URL_VERIFY" context_field="owning_lib"/>
             </actions>
         </permacrud>
 
     </class>
 
+    <class id="uvsbrem" controller="open-ils.cstore open-ils.pcrud" oils_obj:fieldmapper="url_verify::session_biblio_record_entry_map" oils_persist:readonly="true" reporter:label="URL Verify Session Biblio Record Entry Map">
+        <oils_persist:source_definition>
+            SELECT
+                cbrebi.id AS id,  -- so we can have a pkey in our view
+                uvs.id AS session,
+                uvs.owning_lib,
+                cbrebi.target_biblio_record_entry
+            FROM url_verify.session uvs
+            JOIN container.biblio_record_entry_bucket cbreb
+                ON (uvs.container = cbreb.id)
+            JOIN container.biblio_record_entry_bucket_item cbrebi
+                ON (cbrebi.bucket = cbreb.id)
+        </oils_persist:source_definition>
+        <fields oils_persist:primary="id" oils_persist:sequence="container.biblio_record_entry_bucket_item_id_seq">
+            <field reporter:label="Bucket Item ID" name="id" reporter:datatype="id" />
+            <field reporter:label="Session" name="session" reporter:datatype="link" />
+            <field reporter:label="Owning Library" name="owning_lib" reporter:datatype="org_unit" />
+            <field reporter:label="Target Biblio Record Entry" name="target_biblio_record_entry" reporter:datatype="link" />
+        </fields>
+        <links>
+            <link field="target_biblio_record_entry" reltype="has_a" key="id" map="" class="bre" />
+            <link field="session" reltype="has_a" key="id" map="" class="uvs" />
+            <link field="owning_lib" reltype="has_a" key="id" map="" class="aou" />
+        </links>
+        <permacrud xmlns="http://open-ils.org/spec/opensrf/IDL/permacrud/v1">
+            <actions>
+                <retrieve permission="URL_VERIFY" context_field="owning_lib" />
+            </actions>
+        </permacrud>
+    </class>
+
     <class
         id="uvus"
         controller="open-ils.cstore open-ils.pcrud"
@@ -9396,16 +9424,16 @@ SELECT  usr,
 
         <permacrud xmlns="http://open-ils.org/spec/opensrf/IDL/permacrud/v1">
             <actions>
-                <create permission="ADMIN_URL_VERIFY">
+                <create permission="URL_VERIFY">
                     <context link="session" field="owning_lib"/>
                 </create>
-                <retrieve permission="ADMIN_URL_VERIFY">
+                <retrieve permission="URL_VERIFY">
                     <context link="session" field="owning_lib"/>
                 </retrieve>
-                <update permission="ADMIN_URL_VERIFY">
+                <update permission="URL_VERIFY">
                     <context link="session" field="owning_lib"/>
                 </update>
-                <delete permission="ADMIN_URL_VERIFY">
+                <delete permission="URL_VERIFY">
                     <context link="session" field="owning_lib"/>
                 </delete>
             </actions>
@@ -9441,23 +9469,23 @@ SELECT  usr,
 
         <links>
             <link field="redirect_from" reltype="has_a" key="id" map="" class="uvu"/>
-            <link field="item" reltype="has_a" key="id" map="" class="cbrebi"/>
+            <link field="item" reltype="has_a" key="id" map="" class="uvsbrem" /><!-- surprise! -->
             <link field="url_selector" reltype="has_a" key="id" map="" class="uvus"/>
         </links>
 
         <permacrud xmlns="http://open-ils.org/spec/opensrf/IDL/permacrud/v1">
             <actions>
-                <create permission="ADMIN_URL_VERIFY">
-                    <context link="url_selector session" field="owning_lib"/>
+                <create permission="URL_VERIFY">
+                    <context link="url_selector" jump="session" field="owning_lib"/>
                 </create>
-                <retrieve permission="ADMIN_URL_VERIFY">
-                    <context link="url_selector session" field="owning_lib"/>
+                <retrieve permission="URL_VERIFY">
+                    <context link="url_selector" jump="session" field="owning_lib"/>
                 </retrieve>
-                <update permission="ADMIN_URL_VERIFY">
-                    <context link="url_selector session" field="owning_lib"/>
+                <update permission="URL_VERIFY">
+                    <context link="url_selector" jump="session" field="owning_lib"/>
                 </update>
-                <delete permission="ADMIN_URL_VERIFY">
-                    <context link="url_selector session" field="owning_lib"/>
+                <delete permission="URL_VERIFY">
+                    <context link="url_selector" jump="session" field="owning_lib"/>
                 </delete>
             </actions>
         </permacrud>
@@ -9486,16 +9514,16 @@ SELECT  usr,
 
         <permacrud xmlns="http://open-ils.org/spec/opensrf/IDL/permacrud/v1">
             <actions>
-                <create permission="ADMIN_URL_VERIFY">
+                <create permission="URL_VERIFY">
                     <context link="session" field="owning_lib"/>
                 </create>
-                <retrieve permission="ADMIN_URL_VERIFY">
+                <retrieve permission="URL_VERIFY">
                     <context link="session" field="owning_lib"/>
                 </retrieve>
-                <update permission="ADMIN_URL_VERIFY">
+                <update permission="URL_VERIFY">
                     <context link="session" field="owning_lib"/>
                 </update>
-                <delete permission="ADMIN_URL_VERIFY">
+                <delete permission="URL_VERIFY">
                     <context link="session" field="owning_lib"/>
                 </delete>
             </actions>
@@ -9529,17 +9557,17 @@ SELECT  usr,
 
         <permacrud xmlns="http://open-ils.org/spec/opensrf/IDL/permacrud/v1">
             <actions>
-                <create permission="ADMIN_URL_VERIFY">
-                    <context link="attempt session" field="owning_lib"/>
+                <create permission="URL_VERIFY">
+                    <context link="attempt" jump="session" field="owning_lib"/>
                 </create>
-                <retrieve permission="ADMIN_URL_VERIFY">
-                    <context link="attempt session" field="owning_lib"/>
+                <retrieve permission="URL_VERIFY">
+                    <context link="attempt" jump="session" field="owning_lib"/>
                 </retrieve>
-                <update permission="ADMIN_URL_VERIFY">
-                    <context link="attempt session" field="owning_lib"/>
+                <update permission="URL_VERIFY">
+                    <context link="attempt" jump="session" field="owning_lib"/>
                 </update>
-                <delete permission="ADMIN_URL_VERIFY">
-                    <context link="attempt session" field="owning_lib"/>
+                <delete permission="URL_VERIFY">
+                    <context link="attempt" jump="session" field="owning_lib"/>
                 </delete>
             </actions>
         </permacrud>
@@ -9569,10 +9597,10 @@ SELECT  usr,
 
         <permacrud xmlns="http://open-ils.org/spec/opensrf/IDL/permacrud/v1">
             <actions>
-                <create permission="ADMIN_URL_VERIFY" context_field="owning_lib"/>
-                <retrieve permission="ADMIN_URL_VERIFY" context_field="owning_lib"/>
-                <update permission="ADMIN_URL_VERIFY" context_field="owning_lib"/>
-                <delete permission="ADMIN_URL_VERIFY" context_field="owning_lib"/>
+                <create permission="URL_VERIFY" context_field="owning_lib"/>
+                <retrieve permission="URL_VERIFY" context_field="owning_lib"/>
+                <update permission="URL_VERIFY" context_field="owning_lib"/>
+                <delete permission="URL_VERIFY" context_field="owning_lib"/>
             </actions>
         </permacrud>
 
index be398d6..64527ef 100644 (file)
@@ -700,6 +700,26 @@ vim:et:ts=4:sw=4:
                 </app_settings>
             </open-ils.trigger>
 
+            <open-ils.url_verify>
+                <keepalive>5</keepalive>
+                <stateless>1</stateless>
+                <language>perl</language>
+                <implementation>OpenILS::Application::URLVerify</implementation>
+                <max_requests>199</max_requests>
+                <unix_config>
+                    <unix_sock>open-ils.url_verify_unix.sock</unix_sock>
+                    <unix_pid>open-ils.url_verify_unix.pid</unix_pid>
+                    <max_requests>1000</max_requests>
+                    <unix_log>open-ils.url_verify_unix.log</unix_log>
+                    <min_children>1</min_children>
+                    <max_children>15</max_children>
+                    <min_spare_children>1</min_spare_children>
+                    <max_spare_children>5</max_spare_children>
+                </unix_config>
+                <app_settings>
+                </app_settings>
+            </open-ils.url_verify>
+
             <opensrf.math>
                 <keepalive>3</keepalive>
                 <stateless>1</stateless>
@@ -1264,6 +1284,7 @@ vim:et:ts=4:sw=4:
                 <appname>open-ils.permacrud</appname>  
                 <appname>open-ils.pcrud</appname>  
                 <appname>open-ils.trigger</appname>  
+                <appname>open-ils.url_verify</appname>
                 <appname>open-ils.fielder</appname>  
                 <appname>open-ils.vandelay</appname>  
                 <appname>open-ils.serial</appname>  
index 7bc022f..39ddbf8 100644 (file)
@@ -34,6 +34,7 @@ Example OpenSRF bootstrap configuration file for Evergreen
           <service>open-ils.resolver</service>
           <service>open-ils.search</service>
           <service>open-ils.supercat</service>
+          <service>open-ils.url_verify</service>
           <service>open-ils.vandelay</service>
           <service>open-ils.serial</service>
         </services>
index 1be90e9..1c458ad 100644 (file)
                <desc xml:lang="en-US">Attempt to suspend a hold after it has been captured.</desc>
        </event>
 
+       <event code='1900' textcode='URL_VERIFY_NOT_SESSION_CREATOR'>
+               <desc xml:lang="en-US">You did not create this URL Verify session, so you cannot change it.  You may be able to clone it.</desc>
+       </event>
+
+       <event code='1901' textcode='URL_VERIFY_SESSION_ALREADY_SEARCHED'>
+               <desc xml:lang="en-US">This session has already been searched.</desc>
+       </event>
 
        <event code='2000' textcode='BAD_PARAMS'>
                <desc xml:lang="en-US">Invalid parameters were encountered in a method</desc>
index 8bc8eda..c967247 100644 (file)
@@ -20,7 +20,7 @@ $Data::Dumper::Indent = 0;
 sub _fm_link_from_class {
     my ($class, $field) = @_;
 
-    return Fieldmapper->publish_fieldmapper->{$class}{links}{$field};
+    return OpenILS::Application->publish_fieldmapper->{$class}{links}{$field};
 }
 
 sub _flattened_search_single_flesh_wad {
index 2f28416..24488e8 100644 (file)
@@ -369,15 +369,15 @@ __PACKAGE__->register_method(
  
 sub _fm_hint_by_class {
     my $class = shift;
-    return Fieldmapper->publish_fieldmapper->{$class}->{hint};
+    return OpenILS::Application->publish_fieldmapper->{$class}->{hint};
 }
 
 sub _fm_class_by_hint {
     my $hint = shift;
 
     my ($class) = grep {
-        Fieldmapper->publish_fieldmapper->{$_}->{hint} eq $hint
-    } keys %{ Fieldmapper->publish_fieldmapper };
+        OpenILS::Application->publish_fieldmapper->{$_}->{hint} eq $hint
+    } keys %{ OpenILS::Application->publish_fieldmapper };
 
     return $class;
 }
index 1aca73e..0f52b73 100644 (file)
@@ -484,8 +484,8 @@ sub _fm_class_by_hint {
     my $hint = shift;
 
     my ($class) = grep {
-        Fieldmapper->publish_fieldmapper->{$_}->{hint} eq $hint
-    } keys %{ Fieldmapper->publish_fieldmapper };
+        OpenILS::Application->publish_fieldmapper->{$_}->{hint} eq $hint
+    } keys %{ OpenILS::Application->publish_fieldmapper };
 
     return $class;
 }
@@ -530,15 +530,15 @@ sub _object_by_path {
 
     my $step = shift(@$path);
 
-    my $fhint = Fieldmapper->publish_fieldmapper->{$context->class_name}{links}{$step}{class};
+    my $fhint = OpenILS::Application->publish_fieldmapper->{$context->class_name}{links}{$step}{class};
     my $fclass = $self->_fm_class_by_hint( $fhint );
 
     OpenSRF::EX::ERROR->throw(
         "$step is not a field on ".$context->class_name."  Please repair the environment.")
         unless $fhint;
 
-    my $ffield = Fieldmapper->publish_fieldmapper->{$context->class_name}{links}{$step}{key};
-    my $rtype = Fieldmapper->publish_fieldmapper->{$context->class_name}{links}{$step}{reltype};
+    my $ffield = OpenILS::Application->publish_fieldmapper->{$context->class_name}{links}{$step}{key};
+    my $rtype = OpenILS::Application->publish_fieldmapper->{$context->class_name}{links}{$step}{reltype};
 
     my $meth = 'retrieve_';
     my $multi = 0;
@@ -572,7 +572,7 @@ sub _object_by_path {
             $obj = $_object_by_path_cache{$def_id}{$str_path}{$step}{$ffield}{$lval} ||
                 (
                     (grep /cstore/, @{
-                        Fieldmapper->publish_fieldmapper->{$fclass}{controller}
+                        OpenILS::Application->publish_fieldmapper->{$fclass}{controller}
                     }) ? $ed : ($red ||= new_rstore_editor(xact=>1))
                 )->$meth( ($multi) ? { $ffield => $lval } : $lval);
 
index 4e86dcd..86d95c3 100644 (file)
@@ -1,4 +1,7 @@
 package OpenILS::Application::URLVerify;
+
+# For code searchability, I'm telling you this is the "link checker."
+
 use base qw/OpenILS::Application/;
 use strict; use warnings;
 use OpenSRF::Utils::Logger qw(:logger);
@@ -8,12 +11,16 @@ use OpenILS::Utils::CStoreEditor q/:funcs/;
 use OpenILS::Application::AppUtils;
 use LWP::UserAgent;
 
+use Data::Dumper;
+
+$Data::Dumper::Indent = 0;
+
 my $U = 'OpenILS::Application::AppUtils';
 
 
 __PACKAGE__->register_method(
-    method => 'validate_session',
-    api_name => 'open-ils.url_verify.session.validate',
+    method => 'verify_session',
+    api_name => 'open-ils.url_verify.session.verify',
     stream => 1,
     signature => {
         desc => q/
@@ -22,7 +29,7 @@ __PACKAGE__->register_method(
         params => [
             {desc => 'Authentication token', type => 'string'},
             {desc => 'Session ID (url_verify.session.id)', type => 'number'},
-            {desc => 'URL ID list (optional).  An empty list will result in no URLs being processed', type => 'array'},
+            {desc => 'URL ID list (optional).  An empty list will result in no URLs being processed, but null will result in all the URLs for the session being processed', type => 'array'},
             {
                 desc => q/
                     Options (optional).
@@ -51,13 +58,16 @@ __PACKAGE__->register_method(
     }
 );
 
-sub validate_session {
+# "verify_session" sounds like something to do with authentication, but it
+# actually means for a given session, verify all the URLs associated with
+# that session.
+sub verify_session {
     my ($self, $client, $auth, $session_id, $url_ids, $options) = @_;
     $options ||= {};
 
     my $e = new_editor(authtoken => $auth, xact => 1);
     return $e->die_event unless $e->checkauth;
-    return $e->die_event unless $e->allowed('VERIFY_URL');
+    return $e->die_event unless $e->allowed('URL_VERIFY');
 
     my $session = $e->retrieve_url_verify_session($session_id)
         or return $e->die_event;
@@ -142,6 +152,7 @@ sub validate_session {
         $e->create_url_verify_verification_attempt($attempt)
             or return $e->die_event;
 
+        $attempt = $e->data;
         $e->commit;
     }
 
@@ -153,7 +164,8 @@ sub validate_session {
         $session->owning_lib,
         'url_verify.verification_batch_size', $e) || 5;
 
-    my $num_processed = 0; # total number processed, including redirects
+    my $total_excluding_redirects = 0;
+    my $total_processed = 0; # total number processed, including redirects
     my $resp_window = 1;
 
     # before we start the real work, let the caller know
@@ -161,7 +173,8 @@ sub validate_session {
 
     $client->respond({
         url_count => $url_count,
-        total_processed => $num_processed,
+        total_processed => $total_processed,
+        total_excluding_redirects => $total_excluding_redirects,
         attempt => $attempt
     });
 
@@ -181,19 +194,20 @@ sub validate_session {
 
                 if ($content) {
 
-                    $num_processed++;
+                    $total_processed++;
 
-                    if ($options->{report_all} or ($num_processed % $resp_window == 0)) {
+                    if ($options->{report_all} or ($total_processed % $resp_window == 0)) {
                         $client->respond({
                             url_count => $url_count,
                             current_verification => $content,
-                            total_processed => $num_processed
+                            total_excluding_redirects => $total_excluding_redirects,
+                            total_processed => $total_processed
                         });
                     }
 
                     # start off responding quickly, then throttle
                     # back to only relaying every 256 messages.
-                    $resp_window *= 2 unless $resp_window == 256;
+                    $resp_window *= 2 unless $resp_window >= 256;
                 }
             }
         },
@@ -206,7 +220,9 @@ sub validate_session {
         }
     );
 
-    sort_and_fire_domains($e, $auth, $attempt, $url_ids, $multises);
+    sort_and_fire_domains(
+        $e, $auth, $attempt, $url_ids, $multises, \$total_excluding_redirects
+    );
 
     # Wait for all requests to be completed
     $multises->session_wait(1);
@@ -215,24 +231,36 @@ sub validate_session {
     $attempt->finish_time('now');
 
     $e->xact_begin;
-    $e->update_url_verify_verification_attempt($attempt) or return $e->die_event;
+    $e->update_url_verify_verification_attempt($attempt) or
+        return $e->die_event;
+
     $e->xact_commit;
 
+    # This way the caller gets an actual timestamp in the "finish_time" field
+    # instead of the string "now".
+    $attempt = $e->retrieve_url_verify_verification_attempt($e->data) or
+        return $e->die_event;
+
+    $e->disconnect;
+
     return {
         url_count => $url_count,
-        total_processed => $num_processed,
+        total_processed => $total_processed,
+        total_excluding_redirects => $total_excluding_redirects,
         attempt => $attempt
     };
 }
 
-# retrieves the URL domains and sorts them into buckets
+# retrieves the URL domains and sorts them into buckets*
 # Iterates over the buckets and fires the multi-session call
 # the main drawback to this domain sorting approach is that
 # any domain used a lot more than the others will be the
 # only domain standing after the others are exhausted, which
 # means it will take a beating at the end of the batch.
+#
+# * local data structures, not container.* buckets
 sub sort_and_fire_domains {
-    my ($e, $auth, $attempt, $url_ids, $multises) = @_;
+    my ($e, $auth, $attempt, $url_ids, $multises, $count) = @_;
 
     # there is potential here for data sets to be too large
     # for delivery, but it's not likely, since we're only
@@ -263,11 +291,16 @@ sub sort_and_fire_domains {
             $multises->request(
                 'open-ils.url_verify.verify_url',
                 $auth, $attempt->id, $url_id);
+            
+            $$count++;  # sic, a reference to a scalar
         }
     }
 }
 
 
+# XXX I really want to move this method to open-ils.storage, so we don't have
+# to authenticate a zillion times. LFW
+
 __PACKAGE__->register_method(
     method => 'verify_url',
     api_name => 'open-ils.url_verify.verify_url',
@@ -316,7 +349,7 @@ sub verify_url {
         collect_verify_attempt_and_settings($e, $attempt_id);
 
     return $e->event unless $e->allowed(
-        'VERIFY_URL', $attempt->session->owning_lib);
+        'URL_VERIFY', $attempt->session->owning_lib);
 
     my $cur_url = $url;
     my $loop_detected = 0;
@@ -584,4 +617,233 @@ sub verify_one_url {
 }
 
 
+__PACKAGE__->register_method(
+    method => "create_session",
+    api_name => "open-ils.url_verify.session.create",
+    signature => {
+        desc => q/Create a URL verify session. Also automatically create and
+            link a container./,
+        params => [
+            {desc => "Authentication token", type => "string"},
+            {desc => "session name", type => "string"},
+            {desc => "QueryParser search", type => "string"},
+            {desc => "owning_lib (defaults to ws_ou)", type => "number"},
+        ],
+        return => {desc => "ID of new session or event on error", type => "number"}
+    }
+);
+
+sub create_session {
+    my ($self, $client, $auth, $name, $search, $owning_lib) = @_;
+
+    my $e = new_editor(authtoken => $auth, xact => 1);
+    return $e->die_event unless $e->checkauth;
+
+    $owning_lib ||= $e->requestor->ws_ou;
+    return $e->die_event unless $e->allowed("URL_VERIFY", $owning_lib);
+
+    my $session = Fieldmapper::url_verify::session->new;
+    $session->name($name);
+    $session->owning_lib($owning_lib);
+    $session->creator($e->requestor->id);
+    $session->search($search);
+
+    my $container = Fieldmapper::container::biblio_record_entry_bucket->new;
+    $container->btype("url_verify");
+    $container->owner($e->requestor->id);
+    $container->name($name);
+    $container->description("Automatically generated");
+
+    $e->create_container_biblio_record_entry_bucket($container) or
+        return $e->die_event;
+
+    $session->container($e->data->id);
+    $e->create_url_verify_session($session) or
+        return $e->die_event;
+
+    $e->commit or return $e->die_event;
+
+    return $e->data->id;
+}
+
+# _check_for_existing_bucket_items() is used later by session_search_and_extract()
+sub _check_for_existing_bucket_items {
+    my ($e, $session) = @_;
+
+    my $items = $e->json_query(
+        {
+            select => {cbrebi => ['id']},
+            from => {cbrebi => {}},
+            where => {bucket => $session->container},
+            limit => 1
+        }
+    ) or return $e->die_event;
+
+    return new OpenILS::Event("URL_VERIFY_SESSION_ALREADY_SEARCHED") if @$items;
+
+    return;
+}
+
+# _get_all_search_results() is used later by session_search_and_extract()
+sub _get_all_search_results {
+    my ($client, $session) = @_;
+
+    my @result_ids;
+
+    # Don't loop if the user has specified their own offset.
+    if ($session->search =~ /offset\(\d+\)/) {
+        my $res = $U->simplereq(
+            "open-ils.search",
+            "open-ils.search.biblio.multiclass.query.staff",
+            {}, $session->search
+        );
+
+        return new OpenILS::Event("UNKNOWN") unless $res;
+        return $res if $U->is_event($res);
+
+        @result_ids = map { shift @$_ } @{$res->{ids}}; # IDs nested in array
+    } else {
+        my $count;
+        my $so_far = 0;
+
+        LOOP: { do {    # Fun fact: you cannot "last" out of a do/while in Perl
+                        # unless you wrap it another loop structure.
+            my $search = $session->search . " offset(".scalar(@result_ids).")";
+
+            my $res = $U->simplereq(
+                "open-ils.search",
+                "open-ils.search.biblio.multiclass.query.staff",
+                {}, $search
+            );
+
+            return new OpenILS::Event("UNKNOWN") unless $res;
+            return $res if $U->is_event($res);
+
+            # Search only returns the total count when offset is 0.
+            # We can't get more than one superpage this way, XXX TODO ?
+            $count = $res->{count} unless defined $count;
+
+            my @this_batch = map { shift @$_ } @{$res->{ids}}; # unnest IDs
+            push @result_ids, @this_batch;
+
+            # Send a keepalive in case search is slow, although it'll probably
+            # be the query for the first ten results that's slowest.
+            $client->status(new OpenSRF::DomainObject::oilsContinueStatus);
+
+            last unless @this_batch; # Protect against getting fewer results
+                                     # than count promised.
+
+        } while ($count - scalar(@result_ids) > 0); }
+    }
+
+    return (undef, @result_ids);
+}
+
+
+__PACKAGE__->register_method(
+    method => "session_search_and_extract",
+    api_name => "open-ils.url_verify.session.search_and_extract",
+    stream => 1,
+    signature => {
+        desc => q/
+            Perform the search contained in the session,
+            populating the linked bucket, and extracting URLs /,
+        params => [
+            {desc => "Authentication token", type => "string"},
+            {desc => "url_verify.session id", type => "number"},
+        ],
+        return => {
+            desc => q/stream of numbers: first number of search results, then
+                numbers of extracted URLs for each record, grouped into arrays
+                of 100/,
+            type => "number"
+        }
+    }
+);
+
+sub session_search_and_extract {
+    my ($self, $client, $auth, $ses_id) = @_;
+
+    my $e = new_editor(authtoken => $auth);
+    return $e->die_event unless $e->checkauth;
+
+    my $session = $e->retrieve_url_verify_session(int($ses_id));
+
+    return $e->die_event unless
+        $session and $e->allowed("URL_VERIFY", $session->owning_lib);
+
+    if ($session->creator != $e->requestor->id) {
+        $e->disconnect;
+        return new OpenILS::Event("URL_VERIFY_NOT_SESSION_CREATOR");
+    }
+
+    my $delete_error =
+        _check_for_existing_bucket_items($e, $session);
+
+    if ($delete_error) {
+        $e->disconnect;
+        return $delete_error;
+    }
+
+    my ($search_error, @result_ids) =
+        _get_all_search_results($client, $session);
+
+    if ($search_error) {
+        $e->disconnect;
+        return $search_error;
+    }
+
+    $e->xact_begin;
+
+    # Make and save a bucket item for each search result.
+
+    my $pos = 0;
+    my @item_ids;
+
+    # There's an opportunity below to parallelize the extraction of URLs if
+    # we need to.
+
+    foreach my $bre_id (@result_ids) {
+        my $bucket_item =
+            Fieldmapper::container::biblio_record_entry_bucket_item->new;
+
+        $bucket_item->bucket($session->container);
+        $bucket_item->target_biblio_record_entry($bre_id);
+        $bucket_item->pos($pos++);
+
+        $e->create_container_biblio_record_entry_bucket_item($bucket_item) or
+            return $e->die_event;
+
+        push @item_ids, $e->data->id;
+    }
+
+    $e->xact_commit;
+
+    $client->respond($pos); # first response: the number of items created
+                            # (number of search results)
+
+    # For each contain item, extract URLs.  Report counts of URLs extracted
+    # from each record in batches at every hundred records.  XXX Arbitrary.
+
+    my @url_counts;
+    foreach my $item_id (@item_ids) {
+        my $res = $e->json_query({
+            from => ["url_verify.extract_urls", $ses_id, $item_id]
+        }) or return $e->die_event;
+
+        push @url_counts, $res->[0]{"url_verify.extract_urls"};
+
+        if (scalar(@url_counts) % 100 == 0) {
+            $client->respond([ @url_counts ]);
+            @url_counts = ();
+        }
+    }
+
+    $client->respond([ @url_counts ]) if @url_counts;
+
+    $e->disconnect;
+    return;
+}
+
+
 1;
index 89478ad..a49e5fc 100644 (file)
@@ -74,19 +74,19 @@ BEGIN
     FOR current_selector IN SELECT * FROM url_verify.url_selector s WHERE s.session = session_id LOOP
         current_url_pos := 1;
         LOOP
-            SELECT  (XPATH(current_selector.xpath || '/text()', b.marc))[current_url_pos]::TEXT INTO current_url
+            SELECT  (XPATH(current_selector.xpath || '/text()', b.marc::XML))[current_url_pos]::TEXT INTO current_url
               FROM  biblio.record_entry b
                     JOIN container.biblio_record_entry_bucket_item c ON (c.target_biblio_record_entry = b.id)
               WHERE c.id = item_id;
 
             EXIT WHEN current_url IS NULL;
 
-            SELECT  (XPATH(current_selector.xpath || '/../@tag', b.marc))[current_url_pos]::TEXT INTO current_tag
+            SELECT  (XPATH(current_selector.xpath || '/../@tag', b.marc::XML))[current_url_pos]::TEXT INTO current_tag
               FROM  biblio.record_entry b
                     JOIN container.biblio_record_entry_bucket_item c ON (c.target_biblio_record_entry = b.id)
               WHERE c.id = item_id;
 
-            SELECT  (XPATH(current_selector.xpath || '/@subfield', b.marc))[current_url_pos]::TEXT INTO current_sf
+            SELECT  (XPATH(current_selector.xpath || '/@code', b.marc::XML))[current_url_pos]::TEXT INTO current_sf
               FROM  biblio.record_entry b
                     JOIN container.biblio_record_entry_bucket_item c ON (c.target_biblio_record_entry = b.id)
               WHERE c.id = item_id;
index 2990382..f8a5bad 100644 (file)
@@ -58,23 +58,23 @@ BEGIN
     FOR current_selector IN SELECT * FROM url_verify.url_selector s WHERE s.session = session_id LOOP
         current_url_pos := 1;
         LOOP
-            SELECT  (XPATH(current_selector.xpath || '/text()', b.marc))[current_url_pos]::TEXT INTO current_url
+            SELECT  (XPATH(current_selector.xpath || '/text()', b.marc::XML))[current_url_pos]::TEXT INTO current_url
               FROM  biblio.record_entry b
                     JOIN container.biblio_record_entry_bucket_item c ON (c.target_biblio_record_entry = b.id)
               WHERE c.id = item_id;
-    
+
             EXIT WHEN current_url IS NULL;
-    
-            SELECT  (XPATH(current_selector.xpath || '/../@tag', b.marc))[current_url_pos]::TEXT INTO current_tag
+
+            SELECT  (XPATH(current_selector.xpath || '/../@tag', b.marc::XML))[current_url_pos]::TEXT INTO current_tag
               FROM  biblio.record_entry b
                     JOIN container.biblio_record_entry_bucket_item c ON (c.target_biblio_record_entry = b.id)
               WHERE c.id = item_id;
-    
-            SELECT  (XPATH(current_selector.xpath || '/@subfield', b.marc))[current_url_pos]::TEXT INTO current_sf
+
+            SELECT  (XPATH(current_selector.xpath || '/@code', b.marc::XML))[current_url_pos]::TEXT INTO current_sf
               FROM  biblio.record_entry b
                     JOIN container.biblio_record_entry_bucket_item c ON (c.target_biblio_record_entry = b.id)
               WHERE c.id = item_id;
-    
+
             INSERT INTO url_verify.url (item, url_selector, tag, subfield, ord, full_url)
               VALUES ( item_id, current_selector.id, current_tag, current_sf, current_ord, current_url);
 
diff --git a/Open-ILS/src/templates/url_verify/create_session.tt2 b/Open-ILS/src/templates/url_verify/create_session.tt2
new file mode 100644 (file)
index 0000000..d29ccd9
--- /dev/null
@@ -0,0 +1,130 @@
+[% WRAPPER base.tt2 %]
+[% ctx.page_title = "Link Checker - Create Session" %]
+<script type="text/javascript">
+    dojo.require("dijit.form.Button");
+    dojo.require("dijit.form.CheckBox");
+    dojo.require("dijit.form.TextBox");
+    dojo.require("openils.Util");
+    dojo.require("openils.widget.ProgressDialog");
+    dojo.require("openils.URLVerify.CreateSession");
+
+    var module;
+
+    openils.Util.addOnLoad(
+        function() {
+            module = openils.URLVerify.CreateSession;
+            module.setup("saved-searches", "org-selector", progress_dialog);
+        }
+    );
+</script>
+<style type="text/css">
+    #uv-search { width: 20em; }
+    .note { font-style: italic; background-color: #eee; }
+    #uv-tags-and-subfields { background-color: #ddd; }
+    table.create-session-form th { text-align: right; padding-right: 1em; }
+    table.create-session-form {
+        border-collapse: separate;
+        border-spacing: 0.5ex;
+    }
+</style>
+<div dojoType="dijit.layout.ContentPane" layoutAlign="client">
+    <div dojoType="dijit.layout.ContentPane"
+         layoutAlign="top" class="oils-header-panel">
+        <div> [% ctx.page_title %] </div>
+        <div> <!-- buttons could go here --></div>
+    </div>
+    <div>
+        <table class="create-session-form">
+            <tr>
+                <th>
+                    <label for="uv-session-name">[% l("Sesssion name:") %]</label>
+                </th>
+                <td>
+                    <input dojoType="dijit.form.TextBox"
+                        id="uv-session-name" jsId="uv_session_name" />
+                </td>
+                <td class="note">
+                </td>
+            </tr>
+
+            <tr>
+                <th>
+                    <label for="org-selector">[% l('Search scope:') %]</label>
+                </th>
+                <td>
+                    <div id="org-selector"></div>
+                </td>
+                <td class="note">
+                    [% l("This will only be used if your search doesn't contain a hand-entered filter such as site(BR1)") %]
+                </td>
+            </tr>
+
+            <!-- XXX TODO I bet we want a depth selector here too -->
+
+            <tr>
+                <th>
+                    <label for="uv-search">[% l('Search:') %]</label>
+                </th>
+                <td>
+                    <input dojoType="dijit.form.TextBox" id="uv-search"
+                        jsId="uv_search" />
+                </td>
+                <td class="note">
+                </td>
+            </tr>
+
+            <tr>
+                <th>
+                    <label for="saved-searches">[% l("Saved searches:") %]</label>
+                </th>
+                <td><!-- XXX we're just assuming this list won't grow so
+                    large as to be unrepresentable in a multiselect?  We
+                    could switch to a PCrudAutocompleteBox if needed for
+                    constant load time regardless of dataset size. -->
+                    <select id="saved-searches" multiple="true" size="6"></select>
+                </td>
+                <td class="note">[% l("Optionally select one or more to combine with 'Search' field above.") %]
+                </td>
+            </tr>
+
+            <tr>
+                <th>
+                    <label for="no-url-selection">[% l('Process immediately?') %]</label>
+                </th>
+                <td>
+                    <input dojoType="dijit.form.CheckBox" id="no-url-selection"
+                        jsId="no_url_selection" />
+                </td>
+                <td class="note">
+                </td>
+            </tr>
+
+            <tr>
+                <th>
+                    [% l('Tags and subfields possibly containing URLs:') %]
+                </th>
+                <td>
+                    <div id="uv-tags-and-subfields">
+                    </div>
+                    <div class="tag-and-subfield-add-another">
+                        [% l("Tag") %]
+                        <input type="text" size="4" maxlength="3" />
+                        [% l("Subfield(s)") %]
+                        <input type="text" size="15" />
+                        <a href="javascript:module.tag_and_subfields.add();">[% l('Add') %]</a>
+                    </div>
+                </td>
+                <td class="note">
+                </td>
+            </tr>
+        </table>
+
+        <div>
+            <button dojoType="dijit.form.Button"
+                onClick="module.begin();">[% l("Begin") %]</button>
+
+        </div>
+    </div>
+</div>
+<div dojoType="openils.widget.ProgressDialog" jsId="progress_dialog"></div>
+[% END %]
diff --git a/Open-ILS/src/templates/url_verify/select_urls.tt2 b/Open-ILS/src/templates/url_verify/select_urls.tt2
new file mode 100644 (file)
index 0000000..b3985d9
--- /dev/null
@@ -0,0 +1,66 @@
+[% WRAPPER base.tt2 %]
+[% ctx.page_title = "Link Checker - Select URLs" %]
+<script type="text/javascript">
+    dojo.require("dijit.form.Button");
+    dojo.require("openils.widget.FlattenerGrid");
+    dojo.require("openils.widget.ProgressDialog");
+    dojo.require("openils.Util");
+    dojo.require("openils.CGI");
+    dojo.require("openils.URLVerify.SelectURLs");
+
+    /* Minimize namespace pollution, but save us some typing later. */
+    var module = openils.URLVerify.SelectURLs;
+
+    openils.Util.addOnLoad(
+        function() {
+            module.setup(grid, progress_dialog);
+        }
+    );
+</script>
+<style type="text/css">
+    .url-verify-attempt-info { font-style: italic; }
+</style>
+<div dojoType="dijit.layout.ContentPane" layoutAlign="client">
+    <div dojoType="dijit.layout.ContentPane"
+         layoutAlign="top" class="oils-header-panel">
+        <div>[% ctx.page_title %]</div>
+        <div>
+            <button dojoType="dijit.form.Button"
+                onClick="module.verify_selected();">[%
+                l("Verify Selected URLs")
+            %]</button>
+        </div>
+    </div>
+    <div class="oils-acq-basic-roomy url-verify-attempt-info">
+        <div id="url-verify-attempt-id"></div>
+        <div id="url-verify-attempt-start"></div>
+        <div id="url-verify-attempt-finish"></div>
+    </div>
+    <table
+        jsid="grid"
+        dojoType="openils.widget.FlattenerGrid"
+        columnPersistKey='"url_verify.select_url"'
+        autoHeight="10"
+        editOnEnter="false"
+        autoFieldFields="null"
+        autoCoreFields="true"
+        autoCoreFieldsUnsorted="true"
+        fetchLock="true"
+        mapExtras="{session_id: {path: 'item.session.id', filter: true}}"
+        showLoadFilter="true"
+        fmClass="'uvu'">
+        <thead>
+            <tr>
+                <th field="title" fpath="item.target_biblio_record_entry.simple_record.title"></th>
+                <th field="author" fpath="item.target_biblio_record_entry.simple_record.author"></th>
+                <th field="isbn" fpath="item.target_biblio_record_entry.simple_record.isbn" _visible="false"></th>
+                <th field="issn" fpath="item.target_biblio_record_entry.simple_record.issn" _visible="false"></th>
+                <th field="bib_id" fpath="item.target_biblio_record_entry.id" _visible="false"></th>
+            </tr>
+        </thead>
+    </table>
+</div>
+<div class="hidden">
+    <div dojoType="openils.widget.ProgressDialog" jsId="progress_dialog"></div>
+</div>
+[% END %]
diff --git a/Open-ILS/web/js/dojo/openils/URLVerify/CreateSession.js b/Open-ILS/web/js/dojo/openils/URLVerify/CreateSession.js
new file mode 100644 (file)
index 0000000..cb79e82
--- /dev/null
@@ -0,0 +1,352 @@
+if (!dojo._hasResource["openils.URLVerify.CreateSession"]) {
+    dojo.require("dojo.data.ItemFileWriteStore");
+    dojo.require("dojox.jsonPath");
+    dojo.require("fieldmapper.OrgUtils");
+    dojo.require("openils.Util");
+    dojo.require("openils.PermaCrud");
+    dojo.require("openils.widget.FilteringTreeSelect");
+
+    dojo.requireLocalization("openils.URLVerify", "URLVerify");
+
+    dojo._hasResource["openils.URLVerify.CreateSession"] = true;
+    dojo.provide("openils.URLVerify.CreateSession");
+
+    dojo.declare("openils.URLVerify.CreateSession", null, {});
+
+    /* Take care that we add nothing to the global namespace. */
+
+(function() {
+    var module = openils.URLVerify.CreateSession;
+    var localeStrings =
+        dojo.i18n.getLocalization("openils.URLVerify", "URLVerify");
+    var uvus_progress = 0;
+
+    /* Take search text box input, selected saved search ids, and selected
+     * scope shortname to produce one search string. */
+    module._prepare_search = function(basic, saved, scope) {
+        if (saved.length) {
+            basic += " " + dojo.map(
+                saved, function(s) { return "saved_query(" + s + ")"; }
+            ).join(" ");
+        }
+
+        if (scope && !basic.match(/site\(.+\)/))
+            basic += " site(" + scope + ")";
+
+        return basic;
+    };
+
+    /* Reacting to the interface's "Begin" button, this function triggers the
+     * first of three server-side processes necessary to create a session:
+     *
+     * 1) create the session itself (API call), */
+    module.begin = function() {
+        var name = uv_session_name.attr("value");
+
+        var scope;
+        try {
+            scope = module.org_selector.store.getValue(
+                module.org_selector.item,
+                "shortname"
+            );
+        } catch (E) {
+            /* probably nothing valid is selected; move on */
+            void(0);
+        }
+
+        var search = module._prepare_search(
+            uv_search.attr("value"),
+            dojo.filter(
+                dojo.byId("saved-searches").options,
+                function(o) { return o.selected; }
+            ).map(
+                function(o) { return o.value; }
+            ),
+            scope
+        );
+
+        if (!module.tag_and_subfields.any()) {
+            alert(localeStrings.NEED_UVUS);
+            return;
+        }
+
+        module.progress_dialog.attr("title", localeStrings.CREATING);
+        module.progress_dialog.show(true);
+        fieldmapper.standardRequest(
+            ["open-ils.url_verify", "open-ils.url_verify.session.create"], {
+                "params": [openils.User.authtoken, name, search],
+                "async": true,
+                "onresponse": function(r) {
+                    if (r = openils.Util.readResponse(r)) {
+                        /* I think we're modal enough to get away with this. */
+                        module.session_id = r;
+                        module.save_tags();
+                    }
+                }
+            }
+        );
+    };
+
+    /* 2) save the tag/subfield sets for URL extraction, */
+    module.save_tags = function() {
+        module.progress_dialog.attr("title", localeStrings.SAVING_TAGS);
+        module.progress_dialog.show(); /* sic */
+
+        uvus_progress = 0;
+
+        /* Note we're not using openils.PermaCrud, which is inadequate
+         * when you want transactions. Thanks for figuring it out, Bill. */
+        var pcrud_raw = new OpenSRF.ClientSession("open-ils.pcrud");
+
+        pcrud_raw.connect();
+
+        pcrud_raw.request({
+            "method": "open-ils.pcrud.transaction.begin",
+            "params": [openils.User.authtoken],
+            "oncomplete": function(r) {
+                module._create_uvus_one_at_a_time(
+                    pcrud_raw,
+                    module.tag_and_subfields.generate_uvus(
+                        module.session_id
+                    )
+                );
+            }
+        }).send();
+    };
+
+    /* 2b */
+    module._create_uvus_one_at_a_time = function(pcrud_raw, uvus_list) {
+        pcrud_raw.request({
+            "method": "open-ils.pcrud.create.uvus",
+            "params": [openils.User.authtoken, uvus_list[0]],
+            "oncomplete": function(r) {
+                var new_uvus = openils.Util.readResponse(r);
+                module.progress_dialog.update(
+                    {"maximum": uvus_list.length, "progress": ++uvus_progress}
+                );
+
+                uvus_list.shift();  /* /now/ actually shorten the list */
+
+                if (uvus_list.length < 1) {
+                    pcrud_raw.request({
+                        "method": "open-ils.pcrud.transaction.commit",
+                        "params": [openils.User.authtoken],
+                        "oncomplete": function(r) {
+                            pcrud_raw.disconnect();
+                            module.perform_search();
+                        }
+                    }).send();
+
+                } else {
+                    module._create_uvus_one_at_a_time(
+                        pcrud_raw, uvus_list
+                    );
+                }
+             }
+        }).send();
+    };
+
+    /* 3) search and populate the container (API call). */
+    var search_result_count = 0;
+    module.perform_search = function() {
+        module.progress_dialog.attr("title", localeStrings.PERFORMING_SEARCH);
+        module.progress_dialog.show(true);
+
+        fieldmapper.standardRequest(
+            ["open-ils.url_verify",
+                "open-ils.url_verify.session.search_and_extract"], {
+                "params": [openils.User.authtoken, module.session_id],
+                "async": true,
+                "onresponse": function(r) {
+                    r = openils.Util.readResponse(r);
+                    if (!search_result_count) {
+                        search_result_count = Number(r);
+
+                        module.progress_dialog.show(); /* sic */
+                        module.progress_dialog.attr(
+                            "title", localeStrings.EXTRACTING_URLS
+                        );
+                        module.progress_dialog.update(
+                            {"maximum": search_result_count, "progress": 0}
+                        );
+                    } else {
+                        module.progress_dialog.update({"progress": r.length})
+                    }
+                },
+                "oncomplete": function() {
+                    module.progress_dialog.attr(
+                        "title", localeStrings.REDIRECTING
+                    );
+                    module.progress_dialog.show(true);
+
+                    if (no_url_selection.checked) {
+                        location.href = oilsBasePath +
+                            "/url_verify/validation_review?" +
+                            "session_id=" + module.session_id +
+                            "&validate=1";
+                    } else {
+                        location.href = oilsBasePath +
+                            "/url_verify/select_urls?session_id=" +
+                            module.session_id;
+                    }
+                }
+            }
+        );
+    };
+
+    /* At least in Dojo 1.3.3 (I know, I know), dijit.form.MultiSelect does
+     * not behave like FilteringSelect, like you might expect, or work from a
+     * data store.  So we'll use a native <select> control, which will have
+     * fewer moving parts that can go haywire anyway.
+     */
+    module._populate_saved_searches = function(node) {
+        var pcrud = new openils.PermaCrud();
+        var list = pcrud.retrieveAll(
+            "asq", {"order_by": {"asq": "label"}}
+        );
+
+        dojo.forEach(
+            list,
+            function(o) {
+                dojo.create(
+                    "option", {
+                        "innerHTML": o.label(),
+                        "value": o.id(),
+                        "title": o.query_text()
+                    }, node, "last"
+                );
+            }
+        );
+
+        pcrud.disconnect();
+    };
+
+    /* set up an all-org-units-in-the-tree selector */
+    module._prepare_org_selector = function(node) {
+        var widget = new openils.widget.FilteringTreeSelect(null, node);
+        widget.searchAttr = "name";
+        widget.labelAttr = "name";
+        widget.tree = fieldmapper.aou.globalOrgTree;
+        widget.parentField = 'parent_ou';
+        widget.startup();
+        widget.attr("value", openils.User.user.ws_ou());
+
+        module.org_selector = widget;
+    };
+
+    module.setup = function(saved_search_id, org_selector_id, progress_dialog) {
+        module.progress_dialog = progress_dialog;
+
+        module.progress_dialog.attr("title", localeStrings.INTERFACE_SETUP);
+        module.progress_dialog.show(true);
+
+        module._populate_saved_searches(dojo.byId(saved_search_id));
+        module._prepare_org_selector(dojo.byId(org_selector_id));
+
+        module.progress_dialog.hide();
+    };
+
+    /* This is the thing that lets you add/remove rows of tab/subfield pairs */
+    function TagAndSubfieldsMgr(container_id) {
+        var self = this;
+
+        this.container_id = container_id;
+        this.counter = 0;
+
+        this.read_new = function() {
+            var controls = dojo.query(".tag-and-subfield-add-another input");
+
+            return {
+                "tag": controls[0].value,
+                "subfields": openils.Util.uniqueElements(
+                    controls[1].value.replace(/[^0-9a-z]/g, "").split("")
+                ).sort().join("")
+            };
+        };
+
+        this.add = function() {
+            var newdata = this.read_new();
+            var newid = "t-and-s-row-" + String(this.counter++);
+            var div = dojo.create(
+                "div", {
+                    "id": newid,
+                    "innerHTML": "<span class='t-and-s-tag'>" +
+                        newdata.tag +
+                        "</span> \u2021<span class='t-and-s-subfields'>" +
+                        newdata.subfields + "</span> "
+                }, this.container_id, "last"
+            );
+            dojo.create(
+                "a", {
+                    "href": "javascript:void(0);",
+                    "onclick": function() {
+                        var me = dojo.byId(newid);
+                        me.parentNode.removeChild(me);
+                    },
+                    "innerHTML": "[X]" /* XXX i18n */
+                }, div, "last"
+            );
+
+            this.counter++;
+        };
+
+        /* return a boolean indicating whether or not we have any rows */
+        this.any = function() {
+            return Boolean(
+                dojo.query(
+                    '[id^="t-and-s-row-"]', dojo.byId(this.container_id)
+                ).length
+            );
+        };
+
+        /* Return one uvus object for each unique tag we have a row for,
+         * and use the given session_id for the uvus.session field. */
+        this.generate_uvus = function(session_id) {
+            var uniquely_grouped = {};
+            dojo.query(
+                '[id^="t-and-s-row-"]', dojo.byId(this.container_id)
+            ).forEach(
+                function(row) {
+                    var tag = dojo.query(".t-and-s-tag", row)[0].innerHTML;
+                    var subfield = dojo.query(".t-and-s-subfields", row)[0].innerHTML;
+
+                    var existing;
+                    if ((existing = uniquely_grouped[tag])) { /* sic, assignment */
+                        existing = openils.Util.uniqueElements(
+                            (existing + subfield).split("")
+                        ).sort().join("");
+                    } else {
+                        uniquely_grouped[tag] = subfield;
+                    }
+                }
+            );
+
+            var uvus_list = [];
+            for (var tag in uniquely_grouped) {
+                var obj = new uvus();
+
+                obj.session(session_id);
+
+                /* XXX TODO handle control fields (no subfields) */
+                obj.xpath(
+                    "//*[@tag='" + tag + "']/*[" +
+                    uniquely_grouped[tag].split("").map(
+                        function(c) { return "@code='" + c + "'"; }
+                    ).join(" or ") +
+                    "]"
+                );
+
+                uvus_list.push(obj);
+            }
+
+            return uvus_list;
+        };
+
+    }
+
+    module.tag_and_subfields =
+        new TagAndSubfieldsMgr("uv-tags-and-subfields");
+
+}());
+
+}
diff --git a/Open-ILS/web/js/dojo/openils/URLVerify/SelectURLs.js b/Open-ILS/web/js/dojo/openils/URLVerify/SelectURLs.js
new file mode 100644 (file)
index 0000000..b8992ac
--- /dev/null
@@ -0,0 +1,106 @@
+if (!dojo._hasResource["openils.URLVerify.SelectURLs"]) {
+    dojo.require("dojo.string");
+    dojo.require("openils.CGI");
+    dojo.require("openils.Util");
+
+    dojo.requireLocalization("openils.URLVerify", "URLVerify");
+
+    dojo._hasResource["openils.URLVerify.SelectURLs"] = true;
+    dojo.provide("openils.URLVerify.SelectURLs");
+
+    dojo.declare("openils.URLVerify.SelectURLs", null, {});
+
+    /* Take care that we add nothing to the global namespace.
+     * This is not an OO module so much as a container for
+     * functions needed by a specific interface. */
+
+(function() {
+    var module = openils.URLVerify.SelectURLs;
+    var localeStrings =
+        dojo.i18n.getLocalization("openils.URLVerify", "URLVerify");
+
+    module.setup = function(grid, progress_dialog) {
+        var cgi = new openils.CGI();
+        module.session_id = cgi.param("session_id");
+
+        module.grid = grid;
+
+        module.grid.attr("query", {"session_id": module.session_id});
+        module.grid.refresh();
+        // Alternative to grid.refresh() once filter is set up
+        //module.grid.fetchLock = false;
+        //module.grid.filterUi.doApply();
+    };
+
+    module.verify_selected = function() {
+        var really_everything = false;
+
+        if (module.grid.everythingSeemsSelected())
+            really_everything = confirm(localeStrings.VERIFY_ALL);
+
+        module.clear_attempt_display();
+        progress_dialog.attr("title", localeStrings.VERIFICATION_BEGIN);
+        progress_dialog.show();
+
+        fieldmapper.standardRequest(
+            ["open-ils.url_verify", "open-ils.url_verify.session.verify"], {
+                "params": [
+                    openils.User.authtoken,
+                    module.session_id,
+                    really_everything ? null : module.grid.getSelectedIDs()
+                ],
+                "async": true,
+                "onresponse": function(r) {
+                    if (r = openils.Util.readResponse(r)) {
+                        progress_dialog.attr(
+                            "title",
+                            dojo.string.substitute(
+                                localeStrings.VERIFICATION_PROGRESS,
+                                [r.total_processed]
+                            )
+                        );
+                        progress_dialog.update({
+                            "maximum": r.url_count,
+                            "progress": r.total_excluding_redirects
+                        });
+
+                        if (r.attempt)
+                            module.update_attempt_display(r.attempt);
+                    }
+                }
+            }
+        )
+
+        module.grid.getSelectedIDs();   
+    };
+
+    module.clear_attempt_display = function() {
+        dojo.empty(dojo.byId("url-verify-attempt-id"));
+        dojo.empty(dojo.byId("url-verify-attempt-start"));
+        dojo.empty(dojo.byId("url-verify-attempt-finish"));
+    };
+
+    module.update_attempt_display = function(attempt) {
+        dojo.byId("url-verify-attempt-id").innerHTML =
+            dojo.string.substitute(
+                localeStrings.VERIFICATION_ATTEMPT_ID,
+                [attempt.id()]
+            );
+        dojo.byId("url-verify-attempt-start").innerHTML =
+            dojo.string.substitute(
+                localeStrings.VERIFICATION_ATTEMPT_START,
+                [attempt.start_time()]
+            );
+
+        if (attempt.finish_time()) {
+            dojo.byId("url-verify-attempt-finish").innerHTML =
+                dojo.string.substitute(
+                    localeStrings.VERIFICATION_ATTEMPT_FINISH,
+                    [attempt.finish_time()]
+                );
+        }
+    };
+
+}());
+
+}
diff --git a/Open-ILS/web/js/dojo/openils/URLVerify/nls/URLVerify.js b/Open-ILS/web/js/dojo/openils/URLVerify/nls/URLVerify.js
new file mode 100644 (file)
index 0000000..d0b6f3d
--- /dev/null
@@ -0,0 +1,15 @@
+{
+    "CREATING": "Creating session ...",
+    "SAVING_TAGS": "Saving tag/subfield selectors ...",
+    "PERFORMING_SEARCH": "Performing search ...",
+    "EXTRACTING_URLS": "Extracting URLs ...",
+    "NEED_UVUS": "You must add some tag and subfield combinations",
+    "REDIRECTING": "Loading next interface ...",
+    "INTERFACE_SETUP": "Setting up interface ...",
+    "VERIFY_ALL": "Click 'OK' to verify ALL the URLs found in this session.  Click 'Cancel' if the selected, visible URLs are the only ones you want verified.",
+    "VERIFICATION_BEGIN": "Verifying URLs. This can take a moment ...",
+    "VERIFICATION_PROGRESS": "Verifying URLs: ${0} URLs processed ...",
+    "VERIFICATION_ATTEMPT_ID": "Last verification attempt ID: ${0}",
+    "VERIFICATION_ATTEMPT_START": "Attempt start time: ${0}",
+    "VERIFICATION_ATTEMPT_FINISH": "Attempt finish time: ${0}"
+}
index 6443f89..7791369 100644 (file)
@@ -16,6 +16,7 @@ tree1.startup();
 if(!dojo._hasResource["openils.widget.FilteringTreeSelect"]){
     dojo.provide("openils.widget.FilteringTreeSelect");
     dojo.require("dijit.form.FilteringSelect");
+    dojo.require("dojo.data.ItemFileWriteStore");
 
     dojo.declare(
         "openils.widget.FilteringTreeSelect", [dijit.form.FilteringSelect], {
index 54e67c2..e83fca9 100644 (file)
@@ -17,6 +17,7 @@ if (!dojo._hasResource["openils.widget.FlattenerGrid"]) {
             "columnReordering": true,
             "columnPersistKey": null,
             "autoCoreFields": false,
+            "autoCoreFieldsUnsorted": false,
             "autoFieldFields": null,
             "showLoadFilter": false,    /* use FlattenerFilter(Dialog|Pane) */
             "filterAlwaysInDiv": null,  /* use FlattenerFilterPane and put its
@@ -304,10 +305,14 @@ if (!dojo._hasResource["openils.widget.FlattenerGrid"]) {
                 var cell_list = this.structure[0].cells[0];
                 var fields = dojo.clone(
                     fieldmapper.IDL.fmclasses[this.fmClass].fields
-                ).sort(
-                    function(a, b) { return a.label > b.label ? 1 : -1; }
                 );
 
+                if (!this.autoCoreFieldsUnsorted) {
+                    fields = fields.sort(
+                        function(a, b) { return a.label > b.label ? 1 : -1; }
+                    );
+                }
+
                 dojo.forEach(
                     fields, function(f) {
                         if (f.datatype == "link" || f.virtual)
@@ -324,8 +329,8 @@ if (!dojo._hasResource["openils.widget.FlattenerGrid"]) {
                         cell_list.push({
                             "field": f.name,
                             "name": f.label,
-                            "fsort": true,
-                            "_visible": false
+                            "fsort": true /*,
+                            "_visible": false */
                         });
                     }
                 );
@@ -869,6 +874,21 @@ if (!dojo._hasResource["openils.widget.FlattenerGrid"]) {
                 );
             },
 
+            /* Return true if every row known to the grid is selected. Code
+             * that calls this function will do so when it thinks the user
+             * might actually mean "select everything this grid could show"
+             * even though we don't necessarily know (and the user hasn't
+             * necessarily noticed) whether the grid has been scrolled as far
+             * down as possible and all the possible results have been
+             * fetched by the grid's store. */
+            "everythingSeemsSelected": function() {
+                return dojo.query(
+                    "[name=autogrid.selector]", this.domNode
+                ).filter(
+                    function(c) { return (!c.disabled && !c.checked) }
+                ).length == 0;
+            },
+
             /* Print the same data that the Flattener is feeding to the
              * grid, sorted the same way too. Remove limit and offset (i.e.,
              * print it all) unless those are passed in to the print() method.