--- /dev/null
+<?xml version="1.0" encoding="UTF-8"?>\r
+<chapter xml:id="opensrf" xmlns="http://docbook.org/ns/docbook" version="5.0" xml:lang="EN"\r
+ xmlns:xi="http://www.w3.org/2001/XInclude" xmlns:xlink="http://www.w3.org/1999/xlink">\r
+ <chapterinfo>\r
+ <title>OpenSRF</title>\r
+ </chapterinfo>\r
+ <abstract id="openSRF_abstract">\r
+ <simpara>One of the claimed advantages of\r
+ Evergreen over alternative integrated library systems is the underlying Open\r
+ Service Request Framework (OpenSRF, pronounced "open surf") architecture. This\r
+ article introduces OpenSRF, demonstrates how to build OpenSRF services through\r
+ simple code examples, and explains the technical foundations on which OpenSRF\r
+ is built. This chapter was taken from Dan Scott's <emphasis>Easing gently into OpenSRF</emphasis> article, June, 2010.</simpara>\r
+ </abstract>\r
+ <section id="_introducing_opensrf">\r
+ <title>Introducing OpenSRF</title>\r
+ <indexterm><primary>OpenSRF</primary></indexterm>\r
+ <simpara>OpenSRF is a message routing network that offers scalability and failover\r
+ support for individual services and entire servers with minimal development and\r
+ deployment overhead. You can use OpenSRF to build loosely-coupled applications\r
+ that can be deployed on a single server or on clusters of geographically\r
+ distributed servers using the same code and minimal configuration changes.\r
+ Although copyright statements on some of the OpenSRF code date back to Mike\r
+ Rylander’s original explorations in 2000, Evergreen was the first major\r
+ application to be developed with, and to take full advantage of, the OpenSRF\r
+ architecture starting in 2004. The first official release of OpenSRF was 0.1 in\r
+ February 2005 (<ulink url="http://evergreen-ils.org/blog/?p=21">http://evergreen-ils.org/blog/?p=21</ulink>), but OpenSRF’s development\r
+ continues a steady pace of enhancement and refinement, with the release of\r
+ 1.0.0 in October 2008 and the most recent release of 1.2.2 in February 2010.</simpara>\r
+ <simpara>OpenSRF is a distinct break from the architectural approach used by previous\r
+ library systems and has more in common with modern Web applications. The\r
+ traditional "scale-up" approach to serve more transactions is to purchase a\r
+ server with more CPUs and more RAM, possibly splitting the load between a Web\r
+ server, a database server, and a business logic server. Evergreen, however, is\r
+ built on the Open Service Request Framework (OpenSRF) architecture, which\r
+ firmly embraces the "scale-out" approach of spreading transaction load over\r
+ cheap commodity servers. The <ulink url="http://evergreen-ils.org/blog/?p=56">initial GPLS\r
+ PINES hardware cluster</ulink>, while certainly impressive, may have offered the\r
+ misleading impression that Evergreen requires a lot of hardware to run.\r
+ However, Evergreen and OpenSRF easily scale down to a single server; many\r
+ Evergreen libraries run their entire library system on a single server, and\r
+ most OpenSRF and Evergreen development occurs on a virtual machine running on a\r
+ single laptop or desktop image.</simpara>\r
+ <simpara>Another common concern is that the flexibility of OpenSRF’s distributed\r
+ architecture makes it complex to configure and to write new applications. This\r
+ article demonstrates that OpenSRF itself is an extremely simple architecture on\r
+ which one can easily build applications of many kinds – not just library\r
+ applications – and that you can use a number of different languages to call and\r
+ implement OpenSRF methods with a minimal learning curve. With an application\r
+ built on OpenSRF, when you identify a bottleneck in your application’s business\r
+ logic layer, you can adjust the number of the processes serving that particular\r
+ bottleneck on each of your servers; or if the problem is that your service is\r
+ resource-hungry, you could add an inexpensive server to your cluster and\r
+ dedicate it to running that resource-hungry service.</simpara>\r
+ <simplesect id="_programming_language_support">\r
+ <title>Programming language support</title>\r
+ <simpara>If you need to develop an entirely new OpenSRF service, you can choose from a\r
+ number of different languages in which to implement that service. OpenSRF\r
+ client language bindings have been written for C, Java, JavaScript, Perl, and\r
+ Python, and service language bindings have been written for C, Perl, and Python.\r
+ This article uses Perl examples as a lowest common denominator programming\r
+ language. Writing an OpenSRF binding for another language is a relatively small\r
+ task if that language offers libraries that support the core technologies on\r
+ which OpenSRF depends:</simpara>\r
+ <itemizedlist>\r
+ <listitem>\r
+ <simpara>\r
+ <ulink url="http://tools.ietf.org/html/rfc3920">Extensible Messaging and Presence\r
+ Protocol</ulink> (XMPP, sometimes referred to as Jabber) - provides the base messaging\r
+ infrastructure between OpenSRF clients and services\r
+ </simpara>\r
+ <indexterm><primary>XMPP</primary></indexterm>\r
+ </listitem>\r
+ <listitem>\r
+ <simpara>\r
+ <ulink url="http://json.org">JavaScript Object Notation</ulink> (JSON) - serializes the content\r
+ of each XMPP message in a standardized and concise format\r
+ </simpara>\r
+ </listitem>\r
+ <listitem>\r
+ <simpara>\r
+ <ulink url="http://memcached.org">memcached</ulink> - provides the caching service\r
+ </simpara>\r
+ <indexterm><primary>memcached</primary></indexterm>\r
+ </listitem>\r
+ <listitem>\r
+ <simpara>\r
+ <ulink url="http://tools.ietf.org/html/rfc5424">syslog</ulink> - the standard UNIX logging\r
+ service\r
+ </simpara>\r
+ <indexterm><primary>syslog</primary></indexterm>\r
+ </listitem>\r
+ </itemizedlist>\r
+ <simpara>Unfortunately, the\r
+ <ulink url="http://evergreen-ils.org/dokuwiki/doku.php?id=osrf-devel:primer">OpenSRF\r
+ reference documentation</ulink>, although augmented by the\r
+ <ulink url="http://evergreen-ils.org/dokuwiki/doku.php?id=osrf-devel:terms">OpenSRF\r
+ glossary</ulink>, blog posts like <ulink url="http://evergreen-ils.org/blog/?p=36">the description\r
+ of OpenSRF and Jabber</ulink>, and even this article, is not a sufficient substitute\r
+ for a complete specification on which one could implement a language binding.\r
+ The recommended option for would-be developers of another language binding is\r
+ to use the Python implementation as the cleanest basis for a port to another\r
+ language.</simpara>\r
+ <indexterm><primary>Python</primary></indexterm>\r
+ </simplesect>\r
+ </section>\r
+ <section id="writing_an_opensrf_service">\r
+ <title>Writing an OpenSRF Service</title>\r
+ <simpara>Imagine an application architecture in which 10 lines of Perl or Python, using\r
+ the data types native to each language, are enough to implement a method that\r
+ can then be deployed and invoked seamlessly across hundreds of servers. You\r
+ have just imagined developing with OpenSRF – it is truly that simple. Under the\r
+ covers, of course, the OpenSRF language bindings do an incredible amount of\r
+ work on behalf of the developer. An OpenSRF application consists of one or more\r
+ OpenSRF services that expose methods: for example, the <literal>opensrf.simple-text</literal>\r
+ <ulink url="http://svn.open-ils.org/trac/OpenSRF/browser/trunk/src/perl/lib/OpenSRF/Application/Demo/SimpleText.pm">demonstration\r
+ service</ulink> exposes the <literal>opensrf.simple-text.split()</literal> and\r
+ <literal>opensrf.simple-text.reverse()</literal> methods. Each method accepts zero or more\r
+ arguments and returns zero or one results. The data types supported by OpenSRF\r
+ arguments and results are typical core language data types: strings, numbers,\r
+ booleans, arrays, and hashes.</simpara>\r
+ <simpara>To implement a new OpenSRF service, perform the following steps:</simpara>\r
+ <orderedlist numeration="arabic">\r
+ <listitem>\r
+ <simpara>\r
+ Include the base OpenSRF support libraries\r
+ </simpara>\r
+ </listitem>\r
+ <listitem>\r
+ <simpara>\r
+ Write the code for each of your OpenSRF methods as separate procedures\r
+ </simpara>\r
+ </listitem>\r
+ <listitem>\r
+ <simpara>\r
+ Register each method\r
+ </simpara>\r
+ </listitem>\r
+ <listitem>\r
+ <simpara>\r
+ Add the service definition to the OpenSRF configuration files\r
+ </simpara>\r
+ </listitem>\r
+ </orderedlist>\r
+ <simpara>For example, the following code implements an OpenSRF service. The service\r
+ includes one method named <literal>opensrf.simple-text.reverse()</literal> that accepts one\r
+ string as input and returns the reversed version of that string:</simpara>\r
+<programlisting language="perl" linenumbering="unnumbered">\r
+#!/usr/bin/perl\r
+\r
+package OpenSRF::Application::Demo::SimpleText;\r
+\r
+use strict;\r
+\r
+use OpenSRF::Application;\r
+use parent qw/OpenSRF::Application/;\r
+\r
+sub text_reverse {\r
+ my ($self , $conn, $text) = @_;\r
+ my $reversed_text = scalar reverse($text);\r
+ return $reversed_text;\r
+}\r
+\r
+__PACKAGE__->register_method(\r
+ method => 'text_reverse',\r
+ api_name => 'opensrf.simple-text.reverse'\r
+);\r
+</programlisting>\r
+ <simpara>Ten lines of code, and we have a complete OpenSRF service that exposes a single\r
+ method and could be deployed quickly on a cluster of servers to meet your\r
+ application’s ravenous demand for reversed strings! If you’re unfamiliar with\r
+ Perl, the <literal>use OpenSRF::Application; use parent qw/OpenSRF::Application/;</literal>\r
+ lines tell this package to inherit methods and properties from the\r
+ <literal>OpenSRF::Application</literal> module. For example, the call to\r
+ <literal>__PACKAGE__->register_method()</literal> is defined in <literal>OpenSRF::Application</literal> but due to\r
+ inheritance is available in this package (named by the special Perl symbol\r
+ <literal>__PACKAGE__</literal> that contains the current package name). The <literal>register_method()</literal>\r
+ procedure is how we introduce a method to the rest of the OpenSRF world.</simpara>\r
+ <simplesect id="serviceRegistration">\r
+ <title>Registering a service with the OpenSRF configuration files</title>\r
+ <simpara>Two files control most of the configuration for OpenSRF:</simpara>\r
+ <itemizedlist>\r
+ <listitem>\r
+ <simpara>\r
+ <literal>opensrf.xml</literal> contains the configuration for the service itself, as well as\r
+ a list of which application servers in your OpenSRF cluster should start\r
+ the service.\r
+ </simpara>\r
+ </listitem>\r
+ <listitem>\r
+ <simpara>\r
+ <literal>opensrf_core.xml</literal> (often referred to as the "bootstrap configuration"\r
+ file) contains the OpenSRF networking information, including the XMPP server\r
+ connection credentials for the public and private routers. You only need to touch\r
+ this for a new service if the new service needs to be accessible via the\r
+ public router.\r
+ </simpara>\r
+ <indexterm><primary>configuration files</primary><secondary>opensrf_core.xml</secondary></indexterm>\r
+ </listitem>\r
+ </itemizedlist>\r
+ <simpara>Begin by defining the service itself in <literal>opensrf.xml</literal>. To register the\r
+ <literal>opensrf.simple-text</literal> service, add the following section to the <literal><apps></literal>\r
+ element (corresponding to the XPath <literal>/opensrf/default/apps/</literal>):</simpara>\r
+ <indexterm><primary>configuration files</primary><secondary>opensrf.xml</secondary></indexterm>\r
+<programlisting language="xml" linenumbering="unnumbered">\r
+<apps>\r
+ <opensrf.simple-text> <co id="CO1-1"/> \r
+ <keepalive>3</keepalive><co id="CO1-2"/> \r
+ <stateless>1</stateless><co id="CO1-3"/>\r
+ <language>perl</language><co id="CO1-4"/> \r
+ <implementation>OpenSRF::Application::Demo::SimpleText</implementation><co id="CO1-5"/> \r
+ <max_requests>100</max_requests><co id="CO1-6"/> \r
+ <unix_config>\r
+ <max_requests>1000</max_requests> <co id="CO1-7"/> \r
+ <unix_log>opensrf.simple-text_unix.log</unix_log> <co id="CO1-8"/> \r
+ <unix_sock>opensrf.simple-text_unix.sock</unix_sock><co id="CO1-9"/> \r
+ <unix_pid>opensrf.simple-text_unix.pid</unix_pid> <co id="CO1-10"/> \r
+ <min_children>5</min_children> <co id="CO1-11"/> \r
+ <max_children>15</max_children><co id="CO1-12"/> \r
+ <min_spare_children>2</min_spare_children><co id="CO1-13"/> \r
+ <max_spare_children>5</max_spare_children> <co id="CO1-14"/> \r
+ </unix_config>\r
+ </opensrf.simple-text>\r
+\r
+ <!-- other OpenSRF services registered here... -->\r
+</apps>\r
+</programlisting>\r
+ <calloutlist>\r
+ <callout arearefs="CO1-1">\r
+ <simpara>\r
+ The element name is the name that the OpenSRF control scripts use to refer\r
+ to the service.\r
+ </simpara>\r
+ </callout>\r
+ <callout arearefs="CO1-2">\r
+ <simpara>\r
+ The <literal><keepalive></literal> element specifies the interval (in seconds) between\r
+ checks to determine if the service is still running.\r
+ </simpara>\r
+ </callout>\r
+ <callout arearefs="CO1-3">\r
+ <simpara>\r
+ The <literal><stateless></literal> element specifies whether OpenSRF clients can call\r
+ methods from this service without first having to create a connection to a\r
+ specific service backend process for that service. If the value is <literal>1</literal>, then\r
+ the client can simply issue a request and the router will forward the request\r
+ to an available service and the result will be returned directly to the client.\r
+ </simpara>\r
+ </callout>\r
+ <callout arearefs="CO1-4">\r
+ <simpara>\r
+ The <literal><language></literal> element specifies the programming language in which the\r
+ service is implemented.\r
+ </simpara>\r
+ </callout>\r
+ <callout arearefs="CO1-5">\r
+ <simpara>\r
+ The <literal><implementation></literal> element pecifies the name of the library or module\r
+ in which the service is implemented.\r
+ </simpara>\r
+ </callout>\r
+ <callout arearefs="CO1-6">\r
+ <simpara>\r
+ (C implementations only): The <literal><max_requests></literal> element, as a direct child\r
+ of the service element name, specifies the maximum number of requests a process\r
+ serves before it is killed and replaced by a new process.\r
+ </simpara>\r
+ </callout>\r
+ <callout arearefs="CO1-7">\r
+ <simpara>\r
+ (Perl implementations only): The <literal><max_requests></literal> element, as a direct\r
+ child of the <literal><unix_config></literal> element, specifies the maximum number of requests\r
+ a process serves before it is killed and replaced by a new process.\r
+ </simpara>\r
+ </callout>\r
+ <callout arearefs="CO1-8">\r
+ <simpara>\r
+ The <literal><unix_log></literal> element specifies the name of the log file for\r
+ language-specific log messages such as syntax warnings.\r
+ </simpara>\r
+ </callout>\r
+ <callout arearefs="CO1-9">\r
+ <simpara>\r
+ The <literal><unix_sock></literal> element specifies the name of the UNIX socket used for\r
+ inter-process communications.\r
+ </simpara>\r
+ </callout>\r
+ <callout arearefs="CO1-10">\r
+ <simpara>\r
+ The <literal><unix_pid></literal> element specifies the name of the PID file for the\r
+ master process for the service.\r
+ </simpara>\r
+ </callout>\r
+ <callout arearefs="CO1-11">\r
+ <simpara>\r
+ The <literal><min_children></literal> element specifies the minimum number of child\r
+ processes that should be running at any given time.\r
+ </simpara>\r
+ </callout>\r
+ <callout arearefs="CO1-12">\r
+ <simpara>\r
+ The <literal><max_children></literal> element specifies the maximum number of child\r
+ processes that should be running at any given time.\r
+ </simpara>\r
+ </callout>\r
+ <callout arearefs="CO1-13">\r
+ <simpara>\r
+ The <literal><min_spare_children></literal> element specifies the minimum number of idle\r
+ child processes that should be available to handle incoming requests. If there\r
+ are fewer than this number of spare child processes, new processes will be\r
+ spawned.\r
+ </simpara>\r
+ </callout>\r
+ <callout arearefs="CO1-14">\r
+ <simpara>\r
+ The`<max_spare_children>` element specifies the maximum number of idle\r
+ child processes that should be available to handle incoming requests. If there\r
+ are more than this number of spare child processes, the extra processes will be\r
+ killed.\r
+ </simpara>\r
+ </callout>\r
+ </calloutlist>\r
+ <simpara>To make the service accessible via the public router, you must also\r
+ edit the <literal>opensrf_core.xml</literal> configuration file to add the service to the list\r
+ of publicly accessible services:</simpara>\r
+ <formalpara><title>Making a service publicly accessible in <literal>opensrf_core.xml</literal></title><para>\r
+<programlisting language="xml" linenumbering="unnumbered">\r
+<router><co id="CO2-1"/> \r
+ <!-- This is the public router. On this router, we only register applications\r
+ which should be accessible to everyone on the opensrf network -->\r
+ <name>router</name>\r
+ <domain>public.localhost</domain><co id="CO2-2"/>\r
+ <services>\r
+ <service>opensrf.math</service>\r
+ <service>opensrf.simple-text</service> <co id="CO2-3"/> \r
+ </services>\r
+</router>\r
+</programlisting>\r
+ </para></formalpara>\r
+ <calloutlist>\r
+ <callout arearefs="CO2-1">\r
+ <simpara>\r
+ This section of the <literal>opensrf_core.xml</literal> file is located at XPath\r
+ <literal>/config/opensrf/routers/</literal>.\r
+ </simpara>\r
+ </callout>\r
+ <callout arearefs="CO2-2">\r
+ <simpara>\r
+ <literal>public.localhost</literal> is the canonical public router domain in the OpenSRF\r
+ installation instructions.\r
+ </simpara>\r
+ </callout>\r
+ <callout arearefs="CO2-3">\r
+ <simpara>\r
+ Each <literal><service></literal> element contained in the <literal><services></literal> element\r
+ offers their services via the public router as well as the private router.\r
+ </simpara>\r
+ </callout>\r
+ </calloutlist>\r
+ <simpara>Once you have defined the new service, you must restart the OpenSRF Router\r
+ to retrieve the new configuration and start or restart the service itself.</simpara>\r
+ </simplesect>\r
+ <simplesect id="_calling_an_opensrf_method">\r
+ <title>Calling an OpenSRF method</title>\r
+ <indexterm><primary>srfsh</primary></indexterm>\r
+ <simpara>OpenSRF clients in any supported language can invoke OpenSRF services in any\r
+ supported language. So let’s see a few examples of how we can call our fancy\r
+ new <literal>opensrf.simple-text.reverse()</literal> method:</simpara>\r
+ <simplesect id="_calling_opensrf_methods_from_the_srfsh_client">\r
+ <title>Calling OpenSRF methods from the srfsh client</title>\r
+ <simpara><literal>srfsh</literal> is a command-line tool installed with OpenSRF that you can use to call\r
+ OpenSRF methods. To call an OpenSRF method, issue the <literal>request</literal> command and\r
+ pass the OpenSRF service and method name as the first two arguments; then pass\r
+ one or more JSON objects delimited by commas as the arguments to the method\r
+ being invoked.</simpara>\r
+ <simpara>The following example calls the <literal>opensrf.simple-text.reverse</literal> method of the\r
+ <literal>opensrf.simple-text</literal> OpenSRF service, passing the string <literal>"foobar"</literal> as the\r
+ only method argument:</simpara>\r
+<programlisting language="sh" linenumbering="unnumbered">\r
+$ srfsh\r
+srfsh # request opensrf.simple-text opensrf.simple-text.reverse "foobar"\r
+\r
+Received Data: "raboof"\r
+\r
+=------------------------------------\r
+Request Completed Successfully\r
+Request Time in seconds: 0.016718\r
+=------------------------------------\r
+</programlisting>\r
+ </simplesect>\r
+ <simplesect id="opensrfIntrospection">\r
+ <title>Getting documentation for OpenSRF methods from the srfsh client</title>\r
+ <simpara>The <literal>srfsh</literal> client also gives you command-line access to retrieving metadata\r
+ about OpenSRF services and methods. For a given OpenSRF method, for example,\r
+ you can retrieve information such as the minimum number of required arguments,\r
+ the data type and a description of each argument, the package or library in\r
+ which the method is implemented, and a description of the method. To retrieve\r
+ the documentation for an opensrf method from <literal>srfsh</literal>, issue the <literal>introspect</literal>\r
+ command, followed by the name of the OpenSRF service and (optionally) the\r
+ name of the OpenSRF method. If you do not pass a method name to the <literal>introspect</literal>\r
+ command, <literal>srfsh</literal> lists all of the methods offered by the service. If you pass\r
+ a partial method name, <literal>srfsh</literal> lists all of the methods that match that portion\r
+ of the method name.</simpara>\r
+ <note><simpara>The quality and availability of the descriptive information for each\r
+ method depends on the developer to register the method with complete and\r
+ accurate information. The quality varies across the set of OpenSRF and\r
+ Evergreen APIs, although some effort is being put towards improving the\r
+ state of the internal documentation.</simpara></note>\r
+<programlisting language="sh" linenumbering="unnumbered">\r
+srfsh# introspect opensrf.simple-text "opensrf.simple-text.reverse"\r
+--> opensrf.simple-text\r
+\r
+Received Data: {\r
+ "__c":"opensrf.simple-text",\r
+ "__p":{\r
+ "api_level":1,\r
+ "stream":0, <co id="CO3-1"/>\r
+ "object_hint":"OpenSRF_Application_Demo_SimpleText",\r
+ "remote":0,\r
+ "package":"OpenSRF::Application::Demo::SimpleText", <co id="CO3-2"/>\r
+ "api_name":"opensrf.simple-text.reverse",<co id="CO3-3"/>\r
+ "server_class":"opensrf.simple-text",\r
+ "signature":{ <co id="CO3-4"/>\r
+ "params":[ <co id="CO3-5"/>\r
+ {\r
+ "desc":"The string to reverse",\r
+ "name":"text",\r
+ "type":"string"\r
+ }\r
+ ],\r
+ "desc":"Returns the input string in reverse order\n", <co id="CO3-6"/>\r
+ "return":{ <co id="CO3-7"/>\r
+ "desc":"Returns the input string in reverse order",\r
+ "type":"string"\r
+ }\r
+ },\r
+ "method":"text_reverse", <co id="CO3-8"/>\r
+ "argc":1 <co id="CO3-9"/>\r
+ }\r
+}\r
+</programlisting>\r
+ <calloutlist>\r
+ <callout arearefs="CO3-1">\r
+ <simpara>\r
+ <literal>stream</literal> denotes whether the method supports streaming responses or not.\r
+ </simpara>\r
+ </callout>\r
+ <callout arearefs="CO3-2">\r
+ <simpara>\r
+ <literal>package</literal> identifies which package or library implements the method.\r
+ </simpara>\r
+ </callout>\r
+ <callout arearefs="CO3-3">\r
+ <simpara>\r
+ <literal>api_name</literal> identifies the name of the OpenSRF method.\r
+ </simpara>\r
+ </callout>\r
+ <callout arearefs="CO3-4">\r
+ <simpara>\r
+ <literal>signature</literal> is a hash that describes the parameters for the method.\r
+ </simpara>\r
+ </callout>\r
+ <callout arearefs="CO3-5">\r
+ <simpara>\r
+ <literal>params</literal> is an array of hashes describing each parameter in the method;\r
+ each parameter has a description (<literal>desc</literal>), name (<literal>name</literal>), and type (<literal>type</literal>).\r
+ </simpara>\r
+ </callout>\r
+ <callout arearefs="CO3-6">\r
+ <simpara>\r
+ <literal>desc</literal> is a string that describes the method itself.\r
+ </simpara>\r
+ </callout>\r
+ <callout arearefs="CO3-7">\r
+ <simpara>\r
+ <literal>return</literal> is a hash that describes the return value for the method; it\r
+ contains a description of the return value (<literal>desc</literal>) and the type of the\r
+ returned value (<literal>type</literal>).\r
+ </simpara>\r
+ </callout>\r
+ <callout arearefs="CO3-8">\r
+ <simpara>\r
+ <literal>method</literal> identifies the name of the function or method in the source\r
+ implementation.\r
+ </simpara>\r
+ </callout>\r
+ <callout arearefs="CO3-9">\r
+ <simpara>\r
+ <literal>argc</literal> is an integer describing the minimum number of arguments that\r
+ must be passed to this method.\r
+ </simpara>\r
+ </callout>\r
+ </calloutlist>\r
+ </simplesect>\r
+ <simplesect id="_calling_opensrf_methods_from_perl_applications">\r
+ <title>Calling OpenSRF methods from Perl applications</title>\r
+ <simpara>To call an OpenSRF method from Perl, you must connect to the OpenSRF service,\r
+ issue the request to the method, and then retrieve the results.</simpara>\r
+<programlisting language="perl" linenumbering="unnumbered">\r
+#/usr/bin/perl\r
+use strict;\r
+use OpenSRF::AppSession;\r
+use OpenSRF::System;\r
+\r
+OpenSRF::System->bootstrap_client(config_file => '/openils/conf/opensrf_core.xml');<co id="CO4-1"/>\r
+\r
+my $session = OpenSRF::AppSession->create("opensrf.simple-text");<co id="CO4-2"/>\r
+\r
+print "substring: Accepts a string and a number as input, returns a string\n";\r
+my $result = $session->request("opensrf.simple-text.substring", "foobar", 3);<co id="CO4-3"/>\r
+my $request = $result->gather(); <co id="CO4-4"/>\r
+print "Substring: $request\n\n";\r
+\r
+print "split: Accepts two strings as input, returns an array of strings\n";\r
+$request = $session->request("opensrf.simple-text.split", "This is a test", " ");<co id="CO4-5"/>\r
+my $output = "Split: [";\r
+my $element;\r
+while ($element = $request->recv()) { <co id="CO4-6"/>\r
+ $output .= $element->content . ", "; <co id="CO4-7"/>\r
+}\r
+$output =~ s/, $/]/;\r
+print $output . "\n\n";\r
+\r
+print "statistics: Accepts an array of strings as input, returns a hash\n";\r
+my @many_strings = [\r
+ "First I think I'll have breakfast",\r
+ "Then I think that lunch would be nice",\r
+ "And then seventy desserts to finish off the day"\r
+];\r
+\r
+$result = $session->request("opensrf.simple-text.statistics", @many_strings); <co id="CO4-8"/>\r
+$request = $result->gather(); <co id="CO4-9"/>\r
+print "Length: " . $result->{'length'} . "\n";\r
+print "Word count: " . $result->{'word_count'} . "\n";\r
+\r
+$session->disconnect(); <co id="CO4-10"/>\r
+</programlisting>\r
+ <calloutlist>\r
+ <callout arearefs="CO4-1">\r
+ <simpara>\r
+ The <literal>OpenSRF::System->bootstrap_client()</literal> method reads the OpenSRF\r
+ configuration information from the indicated file and creates an XMPP client\r
+ connection based on that information.\r
+ </simpara>\r
+ </callout>\r
+ <callout arearefs="CO4-2">\r
+ <simpara>\r
+ The <literal>OpenSRF::AppSession->create()</literal> method accepts one argument - the name\r
+ of the OpenSRF service to which you want to want to make one or more requests -\r
+ and returns an object prepared to use the client connection to make those\r
+ requests.\r
+ </simpara>\r
+ </callout>\r
+ <callout arearefs="CO4-3">\r
+ <simpara>\r
+ The <literal>OpenSRF::AppSession->request()</literal> method accepts a minimum of one\r
+ argument - the name of the OpenSRF method to which you want to make a request -\r
+ followed by zero or more arguments to pass to the OpenSRF method as input\r
+ values. This example passes a string and an integer to the\r
+ <literal>opensrf.simple-text.substring</literal> method defined by the <literal>opensrf.simple-text</literal>\r
+ OpenSRF service.\r
+ </simpara>\r
+ </callout>\r
+ <callout arearefs="CO4-4">\r
+ <simpara>\r
+ The <literal>gather()</literal> method, called on the result object returned by the\r
+ <literal>request()</literal> method, iterates over all of the possible results from the result\r
+ object and returns a single variable.\r
+ </simpara>\r
+ </callout>\r
+ <callout arearefs="CO4-5">\r
+ <simpara>\r
+ This <literal>request()</literal> call passes two strings to the <literal>opensrf.simple-text.split</literal>\r
+ method defined by the <literal>opensrf.simple-text</literal> OpenSRF service and returns (via\r
+ <literal>gather()</literal>) a reference to an array of results.\r
+ </simpara>\r
+ </callout>\r
+ <callout arearefs="CO4-6">\r
+ <simpara>\r
+ The <literal>opensrf.simple-text.split()</literal> method is a streaming method that\r
+ returns an array of results with one element per <literal>recv()</literal> call on the\r
+ result object. We could use the <literal>gather()</literal> method to retrieve all of the\r
+ results in a single array reference, but instead we simply iterate over\r
+ the result variable until there are no more results to retrieve.\r
+ </simpara>\r
+ </callout>\r
+ <callout arearefs="CO4-7">\r
+ <simpara>\r
+ While the <literal>gather()</literal> convenience method returns only the content of the\r
+ complete set of results for a given request, the <literal>recv()</literal> method returns an\r
+ OpenSRF result object with <literal>status</literal>, <literal>statusCode</literal>, and <literal>content</literal> fields as\r
+ we saw in <link linkend="OpenSRFOverHTTP">the HTTP results example</link>.\r
+ </simpara>\r
+ </callout>\r
+ <callout arearefs="CO4-8">\r
+ <simpara>\r
+ This <literal>request()</literal> call passes an array to the\r
+ <literal>opensrf.simple-text.statistics</literal> method defined by the <literal>opensrf.simple-text</literal>\r
+ OpenSRF service.\r
+ </simpara>\r
+ </callout>\r
+ <callout arearefs="CO4-9">\r
+ <simpara>\r
+ The result object returns a hash reference via <literal>gather()</literal>. The hash\r
+ contains the <literal>length</literal> and <literal>word_count</literal> keys we defined in the method.\r
+ </simpara>\r
+ </callout>\r
+ <callout arearefs="CO4-10">\r
+ <simpara>\r
+ The <literal>OpenSRF::AppSession->disconnect()</literal> method closes the XMPP client\r
+ connection and cleans up resources associated with the session.\r
+ </simpara>\r
+ </callout>\r
+ </calloutlist>\r
+ </simplesect>\r
+ </simplesect>\r
+ <simplesect id="_accepting_and_returning_more_interesting_data_types">\r
+ <title>Accepting and returning more interesting data types</title>\r
+ <simpara>Of course, the example of accepting a single string and returning a single\r
+ string is not very interesting. In real life, our applications tend to pass\r
+ around multiple arguments, including arrays and hashes. Fortunately, OpenSRF\r
+ makes that easy to deal with; in Perl, for example, returning a reference to\r
+ the data type does the right thing. In the following example of a method that\r
+ returns a list, we accept two arguments of type string: the string to be split,\r
+ and the delimiter that should be used to split the string.</simpara>\r
+ <formalpara><title>Basic text splitting method</title><para>\r
+<programlisting language="perl" linenumbering="unnumbered">\r
+sub text_split {\r
+ my $self = shift;\r
+ my $conn = shift;\r
+ my $text = shift;\r
+ my $delimiter = shift || ' ';\r
+\r
+ my @split_text = split $delimiter, $text;\r
+ return \@split_text;\r
+}\r
+\r
+__PACKAGE__->register_method(\r
+ method => 'text_split',\r
+ api_name => 'opensrf.simple-text.split'\r
+);\r
+</programlisting>\r
+ </para></formalpara>\r
+ <simpara>We simply return a reference to the list, and OpenSRF does the rest of the work\r
+ for us to convert the data into the language-independent format that is then\r
+ returned to the caller. As a caller of a given method, you must rely on the\r
+ documentation used to register to determine the data structures - if the developer has\r
+ added the appropriate documentation.</simpara>\r
+ </simplesect>\r
+ <simplesect id="_accepting_and_returning_evergreen_objects">\r
+ <title>Accepting and returning Evergreen objects</title>\r
+ <simpara>OpenSRF is agnostic about objects; its role is to pass JSON back and forth\r
+ between OpenSRF clients and services, and it allows the specific clients and\r
+ services to define their own semantics for the JSON structures. On top of that\r
+ infrastructure, Evergreen offers the fieldmapper: an object-relational mapper\r
+ that provides a complete definition of all objects, their properties, their\r
+ relationships to other objects, the permissions required to create, read,\r
+ update, or delete objects of that type, and the database table or view on which\r
+ they are based.</simpara>\r
+ <indexterm><primary>Fieldmapper</primary></indexterm>\r
+ <simpara>The Evergreen fieldmapper offers a great deal of convenience for working with\r
+ complex system objects beyond the basic mapping of classes to database\r
+ schemas. Although the result is passed over the wire as a JSON object\r
+ containing the indicated fields, fieldmapper-aware clients then turn those\r
+ JSON objects into native objects with setter / getter methods for each field.</simpara>\r
+ <simpara>All of this metadata about Evergreen objects is defined in the\r
+ fieldmapper configuration file (<literal>/openils/conf/fm_IDL.xml</literal>), and access to\r
+ these classes is provided by the <literal>open-ils.cstore</literal>, <literal>open-ils.pcrud</literal>, and\r
+ <literal>open-ils.reporter-store</literal> OpenSRF services which parse the fieldmapper\r
+ configuration file and dynamically register OpenSRF methods for creating,\r
+ reading, updating, and deleting all of the defined classes.</simpara>\r
+ <formalpara><title>Example fieldmapper class definition for "Open User Summary"</title><para>\r
+<programlisting language="xml" linenumbering="unnumbered">\r
+<class id="mous" controller="open-ils.cstore open-ils.pcrud"\r
+ oils_obj:fieldmapper="money::open_user_summary"\r
+ oils_persist:tablename="money.open_usr_summary"\r
+ reporter:label="Open User Summary"> <co id="CO5-1"/>\r
+ <fields oils_persist:primary="usr" oils_persist:sequence=""> <co id="CO5-2"/> \r
+ <field name="balance_owed" reporter:datatype="money" /> <co id="CO5-3"/> \r
+ <field name="total_owed" reporter:datatype="money" />\r
+ <field name="total_paid" reporter:datatype="money" />\r
+ <field name="usr" reporter:datatype="link"/>\r
+ </fields>\r
+ <links>\r
+ <link field="usr" reltype="has_a" key="id" map="" class="au"/><co id="CO5-4"/> \r
+ </links>\r
+ <permacrud xmlns="http://open-ils.org/spec/opensrf/IDL/permacrud/v1"><co id="CO5-5"/> \r
+ <actions>\r
+ <retrieve permission="VIEW_USER"><co id="CO5-6"/> \r
+ <context link="usr" field="home_ou"/><co id="CO5-7"/>\r
+ </retrieve>\r
+ </actions>\r
+ </permacrud>\r
+</class>\r
+</programlisting>\r
+ </para></formalpara>\r
+ <calloutlist>\r
+ <callout arearefs="CO5-1">\r
+ <simpara>\r
+ The <literal><class></literal> element defines the class:\r
+ </simpara>\r
+ <itemizedlist>\r
+ <listitem>\r
+ <simpara>\r
+ The <literal>id</literal> attribute defines the <emphasis>class hint</emphasis> that identifies the class both\r
+ elsewhere in the fieldmapper configuration file, such as in the value of the\r
+ <literal>field</literal> attribute of the <literal><link></literal> element, and in the JSON object itself when\r
+ it is instantiated. For example, an "Open User Summary" JSON object would have\r
+ the top level property of <literal>"__c":"mous"</literal>.\r
+ </simpara>\r
+ </listitem>\r
+ <listitem>\r
+ <simpara>\r
+ The <literal>controller</literal> attribute identifies the services that have direct access\r
+ to this class. If <literal>open-ils.pcrud</literal> is not listed, for example, then there is\r
+ no means to directly access members of this class through a public service.\r
+ </simpara>\r
+ </listitem>\r
+ <listitem>\r
+ <simpara>\r
+ The <literal>oils_obj:fieldmapper</literal> attribute defines the name of the Perl\r
+ fieldmapper class that will be dynamically generated to provide setter and\r
+ getter methods for instances of the class.\r
+ </simpara>\r
+ </listitem>\r
+ <listitem>\r
+ <simpara>\r
+ The <literal>oils_persist:tablename</literal> attribute identifies the schema name and table\r
+ name of the database table that stores the data that represents the instances\r
+ of this class. In this case, the schema is <literal>money</literal> and the table is\r
+ <literal>open_usr_summary</literal>.\r
+ </simpara>\r
+ </listitem>\r
+ <listitem>\r
+ <simpara>\r
+ The <literal>reporter:label</literal> attribute defines a human-readable name for the class\r
+ used in the reporting interface to identify the class. These names are defined\r
+ in English in the fieldmapper configuration file; however, they are extracted\r
+ so that they can be translated and served in the user’s language of choice.\r
+ </simpara>\r
+ </listitem>\r
+ </itemizedlist>\r
+ </callout>\r
+ <callout arearefs="CO5-2">\r
+ <simpara>\r
+ The <literal><fields></literal> element lists all of the fields that belong to the object.\r
+ </simpara>\r
+ <itemizedlist>\r
+ <listitem>\r
+ <simpara>\r
+ The <literal>oils_persist:primary</literal> attribute identifies the field that acts as the\r
+ primary key for the object; in this case, the field with the name <literal>usr</literal>.\r
+ </simpara>\r
+ </listitem>\r
+ <listitem>\r
+ <simpara>\r
+ The <literal>oils_persist:sequence</literal> attribute identifies the sequence object\r
+ (if any) in this database provides values for new instances of this class. In\r
+ this case, the primary key is defined by a field that is linked to a different\r
+ table, so no sequence is used to populate these instances.\r
+ </simpara>\r
+ </listitem>\r
+ </itemizedlist>\r
+ </callout>\r
+ <callout arearefs="CO5-3">\r
+ <simpara>\r
+ Each <literal><field></literal> element defines a single field with the following attributes:\r
+ </simpara>\r
+ <itemizedlist>\r
+ <listitem>\r
+ <simpara>\r
+ The <literal>name</literal> attribute identifies the column name of the field in the\r
+ underlying database table as well as providing a name for the setter / getter\r
+ method that can be invoked in the JSON or native version of the object.\r
+ </simpara>\r
+ </listitem>\r
+ <listitem>\r
+ <simpara>\r
+ The <literal>reporter:datatype</literal> attribute defines how the reporter should treat\r
+ the contents of the field for the purposes of querying and display.\r
+ </simpara>\r
+ </listitem>\r
+ <listitem>\r
+ <simpara>\r
+ The <literal>reporter:label</literal> attribute can be used to provide a human-readable name\r
+ for each field; without it, the reporter falls back to the value of the <literal>name</literal>\r
+ attribute.\r
+ </simpara>\r
+ </listitem>\r
+ </itemizedlist>\r
+ </callout>\r
+ <callout arearefs="CO5-4">\r
+ <simpara>\r
+ The <literal><links></literal> element contains a set of zero or more <literal><link></literal> elements,\r
+ each of which defines a relationship between the class being described and\r
+ another class.\r
+ </simpara>\r
+ <itemizedlist>\r
+ <listitem>\r
+ <simpara>\r
+ The <literal>field</literal> attribute identifies the field named in this class that links\r
+ to the external class.\r
+ </simpara>\r
+ </listitem>\r
+ <listitem>\r
+ <simpara>\r
+ The <literal>reltype</literal> attribute identifies the kind of relationship between the\r
+ classes; in the case of <literal>has_a</literal>, each value in the <literal>usr</literal> field is guaranteed\r
+ to have a corresponding value in the external class.\r
+ </simpara>\r
+ </listitem>\r
+ <listitem>\r
+ <simpara>\r
+ The <literal>key</literal> attribute identifies the name of the field in the external\r
+ class to which this field links.\r
+ </simpara>\r
+ </listitem>\r
+ <listitem>\r
+ <simpara>\r
+ The rarely-used <literal>map</literal> attribute identifies a second class to which\r
+ the external class links; it enables this field to define a direct\r
+ relationship to an external class with one degree of separation, to\r
+ avoid having to retrieve all of the linked members of an intermediate\r
+ class just to retrieve the instances from the actual desired target class.\r
+ </simpara>\r
+ </listitem>\r
+ <listitem>\r
+ <simpara>\r
+ The <literal>class</literal> attribute identifies the external class to which this field\r
+ links.\r
+ </simpara>\r
+ </listitem>\r
+ </itemizedlist>\r
+ </callout>\r
+ <callout arearefs="CO5-5">\r
+ <simpara>\r
+ The <literal><permacrud></literal> element defines the permissions that must have been\r
+ granted to a user to operate on instances of this class.\r
+ </simpara>\r
+ </callout>\r
+ <callout arearefs="CO5-6">\r
+ <simpara>\r
+ The <literal><retrieve></literal> element is one of four possible children of the\r
+ <literal><actions></literal> element that define the permissions required for each action:\r
+ create, retrieve, update, and delete.\r
+ </simpara>\r
+ <itemizedlist>\r
+ <listitem>\r
+ <simpara>\r
+ The <literal>permission</literal> attribute identifies the name of the permission that must\r
+ have been granted to the user to perform the action.\r
+ </simpara>\r
+ </listitem>\r
+ <listitem>\r
+ <simpara>\r
+ The <literal>contextfield</literal> attribute, if it exists, defines the field in this class\r
+ that identifies the library within the system for which the user must have\r
+ prvileges to work. If a user has been granted a given permission, but has not been\r
+ granted privileges to work at a given library, they can not perform the action\r
+ at that library.\r
+ </simpara>\r
+ </listitem>\r
+ </itemizedlist>\r
+ </callout>\r
+ <callout arearefs="CO5-7">\r
+ <simpara>\r
+ The rarely-used <literal><context></literal> element identifies a linked field (<literal>link</literal>\r
+ attribute) in this class which links to an external class that holds the field\r
+ (<literal>field</literal> attribute) that identifies the library within the system for which the\r
+ user must have privileges to work.\r
+ </simpara>\r
+ </callout>\r
+ </calloutlist>\r
+ <simpara>When you retrieve an instance of a class, you can ask for the result to\r
+ <emphasis>flesh</emphasis> some or all of the linked fields of that class, so that the linked\r
+ instances are returned embedded directly in your requested instance. In that\r
+ same request you can ask for the fleshed instances to in turn have their linked\r
+ fields fleshed. By bundling all of this into a single request and result\r
+ sequence, you can avoid the network overhead of requiring the client to request\r
+ the base object, then request each linked object in turn.</simpara>\r
+ <simpara>You can also iterate over a collection of instances and set the automatically\r
+ generated <literal>isdeleted</literal>, <literal>isupdated</literal>, or <literal>isnew</literal> properties to indicate that\r
+ the given instance has been deleted, updated, or created respectively.\r
+ Evergreen can then act in batch mode over the collection to perform the\r
+ requested actions on any of the instances that have been flagged for action.</simpara>\r
+ </simplesect>\r
+ <simplesect id="_returning_streaming_results">\r
+ <title>Returning streaming results</title>\r
+ <simpara>In the previous implementation of the <literal>opensrf.simple-text.split</literal> method, we\r
+ returned a reference to the complete array of results. For small values being\r
+ delivered over the network, this is perfectly acceptable, but for large sets of\r
+ values this can pose a number of problems for the requesting client. Consider a\r
+ service that returns a set of bibliographic records in response to a query like\r
+ "all records edited in the past month"; if the underlying database is\r
+ relatively active, that could result in thousands of records being returned as\r
+ a single network request. The client would be forced to block until all of the\r
+ results are returned, likely resulting in a significant delay, and depending on\r
+ the implementation, correspondingly large amounts of memory might be consumed\r
+ as all of the results are read from the network in a single block.</simpara>\r
+ <simpara>OpenSRF offers a solution to this problem. If the method returns results that\r
+ can be divided into separate meaningful units, you can register the OpenSRF\r
+ method as a streaming method and enable the client to loop over the results one\r
+ unit at a time until the method returns no further results. In addition to\r
+ registering the method with the provided name, OpenSRF also registers an additional\r
+ method with <literal>.atomic</literal> appended to the method name. The <literal>.atomic</literal> variant gathers\r
+ all of the results into a single block to return to the client, giving the caller\r
+ the ability to choose either streaming or atomic results from a single method\r
+ definition.</simpara>\r
+ <simpara>In the following example, the text splitting method has been reimplemented to\r
+ support streaming; very few changes are required:</simpara>\r
+ <formalpara><title>Text splitting method - streaming mode</title><para>\r
+<programlisting language="perl" linenumbering="unnumbered">\r
+sub text_split {\r
+ my $self = shift;\r
+ my $conn = shift;\r
+ my $text = shift;\r
+ my $delimiter = shift || ' ';\r
+\r
+ my @split_text = split $delimiter, $text;\r
+ foreach my $string (@split_text) { <co id="CO6-1"/>\r
+ $conn->respond($string);\r
+ }\r
+ return undef;\r
+}\r
+\r
+__PACKAGE__->register_method(\r
+ method => 'text_split',\r
+ api_name => 'opensrf.simple-text.split',\r
+ stream => 1<co id="CO6-2"/>\r
+);\r
+</programlisting>\r
+ </para></formalpara>\r
+ <calloutlist>\r
+ <callout arearefs="CO6-1">\r
+ <simpara>\r
+ Rather than returning a reference to the array, a streaming method loops\r
+ over the contents of the array and invokes the <literal>respond()</literal> method of the\r
+ connection object on each element of the array.\r
+ </simpara>\r
+ </callout>\r
+ <callout arearefs="CO6-2">\r
+ <simpara>\r
+ Registering the method as a streaming method instructs OpenSRF to also\r
+ register an atomic variant (<literal>opensrf.simple-text.split.atomic</literal>).\r
+ </simpara>\r
+ </callout>\r
+ </calloutlist>\r
+ </simplesect>\r
+ <simplesect id="_error_warning_info_debug">\r
+ <title>Error! Warning! Info! Debug!</title>\r
+ <simpara>As hard as it may be to believe, it is true: applications sometimes do not\r
+ behave in the expected manner, particularly when they are still under\r
+ development. The service language bindings for OpenSRF include integrated\r
+ support for logging messages at the levels of ERROR, WARNING, INFO, DEBUG, and\r
+ the extremely verbose INTERNAL to either a local file or to a syslogger\r
+ service. The destination of the log files, and the level of verbosity to be\r
+ logged, is set in the <literal>opensrf_core.xml</literal> configuration file. To add logging to\r
+ our Perl example, we just have to add the <literal>OpenSRF::Utils::Logger</literal> package to our\r
+ list of used Perl modules, then invoke the logger at the desired logging level.</simpara>\r
+ <simpara>You can include many calls to the OpenSRF logger; only those that are higher\r
+ than your configured logging level will actually hit the log. The following\r
+ example exercises all of the available logging levels in OpenSRF:</simpara>\r
+<programlisting language="perl" linenumbering="unnumbered">\r
+use OpenSRF::Utils::Logger;\r
+my $logger = OpenSRF::Utils::Logger;\r
+# some code in some function\r
+{\r
+ $logger->error("Hmm, something bad DEFINITELY happened!");\r
+ $logger->warn("Hmm, something bad might have happened.");\r
+ $logger->info("Something happened.");\r
+ $logger->debug("Something happened; here are some more details.");\r
+ $logger->internal("Something happened; here are all the gory details.")\r
+}\r
+</programlisting>\r
+ <simpara>If you call the mythical OpenSRF method containing the preceding OpenSRF logger\r
+ statements on a system running at the default logging level of INFO, you will\r
+ only see the INFO, WARN, and ERR messages, as follows:</simpara>\r
+ <formalpara><title>Results of logging calls at the default level of INFO</title><para>\r
+<screen>\r
+[2010-03-17 22:27:30] opensrf.simple-text [ERR :5681:SimpleText.pm:277:] \r
+[2010-03-17 22:27:30] opensrf.simple-text [WARN:5681:SimpleText.pm:278:] \r
+[2010-03-17 22:27:30] opensrf.simple-text [INFO:5681:SimpleText.pm:279:] \r
+</screen>\r
+ </para></formalpara>\r
+ <simpara>If you then increase the the logging level to INTERNAL (5), the logs will\r
+ contain much more information, as follows:</simpara>\r
+ <formalpara><title>Results of logging calls at the default level of INTERNAL</title><para>\r
+<screen>\r
+[2010-03-17 22:48:11] opensrf.simple-text [ERR :5934:SimpleText.pm:277:] \r
+[2010-03-17 22:48:11] opensrf.simple-text [WARN:5934:SimpleText.pm:278:] \r
+[2010-03-17 22:48:11] opensrf.simple-text [INFO:5934:SimpleText.pm:279:] \r
+[2010-03-17 22:48:11] opensrf.simple-text [DEBG:5934:SimpleText.pm:280:] \r
+[2010-03-17 22:48:11] opensrf.simple-text [INTL:5934:SimpleText.pm:281:] \r
+[2010-03-17 22:48:11] opensrf.simple-text [ERR :5934:SimpleText.pm:283:] \r
+[2010-03-17 22:48:21] opensrf.simple-text [INTL:5934:Cache.pm:125:] \r
+[2010-03-17 22:48:21] opensrf.simple-text [DEBG:5934:Application.pm:579:] \r
+[2010-03-17 22:48:21] opensrf.simple-text [DEBG:5934:Application.pm:586:] \r
+[2010-03-17 22:48:21] opensrf.simple-text [DEBG:5934:Application.pm:190:] \r
+[2010-03-17 22:48:21] opensrf.simple-text [INTL:5934:AppSession.pm:780:] Calling queue_wait(0)\r
+[2010-03-17 22:48:21] opensrf.simple-text [INTL:5934:AppSession.pm:769:] Resending...0\r
+[2010-03-17 22:48:21] opensrf.simple-text [INTL:5934:AppSession.pm:450:] In send\r
+[2010-03-17 22:48:21] opensrf.simple-text [DEBG:5934:AppSession.pm:506:] \r
+[2010-03-17 22:48:21] opensrf.simple-text [DEBG:5934:AppSession.pm:506:] \r
+...\r
+</screen>\r
+ </para></formalpara>\r
+ <simpara>To see everything that is happening in OpenSRF, try leaving your logging level\r
+ set to INTERNAL for a few minutes - just ensure that you have a lot of free disk\r
+ space available if you have a moderately busy system!</simpara>\r
+ </simplesect>\r
+ <simplesect id="_caching_results_one_secret_of_scalability">\r
+ <title>Caching results: one secret of scalability</title>\r
+ <indexterm><primary>search results</primary><secondary>caching</secondary></indexterm>\r
+ <simpara>If you have ever used an application that depends on a remote Web service\r
+ outside of your control — say, if you need to retrieve results from a\r
+ microblogging service — you know the pain of latency and dependability (or the\r
+ lack thereof). To improve the response time for OpenSRF services, you can take\r
+ advantage of the support offered by the <literal>OpenSRF::Utils::Cache</literal> module for\r
+ communicating with a local instance or cluster of <literal>memcache</literal> daemons to store\r
+ and retrieve persistent values. The following example demonstrates caching\r
+ by sleeping for 10 seconds the first time it receives a given cache key and\r
+ cannot retrieve a corresponding value from the cache:</simpara>\r
+ <formalpara><title>Simple caching OpenSRF service</title><para>\r
+<programlisting language="perl" linenumbering="unnumbered">\r
+use OpenSRF::Utils::Cache;<co id="CO7-1"/>\r
+sub test_cache {\r
+ my $self = shift;\r
+ my $conn = shift;\r
+ my $test_key = shift;\r
+ my $cache = OpenSRF::Utils::Cache->new('global'); <co id="CO7-2"/>\r
+ my $cache_key = "opensrf.simple-text.test_cache.$test_key"; <co id="CO7-3"/>\r
+ my $result = $cache->get_cache($cache_key) || undef; <co id="CO7-4"/>\r
+ if ($result) {\r
+ $logger->info("Resolver found a cache hit");\r
+ return $result;\r
+ }\r
+ sleep 10; <co id="CO7-5"/>\r
+ my $cache_timeout = 300; <co id="CO7-6"/>\r
+ $cache->put_cache($cache_key, "here", $cache_timeout); <co id="CO7-7"/>\r
+ return "There was no cache hit.";\r
+}\r
+</programlisting>\r
+ </para></formalpara>\r
+ <calloutlist>\r
+ <callout arearefs="CO7-1">\r
+ <simpara>\r
+ The OpenSRF::Utils::Cache module provides access to the built-in caching\r
+ support in OpenSRF.\r
+ </simpara>\r
+ </callout>\r
+ <callout arearefs="CO7-2">\r
+ <simpara>\r
+ The constructor for the cache object accepts a single argument to define\r
+ the cache type for the object. Each cache type can use a separate <literal>memcache</literal>\r
+ server to keep the caches separated. Most Evergreen services use the <literal>global</literal>\r
+ cache, while the <literal>anon</literal> cache is used for Web sessions.\r
+ </simpara>\r
+ </callout>\r
+ <callout arearefs="CO7-3">\r
+ <simpara>\r
+ The cache key is simply a string that uniquely identifies the value you\r
+ want to store or retrieve. This line creates a cache key based on the OpenSRF\r
+ method name and request input value.\r
+ </simpara>\r
+ </callout>\r
+ <callout arearefs="CO7-4">\r
+ <simpara>\r
+ The <literal>get_cache()</literal> method checks to see if the cache key already exists. If\r
+ a matching key is found, the service immediately returns the stored value.\r
+ </simpara>\r
+ </callout>\r
+ <callout arearefs="CO7-5">\r
+ <simpara>\r
+ If the cache key does not exist, the code sleeps for 10 seconds to\r
+ simulate a call to a slow remote Web service or an intensive process.\r
+ </simpara>\r
+ </callout>\r
+ <callout arearefs="CO7-6">\r
+ <simpara>\r
+ The <literal>$cache_timeout</literal> variable represents a value for the lifetime of the\r
+ cache key in seconds.\r
+ </simpara>\r
+ </callout>\r
+ <callout arearefs="CO7-7">\r
+ <simpara>\r
+ After the code retrieves its value (or, in the case of this example,\r
+ finishes sleeping), it creates the cache entry by calling the <literal>put_cache()</literal>\r
+ method. The method accepts three arguments: the cache key, the value to be\r
+ stored ("here"), and the timeout value in seconds to ensure that we do not\r
+ return stale data on subsequent calls.\r
+ </simpara>\r
+ </callout>\r
+ </calloutlist>\r
+ </simplesect>\r
+ <simplesect id="_initializing_the_service_and_its_children_child_labour">\r
+ <title>Initializing the service and its children: child labour</title>\r
+ <simpara>When an OpenSRF service is started, it looks for a procedure called\r
+ <literal>initialize()</literal> to set up any global variables shared by all of the children of\r
+ the service. The <literal>initialize()</literal> procedure is typically used to retrieve\r
+ configuration settings from the <literal>opensrf.xml</literal> file.</simpara>\r
+ <simpara>An OpenSRF service spawns one or more children to actually do the work\r
+ requested by callers of the service. For every child process an OpenSRF service\r
+ spawns, the child process clones the parent environment and then each child\r
+ process runs the <literal>child_init()</literal> process (if any) defined in the OpenSRF service\r
+ to initialize any child-specific settings.</simpara>\r
+ <simpara>When the OpenSRF service kills a child process, it invokes the <literal>child_exit()</literal>\r
+ procedure (if any) to clean up any resources associated with the child process.\r
+ Similarly, when the OpenSRF service is stopped, it calls the <literal>DESTROY()</literal>\r
+ procedure to clean up any remaining resources.</simpara>\r
+ </simplesect>\r
+ <simplesect id="_retrieving_configuration_settings">\r
+ <title>Retrieving configuration settings</title>\r
+ <simpara>The settings for OpenSRF services are maintained in the <literal>opensrf.xml</literal> XML\r
+ configuration file. The structure of the XML document consists of a root\r
+ element <literal><opensrf></literal> containing two child elements:</simpara>\r
+ <itemizedlist>\r
+ <listitem>\r
+ <simpara>\r
+ The <literal><default></literal> element contains an <literal><apps></literal> element describing all\r
+ OpenSRF services running on this system — see <xref linkend="serviceRegistration"/> --, as\r
+ well as any other arbitrary XML descriptions required for global configuration\r
+ purposes. For example, Evergreen uses this section for email notification and\r
+ inter-library patron privacy settings.\r
+ </simpara>\r
+ </listitem>\r
+ <listitem>\r
+ <simpara>\r
+ The <literal><hosts></literal> element contains one element per host that participates in\r
+ this OpenSRF system. Each host element must include an <literal><activeapps></literal> element\r
+ that lists all of the services to start on this host when the system starts\r
+ up. Each host element can optionally override any of the default settings.\r
+ </simpara>\r
+ </listitem>\r
+ </itemizedlist>\r
+ <simpara>OpenSRF includes a service named <literal>opensrf.settings</literal> to provide distributed\r
+ cached access to the configuration settings with a simple API:</simpara>\r
+ <itemizedlist>\r
+ <listitem>\r
+ <simpara>\r
+ <literal>opensrf.settings.default_config.get</literal> accepts zero arguments and returns\r
+ the complete set of default settings as a JSON document.\r
+ </simpara>\r
+ </listitem>\r
+ <listitem>\r
+ <simpara>\r
+ <literal>opensrf.settings.host_config.get</literal> accepts one argument (hostname) and\r
+ returns the complete set of settings, as customized for that hostname, as a\r
+ JSON document.\r
+ </simpara>\r
+ </listitem>\r
+ <listitem>\r
+ <simpara>\r
+ <literal>opensrf.settings.xpath.get</literal> accepts one argument (an\r
+ <ulink url="http://www.w3.org/TR/xpath/">XPath</ulink> expression) and returns the portion of\r
+ the configuration file that matches the expression as a JSON document.\r
+ </simpara>\r
+ </listitem>\r
+ </itemizedlist>\r
+ <simpara>For example, to determine whether an Evergreen system uses the opt-in\r
+ support for sharing patron information between libraries, you could either\r
+ invoke the <literal>opensrf.settings.default_config.get</literal> method and parse the\r
+ JSON document to determine the value, or invoke the <literal>opensrf.settings.xpath.get</literal>\r
+ method with the XPath <literal>/opensrf/default/share/user/opt_in</literal> argument to\r
+ retrieve the value directly.</simpara>\r
+ <simpara>In practice, OpenSRF includes convenience libraries in all of its client\r
+ language bindings to simplify access to configuration values. C offers\r
+ osrfConfig.c, Perl offers <literal>OpenSRF::Utils::SettingsClient</literal>, Java offers\r
+ <literal>org.opensrf.util.SettingsClient</literal>, and Python offers <literal>osrf.set</literal>. These\r
+ libraries locally cache the configuration file to avoid network roundtrips for\r
+ every request and enable the developer to request specific values without\r
+ having to manually construct XPath expressions.</simpara>\r
+ </simplesect>\r
+ </section>\r
+ <section id="_getting_under_the_covers_with_opensrf">\r
+ <title>OpenSRF Communication Flows</title>\r
+ <indexterm><primary>OpenSRF</primary><secondary>Communication Flows</secondary></indexterm>\r
+ <simpara>Now that you have seen that it truly is easy to create an OpenSRF service, we\r
+ can take a look at what is going on under the covers to make all of this work\r
+ for you.</simpara>\r
+ <simplesect id="_get_on_the_messaging_bus_safely">\r
+ <title>Get on the messaging bus - safely</title>\r
+ <simpara>One of the core innovations of OpenSRF was to use the Extensible Messaging and\r
+ Presence Protocol (XMPP, more colloquially known as Jabber) as the messaging\r
+ bus that ties OpenSRF services together across servers. XMPP is an "XML\r
+ protocol for near-real-time messaging, presence, and request-response services"\r
+ (<ulink url="http://www.ietf.org/rfc/rfc3920.txt">http://www.ietf.org/rfc/rfc3920.txt</ulink>) that OpenSRF relies on to handle most of\r
+ the complexity of networked communications. OpenSRF requres an XMPP server\r
+ that supports multiple domains such as <ulink url="http://www.ejabberd.im/">ejabberd</ulink>.\r
+ Multiple domain support means that a single server can support XMPP virtual\r
+ hosts with separate sets of users and access privileges per domain. By\r
+ routing communications through separate public and private XMPP domains,\r
+ OpenSRF services gain an additional layer of security.</simpara>\r
+ <simpara>The <ulink url="http://evergreen-ils.org/dokuwiki/doku.php?id=opensrf:1.2:install">OpenSRF\r
+ installation documentation</ulink> instructs you to create two separate hostnames\r
+ (<literal>private.localhost</literal> and <literal>public.localhost</literal>) to use as XMPP domains. OpenSRF\r
+ can control access to its services based on the domain of the client and\r
+ whether a given service allows access from clients on the public domain. When\r
+ you start OpenSRF, the first XMPP clients that connect to the XMPP server are\r
+ the OpenSRF public and private <emphasis>routers</emphasis>. OpenSRF routers maintain a list of\r
+ available services and connect clients to available services. When an OpenSRF\r
+ service starts, it establishes a connection to the XMPP server and registers\r
+ itself with the private router. The OpenSRF configuration contains a list of\r
+ public OpenSRF services, each of which must also register with the public\r
+ router.</simpara>\r
+ </simplesect>\r
+ <simplesect id="_opensrf_communication_flows_over_xmpp">\r
+ <title>OpenSRF communication flows over XMPP</title>\r
+ <indexterm><primary>XMPP</primary></indexterm>\r
+ <simpara>In a minimal OpenSRF deployment, two XMPP users named "router" connect to the\r
+ XMPP server, with one connected to the private XMPP domain and one connected to\r
+ the public XMPP domain. Similarly, two XMPP users named "opensrf" connect to\r
+ the XMPP server via the private and public XMPP domains. When an OpenSRF\r
+ service is started, it uses the "opensrf" XMPP user to advertise its\r
+ availability with the corresponding router on that XMPP domain; the XMPP server\r
+ automatically assigns a Jabber ID (<emphasis>JID</emphasis>) based on the client hostname to each\r
+ service’s listener process and each connected drone process waiting to carry\r
+ out requests. When an OpenSRF router receives a request to invoke a method on a\r
+ given service, it connects the requester to the next available listener in the\r
+ list of registered listeners for that service.</simpara>\r
+ <simpara>Services and clients connect to the XMPP server using a single set of XMPP\r
+ client credentials (for example, <literal>opensrf@private.localhost</literal>), but use XMPP\r
+ resource identifiers to differentiate themselves in the JID for each\r
+ connection. For example, the JID for a copy of the <literal>opensrf.simple-text</literal>\r
+ service with process ID <literal>6285</literal> that has connected to the <literal>private.localhost</literal>\r
+ domain using the <literal>opensrf</literal> XMPP client credentials could be\r
+ <literal>opensrf@private.localhost/opensrf.simple-text_drone_at_localhost_6285</literal>. By\r
+ convention, the user name for OpenSRF clients is <literal>opensrf</literal>, and the user name\r
+ for OpenSRF routers is <literal>router</literal>, so the XMPP server for OpenSRF will have four\r
+ separate users registered:\r
+ * <literal>opensrf@private.localhost</literal> is an OpenSRF client that connects with these\r
+ credentials and which can access any OpenSRF service.\r
+ * <literal>opensrf@public.localhost</literal> is an OpenSRF client that connects with these\r
+ credentials and which can only access OpenSRF services that have registered\r
+ with the public router.\r
+ * <literal>router@private.localhost</literal> is the private OpenSRF router with which all\r
+ services register.\r
+ * <literal>router@public.localhost</literal> is the public OpenSRF router with which only\r
+ services that must be publicly accessible register.</simpara>\r
+ <simpara>All OpenSRF services automatically register themselves with the private XMPP\r
+ domain, but only those services that register themselves with the public XMPP\r
+ domain can be invoked from public OpenSRF clients. The OpenSRF client and\r
+ router user names, passwords, and domain names, along with the list of services\r
+ that should be public, are contained in the <literal>opensrf_core.xml</literal> configuration\r
+ file.</simpara>\r
+ </simplesect>\r
+ <simplesect id="OpenSRFOverHTTP">\r
+ <title>OpenSRF communication flows over HTTP</title>\r
+ <indexterm><primary>HTTP</primary><secondary>translator</secondary></indexterm>\r
+ <simpara>In some contexts, access to a full XMPP client is not a practical option. For\r
+ example, while XMPP clients have been implemented in JavaScript, you might\r
+ be concerned about browser compatibility and processing overhead - or you might\r
+ want to issue OpenSRF requests from the command line with <literal>curl</literal>. Fortunately,\r
+ any OpenSRF service registered with the public router is accessible via the\r
+ OpenSRF HTTP Translator. The OpenSRF HTTP Translator implements the\r
+ <ulink url="http://www.open-ils.org/dokuwiki/doku.php?id=opensrf_over_http">OpenSRF-over-HTTP\r
+ proposed specification</ulink> as an Apache module that translates HTTP requests into\r
+ OpenSRF requests and returns OpenSRF results as HTTP results to the initiating\r
+ HTTP client.</simpara>\r
+ <formalpara><title>Issuing an HTTP POST request to an OpenSRF method via the OpenSRF HTTP Translator</title><para>\r
+<programlisting language="bash" linenumbering="unnumbered">\r
+# curl request broken up over multiple lines for legibility\r
+curl -H "X-OpenSRF-service: opensrf.simple-text"<co id="CO8-1"/>\r
+ --data 'osrf-msg=[ \<co id="CO8-2"/>\r
+ {"__c":"osrfMessage","__p":{"threadTrace":0,"locale":"en-CA", <co id="CO8-3"/>\r
+ "type":"REQUEST","payload": {"__c":"osrfMethod","__p": \r
+ {"method":"opensrf.simple-text.reverse","params":["foobar"]} \r
+ }} \r
+ }]' \r
+http://localhost/osrf-http-translator <co id="CO8-4"/>\r
+</programlisting>\r
+ </para></formalpara>\r
+ <calloutlist>\r
+ <callout arearefs="CO8-1">\r
+ <simpara>\r
+ The <literal>X-OpenSRF-service</literal> header identifies the OpenSRF service of interest.\r
+ </simpara>\r
+ </callout>\r
+ <callout arearefs="CO8-2">\r
+ <simpara>\r
+ The POST request consists of a single parameter, the <literal>osrf-msg</literal> value,\r
+ which contains a JSON array.\r
+ </simpara>\r
+ </callout>\r
+ <callout arearefs="CO8-3">\r
+ <simpara>\r
+ The first object is an OpenSRF message (<literal>"__c":"osrfMessage"</literal>) with a set of\r
+ parameters (<literal>"__p":{}</literal>).\r
+ </simpara>\r
+ <itemizedlist>\r
+ <listitem>\r
+ <simpara>\r
+ The identifier for the request (<literal>"threadTrace":0</literal>); this value is echoed\r
+ back in the result.\r
+ </simpara>\r
+ </listitem>\r
+ <listitem>\r
+ <simpara>\r
+ The message type (<literal>"type":"REQUEST"</literal>).\r
+ </simpara>\r
+ </listitem>\r
+ <listitem>\r
+ <simpara>\r
+ The locale for the message; if the OpenSRF method is locale-sensitive, it\r
+ can check the locale for each OpenSRF request and return different information\r
+ depending on the locale.\r
+ </simpara>\r
+ </listitem>\r
+ <listitem>\r
+ <simpara>\r
+ The payload of the message (<literal>"payload":{}</literal>) containing the OpenSRF method\r
+ request (<literal>"__c":"osrfMethod"</literal>) and its parameters (<literal>"__p:"{}</literal>).\r
+ </simpara>\r
+ <itemizedlist>\r
+ <listitem>\r
+ <simpara>\r
+ The method name for the request (<literal>"method":"opensrf.simple-text.reverse"</literal>).\r
+ </simpara>\r
+ </listitem>\r
+ <listitem>\r
+ <simpara>\r
+ A set of JSON parameters to pass to the method (<literal>"params":["foobar"]</literal>); in\r
+ this case, a single string <literal>"foobar"</literal>.\r
+ </simpara>\r
+ </listitem>\r
+ </itemizedlist>\r
+ </listitem>\r
+ </itemizedlist>\r
+ </callout>\r
+ <callout arearefs="CO8-4">\r
+ <simpara>\r
+ The URL on which the OpenSRF HTTP translator is listening,\r
+ <literal>/osrf-http-translator</literal> is the default location in the Apache example\r
+ configuration files shipped with the OpenSRF source, but this is configurable.\r
+ </simpara>\r
+ </callout>\r
+ </calloutlist>\r
+ <formalpara><title>Results from an HTTP POST request to an OpenSRF method via the OpenSRF HTTP Translator</title><para>\r
+<programlisting language="bash" linenumbering="unnumbered">\r
+# HTTP response broken up over multiple lines for legibility\r
+[{"__c":"osrfMessage","__p": <co id="CO9-1"/>\r
+ {"threadTrace":0, "payload": <co id="CO9-2"/>\r
+ {"__c":"osrfResult","__p": <co id="CO9-3"/>\r
+ {"status":"OK","content":"raboof","statusCode":200} <co id="CO9-4"/>\r
+ },"type":"RESULT","locale":"en-CA" <co id="CO9-5"/>\r
+ }\r
+},\r
+{"__c":"osrfMessage","__p": <co id="CO9-6"/>\r
+ {"threadTrace":0,"payload": <co id="CO9-7"/>\r
+ {"__c":"osrfConnectStatus","__p": <co id="CO9-8"/>\r
+ {"status":"Request Complete","statusCode":205}<co id="CO9-9"/>\r
+ },"type":"STATUS","locale":"en-CA" <co id="CO9-10"/>\r
+ }\r
+}]\r
+</programlisting>\r
+ </para></formalpara>\r
+ <calloutlist>\r
+ <callout arearefs="CO9-1">\r
+ <simpara>\r
+ The OpenSRF HTTP Translator returns an array of JSON objects in its\r
+ response. Each object in the response is an OpenSRF message\r
+ (<literal>"__c":"osrfMessage"</literal>) with a collection of response parameters (<literal>"__p":</literal>).\r
+ </simpara>\r
+ </callout>\r
+ <callout arearefs="CO9-2">\r
+ <simpara>\r
+ The OpenSRF message identifier (<literal>"threadTrace":0</literal>) confirms that this\r
+ message is in response to the request matching the same identifier.\r
+ </simpara>\r
+ </callout>\r
+ <callout arearefs="CO9-3">\r
+ <simpara>\r
+ The message includes a payload JSON object (<literal>"payload":</literal>) with an OpenSRF\r
+ result for the request (<literal>"__c":"osrfResult"</literal>).\r
+ </simpara>\r
+ </callout>\r
+ <callout arearefs="CO9-4">\r
+ <simpara>\r
+ The result includes a status indicator string (<literal>"status":"OK"</literal>), the content\r
+ of the result response - in this case, a single string "raboof"\r
+ (<literal>"content":"raboof"</literal>) - and an integer status code for the request\r
+ (<literal>"statusCode":200</literal>).\r
+ </simpara>\r
+ </callout>\r
+ <callout arearefs="CO9-5">\r
+ <simpara>\r
+ The message also includes the message type (<literal>"type":"RESULT"</literal>) and the\r
+ message locale (<literal>"locale":"en-CA"</literal>).\r
+ </simpara>\r
+ </callout>\r
+ <callout arearefs="CO9-6">\r
+ <simpara>\r
+ The second message in the set of results from the response.\r
+ </simpara>\r
+ </callout>\r
+ <callout arearefs="CO9-7">\r
+ <simpara>\r
+ Again, the message identifier confirms that this message is in response to\r
+ a particular request.\r
+ </simpara>\r
+ </callout>\r
+ <callout arearefs="CO9-8">\r
+ <simpara>\r
+ The payload of the message denotes that this message is an\r
+ OpenSRF connection status message (<literal>"__c":"osrfConnectStatus"</literal>), with some\r
+ information about the particular OpenSRF connection that was used for this\r
+ request.\r
+ </simpara>\r
+ </callout>\r
+ <callout arearefs="CO9-9">\r
+ <simpara>\r
+ The response parameters for an OpenSRF connection status message include a\r
+ verbose status (<literal>"status":"Request Complete"</literal>) and an integer status code for\r
+ the connection status (`"statusCode":205).\r
+ </simpara>\r
+ </callout>\r
+ <callout arearefs="CO9-10">\r
+ <simpara>\r
+ The message also includes the message type (<literal>"type":"RESULT"</literal>) and the\r
+ message locale (<literal>"locale":"en-CA"</literal>).\r
+ </simpara>\r
+ </callout>\r
+ </calloutlist>\r
+ <tip><simpara>Before adding a new public OpenSRF service, ensure that it does\r
+ not introduce privilege escalation or unchecked access to data. For example,\r
+ the Evergreen <literal>open-ils.cstore</literal> private service is an object-relational mapper\r
+ that provides read and write access to the entire Evergreen database, so it\r
+ would be catastrophic to expose that service publicly. In comparison, the\r
+ Evergreen <literal>open-ils.pcrud</literal> public service offers the same functionality as\r
+ <literal>open-ils.cstore</literal> to any connected HTTP client or OpenSRF client, but the\r
+ additional authentication and authorization layer in <literal>open-ils.pcrud</literal> prevents\r
+ unchecked access to Evergreen’s data.</simpara></tip>\r
+ </simplesect>\r
+ <simplesect id="_stateless_and_stateful_connections">\r
+ <title>Stateless and stateful connections</title>\r
+ <simpara>OpenSRF supports both <emphasis>stateless</emphasis> and <emphasis>stateful</emphasis> connections. When an OpenSRF\r
+ client issues a <literal>REQUEST</literal> message in a <emphasis>stateless</emphasis> connection, the router\r
+ forwards the request to the next available service and the service returns the\r
+ result directly to the client.</simpara>\r
+ \r
+ <simpara>When an OpenSRF client issues a <literal>CONNECT</literal> message to create a <emphasis>stateful</emphasis> conection, the\r
+ router returns the Jabber ID of the next available service to the client so\r
+ that the client can issue one or more <literal>REQUEST</literal> message directly to that\r
+ particular service and the service will return corresponding <literal>RESULT</literal> messages\r
+ directly to the client. Until the client issues a <literal>DISCONNECT</literal> message, that\r
+ particular service is only available to the requesting client. Stateful connections\r
+ are useful for clients that need to make many requests from a particular service,\r
+ as it avoids the intermediary step of contacting the router for each request, as\r
+ well as for operations that require a controlled sequence of commands, such as a\r
+ set of database INSERT, UPDATE, and DELETE statements within a transaction.</simpara>\r
+ \r
+ </simplesect>\r
+ <simplesect id="_message_body_format">\r
+ <title>Message body format</title>\r
+ <simpara>OpenSRF was an early adopter of JavaScript Object Notation (JSON). While XMPP\r
+ is an XML protocol, the Evergreen developers recognized that the compactness of\r
+ the JSON format offered a significant reduction in bandwidth for the volume of\r
+ messages that would be generated in an application of that size. In addition,\r
+ the ability of languages such as JavaScript, Perl, and Python to generate\r
+ native objects with minimal parsing offered an attractive advantage over\r
+ invoking an XML parser for every message. Instead, the body of the XMPP message\r
+ is a simple JSON structure. For a simple request, like the following example\r
+ that simply reverses a string, it looks like a significant overhead: but we get\r
+ the advantages of locale support and tracing the request from the requester\r
+ through the listener and responder (drone).</simpara>\r
+ <formalpara><title>A request for opensrf.simple-text.reverse("foobar"):</title><para>\r
+<programlisting language="xml" linenumbering="unnumbered">\r
+<message from='router@private.localhost/opensrf.simple-text'\r
+ to='opensrf@private.localhost/opensrf.simple-text_listener_at_localhost_6275'\r
+ router_from='opensrf@private.localhost/_karmic_126678.3719_6288'\r
+ router_to='' router_class='' router_command='' osrf_xid=''\r
+>\r
+ <thread>1266781414.366573.12667814146288</thread>\r
+ <body>\r
+[\r
+ {"__c":"osrfMessage","__p":\r
+ {"threadTrace":"1","locale":"en-US","type":"REQUEST","payload":\r
+ {"__c":"osrfMethod","__p":\r
+ {"method":"opensrf.simple-text.reverse","params":["foobar"]}\r
+ }\r
+ }\r
+ }\r
+]\r
+ </body>\r
+</message>\r
+</programlisting>\r
+ </para></formalpara>\r
+ <formalpara><title>A response from opensrf.simple-text.reverse("foobar")</title><para>\r
+<programlisting language="xml" linenumbering="unnumbered">\r
+<message from='opensrf@private.localhost/opensrf.simple-text_drone_at_localhost_6285'\r
+ to='opensrf@private.localhost/_karmic_126678.3719_6288'\r
+ router_command='' router_class='' osrf_xid=''\r
+>\r
+ <thread>1266781414.366573.12667814146288</thread>\r
+ <body>\r
+[\r
+ {"__c":"osrfMessage","__p":\r
+ {"threadTrace":"1","payload":\r
+ {"__c":"osrfResult","__p":\r
+ {"status":"OK","content":"raboof","statusCode":200}\r
+ } ,"type":"RESULT","locale":"en-US"}\r
+ },\r
+ {"__c":"osrfMessage","__p":\r
+ {"threadTrace":"1","payload":\r
+ {"__c":"osrfConnectStatus","__p":\r
+ {"status":"Request Complete","statusCode":205}\r
+ },"type":"STATUS","locale":"en-US"}\r
+ }\r
+]\r
+ </body>\r
+</message>\r
+</programlisting>\r
+ </para></formalpara>\r
+ <simpara>The content of the <literal><body></literal> element of the OpenSRF request and result should\r
+ look familiar; they match the structure of the <link linkend="OpenSRFOverHTTP">OpenSRF over HTTP examples</link> that we previously dissected.</simpara>\r
+ </simplesect>\r
+ <simplesect id="_registering_opensrf_methods_in_depth">\r
+ <title>Registering OpenSRF methods in depth</title>\r
+ <simpara>Let’s explore the call to <literal>__PACKAGE__->register_method()</literal>; most of the members\r
+ of the hash are optional, and for the sake of brevity we omitted them in the\r
+ previous example. As we have seen in the results of the <link linkend="opensrfIntrospection">introspection call</link>, a\r
+ verbose registration method call is recommended to better enable the internal\r
+ documentation. Here is the complete set of members that you should pass to\r
+ <literal>__PACKAGE__->register_method()</literal>:</simpara>\r
+ <itemizedlist>\r
+ <listitem>\r
+ <simpara>\r
+ The <literal>method</literal> member specifies the name of the procedure in this module that is being registered as an OpenSRF method.\r
+ </simpara>\r
+ </listitem>\r
+ <listitem>\r
+ <simpara>\r
+ The <literal>api_name</literal> member specifies the invocable name of the OpenSRF method; by convention, the OpenSRF service name is used as the prefix.\r
+ </simpara>\r
+ </listitem>\r
+ <listitem>\r
+ <simpara>\r
+ The optional <literal>api_level</literal> member can be used for versioning the methods to allow the use of a deprecated API, but in practical use is always 1.\r
+ </simpara>\r
+ </listitem>\r
+ <listitem>\r
+ <simpara>\r
+ The optional <literal>argc</literal> member specifies the minimal number of arguments that the method expects.\r
+ </simpara>\r
+ </listitem>\r
+ <listitem>\r
+ <simpara>\r
+ The optional <literal>stream</literal> member, if set to any value, specifies that the method supports returning multiple values from a single call to \r
+ subsequent requests. OpenSRF automatically creates a corresponding method with ".atomic" appended to its name that returns the complete set of results in a \r
+ single request. Streaming methods are useful if you are returning hundreds of records and want to act on the results as they return.\r
+ </simpara>\r
+ </listitem>\r
+ <listitem>\r
+ <simpara>\r
+ The optional <literal>signature</literal> member is a hash that describes the method’s purpose, arguments, and return value.\r
+ </simpara>\r
+ <itemizedlist>\r
+ <listitem>\r
+ <simpara>\r
+ The <literal>desc</literal> member of the <literal>signature</literal> hash describes the method’s purpose.\r
+ </simpara>\r
+ </listitem>\r
+ <listitem>\r
+ <simpara>\r
+ The <literal>params</literal> member of the <literal>signature</literal> hash is an array of hashes in which each array element describes the corresponding method \r
+ argument in order.\r
+ </simpara>\r
+ <itemizedlist>\r
+ <listitem>\r
+ <simpara>\r
+ The <literal>name</literal> member of the argument hash specifies the name of the argument.\r
+ </simpara>\r
+ </listitem>\r
+ <listitem>\r
+ <simpara>\r
+ The <literal>desc</literal> member of the argument hash describes the argument’s purpose.\r
+ </simpara>\r
+ </listitem>\r
+ <listitem>\r
+ <simpara>\r
+ The <literal>type</literal> member of the argument hash specifies the data type of the argument: for example, string, integer, boolean, number, array, or hash.\r
+ </simpara>\r
+ </listitem>\r
+ </itemizedlist>\r
+ </listitem>\r
+ <listitem>\r
+ <simpara>\r
+ The <literal>return</literal> member of the <literal>signature</literal> hash is a hash that describes the return value of the method.\r
+ </simpara>\r
+ <itemizedlist>\r
+ <listitem>\r
+ <simpara>\r
+ The <literal>desc</literal> member of the <literal>return</literal> hash describes the return value.\r
+ </simpara>\r
+ </listitem>\r
+ <listitem>\r
+ <simpara>\r
+ The <literal>type</literal> member of the <literal>return</literal> hash specifies the data type of the return value: for example, string, integer, boolean, number, \r
+ array, or hash.\r
+ </simpara>\r
+ </listitem>\r
+ </itemizedlist>\r
+ </listitem>\r
+ </itemizedlist>\r
+ </listitem>\r
+ </itemizedlist>\r
+ </simplesect>\r
+ </section>\r
+ <section id="_evergreen_specific_opensrf_services">\r
+ <title>Evergreen-specific OpenSRF services</title>\r
+ <simpara>Evergreen is currently the primary showcase for the use of OpenSRF as an\r
+ application architecture. Evergreen 1.6.1 includes the following\r
+ set of OpenSRF services:</simpara>\r
+ <itemizedlist>\r
+ <listitem>\r
+ <simpara>\r
+ The <systemitem class="service">open-ils.actor</systemitem> service supports common tasks for working with user\r
+ accounts and libraries.\r
+ </simpara>\r
+ </listitem>\r
+ <listitem>\r
+ <simpara>\r
+ The <systemitem class="service">open-ils.auth</systemitem> service supports authentication of Evergreen users.\r
+ </simpara>\r
+ </listitem>\r
+ <listitem>\r
+ <simpara>\r
+ The <systemitem class="service">open-ils.booking</systemitem> service supports the management of reservations\r
+ for bookable items.\r
+ </simpara>\r
+ </listitem>\r
+ <listitem>\r
+ <simpara>\r
+ The <systemitem class="service">open-ils.cat</systemitem> service supports common cataloging tasks, such as\r
+ creating, modifying, and merging bibliographic and authority records.\r
+ </simpara>\r
+ </listitem>\r
+ <listitem>\r
+ <simpara>\r
+ The <systemitem class="service">open-ils.circ</systemitem> service supports circulation tasks such as checking\r
+ out items and calculating due dates.\r
+ </simpara>\r
+ </listitem>\r
+ <listitem>\r
+ <simpara>\r
+ The <systemitem class="service">open-ils.collections</systemitem> service supports tasks that assist collections\r
+ agencies in contacting users with outstanding fines above a certain\r
+ threshold.\r
+ </simpara>\r
+ </listitem>\r
+ <listitem>\r
+ <simpara>\r
+ The <systemitem class="service">open-ils.cstore</systemitem> private service supports unrestricted access to\r
+ Evergreen fieldmapper objects.\r
+ </simpara>\r
+ </listitem>\r
+ <listitem>\r
+ <simpara>\r
+ The <systemitem class="service">open-ils.ingest</systemitem> private service supports tasks for importing\r
+ data such as bibliographic and authority records.\r
+ </simpara>\r
+ </listitem>\r
+ <listitem>\r
+ <simpara>\r
+ The <systemitem class="service">open-ils.pcrud</systemitem> service supports permission-based access to Evergreen\r
+ fieldmapper objects.\r
+ </simpara>\r
+ </listitem>\r
+ <listitem>\r
+ <simpara>\r
+ The <systemitem class="service">open-ils.penalty</systemitem> penalty service supports the calculation of\r
+ penalties for users, such as being blocked from further borrowing, for\r
+ conditions such as having too many items checked out or too many unpaid\r
+ fines.\r
+ </simpara>\r
+ </listitem>\r
+ <listitem>\r
+ <simpara>\r
+ The <systemitem class="service">open-ils.reporter</systemitem> service supports the creation and scheduling of\r
+ reports.\r
+ </simpara>\r
+ </listitem>\r
+ <listitem>\r
+ <simpara>\r
+ The <systemitem class="service">open-ils.reporter-store</systemitem> private service supports access to Evergreen\r
+ fieldmapper objects for the reporting service.\r
+ </simpara>\r
+ </listitem>\r
+ <listitem>\r
+ <simpara>\r
+ The <systemitem class="service">open-ils.search</systemitem> service supports searching across bibliographic\r
+ records, authority records, serial records, Z39.50 sources, and ZIP codes.\r
+ </simpara>\r
+ </listitem>\r
+ <listitem>\r
+ <simpara>\r
+ The <systemitem class="service">open-ils.storage</systemitem> private service supports a deprecated method of\r
+ providing access to Evergreen fieldmapper objects. Implemented in Perl,\r
+ this service has largely been replaced by the much faster C-based\r
+ <literal>open-ils.cstore</literal> service.\r
+ </simpara>\r
+ </listitem>\r
+ <listitem>\r
+ <simpara>\r
+ The <systemitem class="service">open-ils.supercat</systemitem> service supports transforms of MARC records into\r
+ other formats, such as MODS, as well as providing Atom and RSS feeds and\r
+ SRU access.\r
+ </simpara>\r
+ </listitem>\r
+ <listitem>\r
+ <simpara>\r
+ The <systemitem class="service">open-ils.trigger</systemitem> private service supports event-based triggers for\r
+ actions such as overdue and holds available notification emails.\r
+ </simpara>\r
+ </listitem>\r
+ <listitem>\r
+ <simpara>\r
+ The <systemitem class="service">open-ils.vandelay</systemitem> service supports the import and export of batches of\r
+ bibliographic and authority records.\r
+ </simpara>\r
+ </listitem>\r
+ </itemizedlist>\r
+ <simpara>Of some interest is that the <systemitem class="service">open-ils.reporter-store</systemitem> and <systemitem class="service">open-ils.cstore</systemitem>\r
+ services have identical implementations. Surfacing them as separate services\r
+ enables a deployer of Evergreen to ensure that the reporting service does not\r
+ interfere with the performance-critical <systemitem class="service">open-ils.cstore</systemitem> service. One can also\r
+ direct the reporting service to a read-only database replica to, again, avoid\r
+ interference with <systemitem class="service">open-ils.cstore</systemitem> which must write to the master database.</simpara>\r
+ <simpara>There are only a few significant services that are not built on OpenSRF in\r
+ Evergreen 1.6.0, such as the SIP and Z39.50 servers. These services implement\r
+ different protocols and build on existing daemon architectures (Simple2ZOOM\r
+ for Z39.50), but still rely on the other OpenSRF services to provide access\r
+ to the Evergreen data. The non-OpenSRF services are reasonably self-contained\r
+ and can be deployed on different servers to deliver the same sort of deployment\r
+ flexibility as OpenSRF services, but have the disadvantage of not being\r
+ integrated into the same configuration and control infrastructure as the\r
+ OpenSRF services.</simpara>\r
+ </section>\r
+</chapter>\r
--- /dev/null
+<?xml version="1.0" encoding="utf-8"?>\r
+<chapter xml:id="Customizing_OPAC" xmlns="http://docbook.org/ns/docbook" version="5.0" xml:lang="EN"\r
+ xmlns:xi="http://www.w3.org/2001/XInclude" xmlns:xlink="http://www.w3.org/1999/xlink">\r
+ <info>\r
+ <title>Customizing the OPAC</title>\r
+ </info>\r
+ <para>While Evergreen is ready to go out of the box, libraries will want to customize Evergreen with their own color scheme, logos and layout. This chapter will explain how to \r
+ customize Evergreen to meet the needs of your users. For these task some knowledge of <systemitem>html</systemitem> and <systemitem>css</systemitem> is required. Many of these \r
+ instructions assume an installation of Evergreen using the default file locations.</para>\r
+ <note>\r
+ <para>Be sure to save a backup copy of all files you edit in a location other than <filename class="directory">/openils/var/web/opac/</filename> as files here could be \r
+ overwritten when you upgrade your copy of Evergreen.</para>\r
+ </note> \r
+ <section xml:id="ColorScheme">\r
+ <title>Change the Color Scheme</title>\r
+ <indexterm><primary>OPAC</primary><secondary>customizing</secondary><tertiary>changing the color scheme</tertiary></indexterm>\r
+ <para>To change the color scheme of the default Evergreen skin, edit <filename>/openils/var/web/opac/theme/default/css/colors.css</filename>. From this one file you can \r
+ change the 4 base color scheme as well as colors of specific elements. \r
+ </para> \r
+ <para>You can also create alternate themes for your users.</para> \r
+ <procedure>\r
+ <step>\r
+ <para>Copy the css folder and its contents from the example alternate theme <filename>/openils/var/web/opac/theme/reddish/</filename> \r
+ to a new folder <filename>/openils/var/web/opac/theme/<emphasis>[your new theme]</emphasis>/</filename>.</para>\r
+ </step> \r
+ <step>\r
+ <para>Edit <filename>/openils/var/web/opac/theme/<emphasis role="bold">[your new theme]</emphasis>/css/colors.css</filename> to use the colors you want.</para>\r
+ </step>\r
+ <step>\r
+ <para>Link to your new style sheet by adding the following to <filename>/openils/var/web/opac/skin/default/xml/common/css_common.xml</filename>.</para>\r
+<programlisting language="xml">\r
+<link type='text/css'\r
+rel="alternate stylesheet" \r
+title='&opac.style.yourtheme;' \r
+href="<!--#echo var='OILS_THEME_BASE'-->/yourtheme/css/colors.css" \r
+name='Default' csstype='color'/>\r
+</programlisting> \r
+ </step> \r
+ <step>\r
+ <para> Give your new theme a name users can select by adding the following to <filename>/openils/var/web/opac/locale/<emphasis role="bold">\r
+ [your locale]</emphasis>/opac.dtd</filename>.</para>\r
+ <screen><!ENTITY opac.style.yourtheme "YourTheme"></screen> \r
+ </step> \r
+ </procedure> \r
+ </section>\r
+ <section xml:id="customizing_opac_text">\r
+ <title>customizing Opac Text and Labels</title>\r
+ <indexterm><primary>OPAC</primary><secondary>customizing</secondary><tertiary>text and labels</tertiary></indexterm>\r
+ <para>To change text and links used throughout the OPAC, edit the following files:</para>\r
+ <itemizedlist>\r
+ <listitem><filename>/openils/var/web/opac/locale/<emphasis role="bold">[your locale]</emphasis>/lang.dtd</filename></listitem>\r
+ <listitem><filename>/openils/var/web/opac/locale/<emphasis role="bold">[your locale]</emphasis>/opac.dtd</filename></listitem>\r
+ </itemizedlist>\r
+ <tip>\r
+ <para>A better way to customize OPAC text is to create custom dtd files for your lang and opac customizations and then add a include \r
+ statement above the default dtd files.</para>\r
+ <programlisting>\r
+ <!DOCTYPE html PUBLIC \r
+ "-//W3C//DTD XHTML 1.0 Transitional//EN"\r
+ "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd" [\r
+ <!--#include virtual="/opac/locale/${locale}/custom_opac.dtd"-->\r
+ <!--#include virtual="/opac/locale/${locale}/opac.dtd"-->\r
+ ]>\r
+ </programlisting>\r
+ <para>position is important here. The first/top included dtd files will take precedence over the subsequent dtd includes.</para> \r
+ </tip>\r
+ <para>While it is possible to add text to the xml files itself, it is a good practice to use the DTD file to control the text and refer to the DTD elements in the xml/html code.</para> \r
+ <para>For example, the <filename>footer.xml</filename> file has this code to generate a copyright statement:</para>\r
+<programlisting>\r
+<div id='copyright_text'>\r
+<span>&footer.copyright;</span>\r
+</programlisting>\r
+ <para>The included <filename>opac.dtd</filename> file in the en-US locale directory has this setting for &footer.copyright text:</para> \r
+ <programlisting><!ENTITY footer.copyright "Copyright © 2006-2010 Georgia Public Library Service, and others"></programlisting>\r
+ </section>\r
+ <section xml:id="Logo_Images">\r
+ <title>Logo Images</title>\r
+ <para>To change the logos used by default to your own logos, replace the following files with images of your own, appropriately sized. </para>\r
+ <itemizedlist>\r
+ <listitem>Large main logo:<filename>/openils/var/web/opac/images/main_logo.jpg</filename></listitem>\r
+ <listitem>Small logo:<filename>/openils/var/web/opac/images/small_logo.jpg</filename></listitem>\r
+ </itemizedlist> \r
+ </section>\r
+ <section xml:id="AddedContent">\r
+ <title>Added Content</title>\r
+ <indexterm><primary>OPAC</primary><secondary>added content</secondary></indexterm>\r
+ <para>By default Evergreen includes customizable <quote>Added Content</quote> features to enhance the OPAC experience for your user. These features include Amazon book covers \r
+ and Google books searching. These features can be turned off or customized.</para>\r
+ <simplesect xml:id="bookcovers">\r
+ <title>Book Covers</title>\r
+ <para>The default install of Evergreen includes Amazon book covers. The settings for this are controlled by the <added_content> section of \r
+ <filename>/openils/conf/opensrf.xml</filename>. Here are the key elements of this configuration:</para> \r
+ <programlisting><module>OpenILS::WWW::AddedContent::Amazon</module></programlisting>\r
+ <para>This calls the Amazon perl module. If you wish to link to a different book cover service other than Amazon, you must create a new perl module and refer to it here. \r
+ You will also need to change other settings accordingly. There are some available book cover perl modules available in \r
+ <ulink url="http://svn.open-ils.org/trac/ILS/browser/trunk/Open-ILS/src/perlmods/OpenILS/WWW/AddedContent">trunk</ulink></para>\r
+ <programlisting><base_url>http://images.amazon.com/images/P/</base_url></programlisting>\r
+ <para>Base URL for Amazon added content fetching. This URL may need to be shortened when new (read: non-image) content fetching \r
+ capabilities are added.</para> \r
+ <programlisting><timeout>1</timeout></programlisting>\r
+ <para>Max number of seconds to wait for an added content request to return data. Data not returned within the timeout is considered a failure.</para> \r
+ <programlisting><retry_timeout>600</retry_timeout></programlisting>\r
+ <para>After added content lookups have been disabled due to too many lookup failures, this is the amount of time to wait before we try again.</para>\r
+ <programlisting><max_errors>15</max_errors></programlisting>\r
+ <para>Maximum number of consecutive lookup errors a given process can live before added content lookups are disabled for everyone.</para> \r
+ <programlisting><userid>MY_USER_ID</userid></programlisting>\r
+ <para>If a userid is required to access the added content.</para>\r
+ </simplesect>\r
+ <simplesect xml:id="googlebookslink">\r
+ <title>Google Books Link</title>\r
+ <indexterm><primary>OPAC</primary><secondary>added content</secondary><tertiary>Google Books</tertiary></indexterm>\r
+ <para>The results page will display a <emphasis role="bold">Browse in Google Books Search</emphasis> link for items in the results page which have corresponding entries\r
+ in <systemitem class="resource">Google Books</systemitem>. \r
+ This will link to Google Books content including table of contents and complete versions of the work if it exists in Google Books. Items not in Google Books will not \r
+ display a link. This feature can be turned off by changing the googleBooksLink variable setting to <emphasis>false</emphasis> in the file \r
+ <filename>/openils/var/web/opac/skin/default/js/result_common.js</filename>. By default, this feature is activated. </para> \r
+ </simplesect> \r
+ </section>\r
+ <section xml:id="resultspage">\r
+ <title>Customizing the Results Page</title>\r
+ <para>The results page is extremely customizable and allows some built in features to be activated with some simple edits or more advanced customizations can be done by more \r
+ experienced web developers.</para>\r
+ <para>There are several critical files to edit if you wish to customize the results page:</para>\r
+ <itemizedlist>\r
+ <listitem><filename>/openils/var/web/opac/skin/default/js/result_common.js</filename> - This file controls the JavaScript for the top level elements on the results \r
+ page and should only be edited by experienced web developers except for the <link linkend='googlebookslink'>Google books link setting</link> mentioned previously.</listitem>\r
+ <listitem><filename>/openils/var/web/opac/skin/default/js/rresult.js</filename> - Has some good controls of results page settings at the top of this file but \r
+ requires web development skills for editing this file.</listitem>\r
+ <listitem><filename>/openils/var/web/opac/skin/default/xml/result/rresult_table.xml</filename> - This controls the layout of the items table on results page.</listitem>\r
+ </itemizedlist> \r
+ </section>\r
+ <section xml:id="deatailspage">\r
+ <title>Customizing the Details Page</title>\r
+ <indexterm><primary>OPAC</primary><secondary>customizing</secondary><tertiary>details page</tertiary></indexterm>\r
+ <para>There are many options when customizing the details page in Evergreen. The default settings are effective for most libraries, but it is important to understand the full potential \r
+ of Evergreen when displaying the details of items.</para> \r
+ <para>Some quick features can be turned on and off by changing variable values in the file <filename>/openils/var/web/opac/skin/default/js/rdedail.js</filename>. \r
+ You will notice the section at the top of this file called <quote>Per-skin configuration settings</quote>. Changing setting in this section can control several features including \r
+ limiting results to local only or showing copy location or displaying serial holdings. Form this section you can also enable refworks and set the Refworks host URL.</para>\r
+ <para>Some copy level details settings can be turned on and off from <filename>/openils/var/web/opac/skin/default/js/copy_details.js</filename> including displaying certain fields \r
+ such as due date in the OPAC.</para>\r
+ <para>An important file is the <filename>/openils/var/web/opac/skin/default/xml/rdetail/rdetail_summary.xml</filename> file. This file allows you to control which field to display in \r
+ the details summary of the record. The new <link linkend='BibTemplate'>BibTemplate</link> feature makes this file even more powerful by allowing you to display any marc fields \r
+ with a variety of formatting options.</para>\r
+ <para>The <filename>/openils/var/web/opac/skin/default/xml/rdetail/rdetail_copyinfo.xml</filename> file allows you to format the display of the copy information.</para> \r
+ </section> \r
+ <section xml:id="BibTemplate">\r
+ <title>BibTemplate</title><indexterm><primary>BibTemplate</primary></indexterm>\r
+ <para>BibTemplate is an Evergreen-custom Dojo module which can be used to retrieve and format XML data served by the Evergreen <systemitem class="protocol">unAPI</systemitem> service. <systemitem class="protocol">unAPI</systemitem> is a protocol for requesting known objects in specific formats, and Evergreen uses this to supply data – bibliographic records, \r
+ metarecords, monograph holdings information, Located URIs, and more to come – in many different formats from MARCXML to MODS to custom XML applications.</para>\r
+ <indexterm><primary>unAPI</primary></indexterm> <indexterm><primary>MARCXML</primary></indexterm><indexterm><primary>MODS</primary></indexterm>\r
+ <para>Managing the display of information from raw XML can be difficult, and the purpose of BibTemplate is to make this simpler, as well as move the display closer to the \r
+ client and away from the source data. This is good from a separation-of-responsibilities perspective, and also makes it easier to contain and control local customization.</para>\r
+ <para>BibTemplate supports the following Evergreen meta data formats:</para>\r
+ <itemizedlist>\r
+ <listitem>MARCXML - datatype='marcxml-full' (default)</listitem> \r
+ <listitem>MODS 3.3: datatype='mods33'</listitem>\r
+ <listitem>Dublin Core: datatype='rdf_dc'</listitem><indexterm><primary>Dublin Core</primary></indexterm>\r
+ <listitem>FGDC: datatype='fgdc'</listitem><indexterm><primary>FGDC</primary></indexterm>\r
+ </itemizedlist> \r
+ <simplesect>\r
+ <title>HTML API</title>\r
+ <para>BibTemplate follows the Dojo convention of adding attributes to existing (X)HTML in order to progressively change its behavior. The 1.6.0 HTML API consists of a \r
+ set of attributes that are added to existing OPAC markup, and fall into two classes:</para><indexterm><primary>Dojo toolkit</primary></indexterm>\r
+ <itemizedlist>\r
+ <listitem> The slot marker – Elements that denote the location of bibliographic data to insert.</listitem>\r
+ <listitem>The slot formatter – Elements that specify how the named data should be formatted for display.</listitem>\r
+ </itemizedlist> \r
+ </simplesect>\r
+ <simplesect>\r
+ <title>Slot Marker</title><indexterm><primary>slot marker</primary></indexterm>\r
+ <para>A slot marker is any displayable HTML element that has a type attribute with a value starting with opac/slot-data. This element will become the container \r
+ for the formatted data. A slot marker is required in order to retrieve, format and display data using BibTemplate. A slot marker must also have an \r
+ attribute called query containing a CSS3 selector. This selector is applied to the XML returned by the unAPI service in order to gather the specific XML \r
+ Nodes that should be considered for formatting.</para><indexterm><primary>CSS3</primary></indexterm>\r
+ <para>The slot marker can also specify the format of the data to be returned from the unAPI service. This can be specified by adding +{format} to the type \r
+ attribute, as in opac/slot-data+mods33-full. The default data format is marcxml-uri, which is an augmented MARCXML record containing Located URI information \r
+ and unAPI links.</para>\r
+ <para>Example of a slot marker:</para>\r
+ <screen><p type='opac/slot-data' query='datafield[tag=245]'></p></screen>\r
+ <para>Most useful attribute match operators include:</para>\r
+ <itemizedlist>\r
+ <listitem> datafield[tag=245] - exact match</listitem>\r
+ <listitem>datafield[tag^=65] - match start of value</listitem>\r
+ </itemizedlist> \r
+ <tip><para>Selectors always narrow, so select broadly and iterate through the NodeList</para></tip>\r
+ </simplesect>\r
+ <simplesect>\r
+ <title>Slot Formatter</title><indexterm><primary>slot formatter</primary></indexterm>\r
+ <para>A slot formatter is any invisible HTML element which has a type attribute with the value of opac/slot-format. (NOTE: before 1.6.0.4, only <script> \r
+ elements were supported, though this restriction is now removed to support Internet Explorer.) Only one slot formatter element is allowed in each slot. The text contents \r
+ of this element are wrapped in a JavaScript<indexterm><primary>JavaScript</primary></indexterm> function and run for each node returned by the query CSS3 selector \r
+ specified on the slot marker. This function is passed \r
+ one argument, called item, which an XML Node captured by the selector. This function should return HTML text. The output for all runs of the slot formatter is \r
+ concatenated into a single string and used to replace the contents of the slot marker.</para>\r
+ <para>The slot formatter is optional, and if not supplied BibTemplate will create a simple function which extracts and returns the text content of the XML Nodes \r
+ specified in the CSS3 selector.</para>\r
+ <para>Example of a slot formatter:</para>\r
+<programlisting>\r
+ <td class='rdetail_item' id='rdetail_online' type='opac/slot-data' \r
+ query='volumes volume uris uri' join=", ">\r
+ <script type='opac/slot-format'><![CDATA[\r
+ var link = '<a href="' + item.getAttribute('href') + '">' + item.getAttribute('label') + '</a>';\r
+ if (item.getAttribute('use_restriction'))\r
+ link += ' (Use restriction: ' + item.getAttribute('use_restriction') + ')';\r
+ return link;\r
+ ]]></script>\r
+ </td>\r
+</programlisting>\r
+ </simplesect>\r
+ <simplesect>\r
+ <title>JavaScript API</title><indexterm><primary>JavaScript</primary></indexterm>\r
+ <para>In order for BibTemplate to find the slot markers and invoke the slot formatters JavaScript renderer must be instantiated and called. This must be done \r
+ for each record that is to contribute to a pages display. The API for this is simple and straight-forward:</para>\r
+ <para>The slot formatter is optional, and if not supplied BibTemplate will create a simple function which extracts and returns the text content of the XML Nodes \r
+ specified in the CSS3 selector.</para><indexterm><primary>CSS3</primary></indexterm>\r
+ <para>Example of a slot formatter:</para>\r
+<programlisting>\r
+ dojo.require('openils.BibTemplate'); // Tell Dojo to load BibTemplate, if it is not already loaded\r
+ \r
+ // Create a renderer supplying the record id and the short name of the org unit, if known, \r
+ // and call the render() method \r
+ new openils.BibTemplate({ record : new CGI().param('r'), org_unit : here.shortname() }).render(); \r
+</programlisting>\r
+ <para>The argument hash supplied to the new <systemitem>openils.BibTemplate()</systemitem> constructor can have the following properties:</para>\r
+ <itemizedlist>\r
+ <listitem>record – The bibliographic record ID.</listitem>\r
+ <listitem>org_unit – The relevant Organizational Unit, used to restrict holdings scope as on a search result or record detail page.</listitem>\r
+ <listitem>root – The root element within the web page that BibTemplate should search for slot markers</listitem>\r
+ </itemizedlist> \r
+ </simplesect>\r
+ <simplesect>\r
+ <title>BibTemplate Examples</title>\r
+ <para>This is all that we had to add to display the contents of an arbitrary MARC field:</para>\r
+<programlisting>\r
+<tr>\r
+ <td>Bibliography note</td>\r
+ <td type='opac/slot-data' query='datafield[tag=504]'></td>\r
+</tr>\r
+</programlisting>\r
+ <note><para>If multiple fields match, they are displayed on consecutive lines within the same left-hand cell.</para></note>\r
+ <para>To display a specific MARC subfield, add that subfield to the query attribute.</para><indexterm><primary>MARC</primary></indexterm>\r
+ <para>For example, subfield $a is the only user-oriented subfield in field 586 (Awards Note)</para>\r
+<programlisting>\r
+<tr>\r
+ <td>Awards note</td>\r
+ <td type='opac/slot-data' query='datafield[tag=586] subfield[code=a]'></td>\r
+</tr>\r
+</programlisting>\r
+ <para>Hide empty rows by default, and display them only if they have content:</para>\r
+<programlisting>\r
+ <tr class='hide_me' id='tag504'>\r
+ <td>Bibliographic note</td>\r
+ <td type='opac/slot-data' query='datafield[tag=504]'>\r
+ <script type='opac/slot-format'><![CDATA[\r
+ dojo.query('#tag504').removeClass('hide_me');\r
+ return '<span>' + dojox.data.dom.textContent(item) +\r
+ '</span><br/>';\r
+ ]]></script>\r
+ </td></tr>\r
+</programlisting>\r
+ <itemizedlist>\r
+ <listitem><![CDATA[ ... ]]> tells Evergreen Web server to treat the contents as literal <quote>character data</quote> - \r
+ avoids hilarity of entity substitution</listitem>\r
+ <listitem><script type='opac/slot-format'>...</script>, contained within an <quote>opac/slot-data</quote> element, receives a variable named item \r
+ containing the results of the query (a NodeList)</listitem> \r
+ </itemizedlist> \r
+ <para>Suppressing a subfield:</para>\r
+<programlisting>\r
+<tr class='hide_me' id='tag700'>\r
+ <td>Additional authors</td>\r
+ <td type='opac/slot-data' query='datafield[tag=700]'>\r
+ <script type='opac/slot-format'><![CDATA[\r
+ dojo.query('#tag700').removeClass('hide_me');\r
+ var text = '';\r
+ var list = dojo.query('subfield:not([code=4])', item);\r
+ for (var i =0; i < list.length; i++) {\r
+ text += dojox.data.dom.textContent(list[i]) + ' ';\r
+ }\r
+ return '<span>' + text + '</span><br/>';\r
+ ]]></script>\r
+ </td></tr>\r
+</programlisting>\r
+ </simplesect>\r
+ </section>\r
+ <section xml:id="customizingslimpac">\r
+ <title>Customizing the Slimpac</title>\r
+ <para>The Slimpac is the an alternative OPAC display for browsers or devices without JavaScript or which may have screen size limitations. There is both a simple and advanced search \r
+ option for the Slimpac.</para> \r
+ <para>The html files for customizing the Slimpac search display are located in the folder <filename class="directory">/openils/var/web/opac/extras/slimpac</filename>.</para>\r
+ <para><filename>start.html</filename> is the basic search display and <filename>advanced.html</filename> is the display for the advanced search option.</para> \r
+ <para>By default, the Slimpac files include the same locale dtd as the regular OPAC (<filename>opac.dtd</filename>). However, the slimpac files do not use the same CSS files as the \r
+ regular OPAC which means that if you change the OPAC color scheme, you must also edit the Slimpac files.</para>\r
+ <simplesect>\r
+ <title>Customizing the Slimpac Results Display</title>\r
+ <para>Two files control the display results for the slimpac. Edit the XSL stylesheet (<filename>/openils/var/xsl/ATOM2XHTML.xsl</filename>) to edit the elements of the \r
+ record as pulled from the XML output. \r
+ You may also change the style of the page by editing the CSS stylesheet for the results display (<filename>/openils/var/web/opac/extras/os.css</filename>).</para> \r
+ </simplesect>\r
+ <simplesect>\r
+ <title>Customizing the Slimpac Details/Holdings Display</title> \r
+ <para>It is also possible to customize the details page when viewing specific items from the results list. To edit the holdings display which contains the details of the specific \r
+ record linked from the results display, edit the CSS stylesheet for the holdings/details page \r
+ (<filename>/openils/var/web/opac/extras/htmlcard.css</filename>). You may also control the content of the record by editing <filename>MARC21slim2HTMLCard.xsl</filename>. \r
+ Holdings data may also be controlled by editing <filename>MARC21slim2HTMLCard-holdings.xsl</filename>.</para> \r
+ </simplesect>\r
+ </section>\r
+ <section xml:id="integratingsearchforms">\r
+ <title>Integrating a Evergreen Search Form on a Web Page</title>\r
+ <para>It is possible to embed a simple search form into an html page which will allow users to search for materials in your Evergreen catalog. Here is code which can be embedded \r
+ anywhere in the body of your web page:</para> \r
+<programlisting language="html"><![CDATA[\r
+<form action="http://[domain name]/opac/[locale]/skin/default/xml/rresult.xml" method="get">\r
+<div>\r
+ Quick Catalog Search:<br />\r
+ <input type="text" alt="Input Box for Catalog Search" maxlength="250" \r
+ size="20" id="t" name="t" value="" />\r
+ <input type="hidden" id="rt" name="rt" value="keyword" />\r
+ <input type="hidden" id="tp" name="tp" value="keyword" />\r
+ <input type="hidden" id="l" name="l" value="2" />\r
+ <input type="hidden" id="d" name="d" value="" />\r
+ <input type="hidden" id="f" name="f" value="" />\r
+ <input type="submit" value="Search" class="form-submit" />\r
+ </div>\r
+</form>\r
+]]>\r
+</programlisting>\r
+ <para> Replace <emphasis>[domain name]</emphasis> with the domain name of your Evergreen server and replace <emphasis>[locale]</emphasis> with the desired locale of \r
+ your Evergreen instance (e.g. en-US). This does a basic keyword search. Different types of searches and more advanced search forms can be developed. For further information on the url parameters used by Evergreen, see <xref linkend="search_url" /> for more details.</para> \r
+ </section>\r
+</chapter>\r
+\r
--- /dev/null
+<?xml version='1.0' encoding='UTF-8'?>\r
+<chapter xmlns="http://docbook.org/ns/docbook" xmlns:xi="http://www.w3.org/2001/XInclude"\r
+ xmlns:xl="http://www.w3.org/1999/xlink" version="5.0" xml:id="customizingstaffclient">\r
+ <title>Customizing the Staff Client</title>\r
+ <indexterm><primary>staff client</primary><secondary>customizing</secondary></indexterm>\r
+ <para>This chapter will give you some guidance on customizing the staff client.</para> \r
+ <para>The files related to the staff client are located in the directory <filename class="directory">/openils/var/web/xul/[staff client version]/server/</filename></para> \r
+ <section xml:id="staffclient_Changingcolors">\r
+ <title>Changing Colors and Images</title>\r
+ <para>To change or adjust the image on the main screen edit <filename>/openils/var/web/xul/index.xhtml</filename>. By default, the image on this page is \r
+ <filename>main_logo.jpg</filename> which is the same main logo used in the OPAC.</para>\r
+ <para>To adjust colors on various staff client pages edit the corresponding cascading style sheets located in \r
+ <filename class="directory">/openils/var/web/xul/[staff client version]/server/skin/</filename>. Other display aspects can also be adjusted using these cascading style sheets.</para> \r
+ </section>\r
+ <section xml:id="staffclient_Changinglabels">\r
+ <title>Changing Labels and Messages</title>\r
+ <indexterm><primary>staff client</primary><secondary>customizing</secondary><tertiary>labels and messages</tertiary></indexterm>\r
+ <para>You can customize labels in the staff client by editing the corresponding DTD files. The staff client uses the same <filename>lang.dtd</filename> used by the OPAC. This file is located in <filename class="directory">/openils/var/web/opac/locale/[<emphasis>your locale</emphasis>]</filename>. Other labels are controlled by the staff client specific <filename>lang.dtd</filename> file in <filename class="directory">/openils/var/web/xul/<emphasis>client version</emphasis>]/server/locale/[<emphasis>your locale</emphasis>]/</filename>.</para>\r
+ </section>\r
+ <section xml:id="staffclient_searchskin">\r
+ <title>Changing the Search Skin</title>\r
+ <para>There are a few ways to change the custom skin for OPAC searching in staff client.</para> \r
+ <simplesect>\r
+ <title>Changing the Search Skin on Server - Overriding Local Settings</title> \r
+ <para>To change the opac search skins used by the staff client create a file named <filename>custom.js</filename> and place it in the \r
+ <filename class="directory">/openils/var/web/xul/[staff client version]/server/skin/</filename> directory. This will effect all staff clients since these settings will \r
+ override local settings.</para> \r
+ <para>For example, the following text in <filename>custom.js</filename> would set the staff client opac, details page, results page and browse function to the craftsman \r
+ skin:</para> \r
+<programlisting language="JavaScript">\r
+urls['opac'] = '/opac/' + LOCALE + '/skin/craftsman/xml/advanced.xml?nps=1';\r
+urls['opac_rdetail'] = '/opac/' + LOCALE + '/skin/craftsman/xml/rdetail.xml';\r
+urls['opac_rresult'] = '/opac/' + LOCALE + '/skin/craftsman/xml/rresult.xml';\r
+urls['browser'] = '/opac/' + LOCALE + '/skin/craftsman/xml/advanced.xml?nps=1';\r
+</programlisting>\r
+ <para>Restart the staff client to see the changes.</para> \r
+ </simplesect>\r
+ <simplesect>\r
+ <title>Changing the Search Skin on an individual Machine</title> \r
+ <para>To change the search skin on an individual machine for personal preferences or needs, edit the file \r
+ <filename>/[Evergreen staff client path]/build/chrome/content/main/constants.js</filename>.</para> \r
+ <para>Find the lines which point to the urls for the OPAC and edit accordingly. For example, here is an example to set the opac, details page, results page and browse \r
+ function to the craftsman skin:</para> \r
+<programlisting language="JavaScript">\r
+ 'opac' : '/opac/' + LOCALE + '/skin/craftsman/xml/advanced.xml?nps=1',\r
+'opac_rdetail' : '/opac/' + LOCALE + '/skin/craftsman/xml/rdetail.xml',\r
+'opac_rresult' : '/opac/' + LOCALE + '/skin/craftsman/xml/rresult.xml',\r
+...\r
+'browser' : '/opac/' + LOCALE + '/skin/craftsman/xml/advanced.xml?nps=1',\r
+</programlisting>\r
+ <para>After editing this file, save it and restart the staff client for the changes to take effect.</para> \r
+ </simplesect>\r
+ </section>\r
+</chapter>\r
--- /dev/null
+<?xml version="1.0" encoding="UTF-8"?>\r
+<chapter xml:id="data_models_and_access" xmlns="http://docbook.org/ns/docbook" version="5.0" xml:lang="EN"\r
+ xmlns:xi="http://www.w3.org/2001/XInclude" xmlns:xlink="http://www.w3.org/1999/xlink">\r
+ <chapterinfo>\r
+ <title>Evergreen Data Models and Access</title>\r
+ </chapterinfo>\r
+ <abstract id="DM_abstract">\r
+ <simpara>This chapter was taken from Dan Scott's <emphasis>Developer Workshop</emphasis>, February 2010.</simpara>\r
+ </abstract>\r
+ <section id="exploring_database_schema">\r
+ <title>Exploring the Database Schema</title>\r
+ <simpara>The database schema is tied pretty tightly to PostgreSQL. Although PostgreSQL<indexterm><primary>databases</primary><secondary>PostgreSQL</secondary></indexterm>\r
+ adheres closely to ANSI SQL standards, the use of schemas, SQL functions<indexterm><primary>ANSI</primary></indexterm>\r
+ implemented in both <systemitem>plpgsql</systemitem> and <systemitem>plperl</systemitem>, and PostgreSQL’s native full-text\r
+ search would make it… challenging… to port to other database platforms.</simpara>\r
+ <simpara>A few common PostgreSQL interfaces for poking around the schema and\r
+ manipulating data are:</simpara>\r
+ <itemizedlist>\r
+ <listitem>\r
+ <simpara>\r
+ psql (the command line client)<indexterm><primary>databases</primary><secondary>PostgreSQL</secondary><tertiery>psql</tertiery></indexterm>\r
+ </simpara>\r
+ </listitem>\r
+ <listitem>\r
+ <simpara>\r
+ pgadminIII (a GUI client).<indexterm><primary>databases</primary><secondary>PostgreSQL</secondary><tertiery>pgadminIII</tertiery></indexterm>\r
+ </simpara>\r
+ </listitem>\r
+ </itemizedlist>\r
+ <simpara>Or you can read through the source files in <filename class="directoy">Open-ILS/src/sql/Pg</filename>.</simpara>\r
+ <simpara>Let’s take a quick tour through the schemas, pointing out some highlights\r
+ and some key interdependencies:</simpara>\r
+ <itemizedlist>\r
+ <listitem>\r
+ <simpara>\r
+ actor.org_unit → asset.copy_location\r
+ </simpara>\r
+ </listitem>\r
+ <listitem>\r
+ <simpara>\r
+ actor.usr → actor.card\r
+ </simpara>\r
+ </listitem>\r
+ <listitem>\r
+ <simpara>\r
+ biblio.record_entry → asset.call_number → asset.copy\r
+ </simpara>\r
+ </listitem>\r
+ <listitem>\r
+ <simpara>\r
+ config.metabib_field → metabib.*_field_entry\r
+ </simpara>\r
+ </listitem>\r
+ </itemizedlist>\r
+ This documentation also contains an Appendix for the Evergreen <xref linkend="databaseschema"/>. \r
+ </section>\r
+ <section id="_database_access_methods">\r
+ <title>Database access methods</title>\r
+ <simpara>You could use direct access to the database via Perl DBI, JDBC, etc,\r
+ but Evergreen offers several database CRUD services for\r
+ creating / retrieving / updating / deleting data. These avoid tying\r
+ you too tightly to the current database schema and they funnel database\r
+ access through the same mechanism, rather than tying up connections\r
+ with other interfaces.</simpara>\r
+ </section>\r
+ <section id="_evergreen_interface_definition_language_idl">\r
+ <title>Evergreen Interface Definition Language (IDL)</title>\r
+ <indexterm><primary>Evergreen Interface Definition Language (IDL)</primary></indexterm>\r
+ <simpara>Defines properties and required permissions for Evergreen classes.\r
+ To reduce network overhead, a given object is identified via a\r
+ class-hint and serialized as a JSON array of properties (no named properties).</simpara>\r
+ <simpara>As of 1.6, fields will be serialized in the order in which they appear\r
+ in the IDL definition file, and the is_new / is_changed / is_deleted\r
+ properties are automatically added. This has greatly reduced the size of\r
+ the <literal>fm_IDL.xml</literal> file and makes DRY people happier :)</simpara>\r
+ <itemizedlist>\r
+ <listitem>\r
+ <simpara>\r
+ … oils_persist:readonly tells us, if true, that the data lives in the database, but is pulled from the SELECT statement defined in the <oils_persist:source_definition> \r
+ child element\r
+ </simpara>\r
+ </listitem>\r
+ </itemizedlist>\r
+ <simplesect id="_idl_basic_example_config_language_map">\r
+ <title>IDL basic example (config.language_map)</title>\r
+<programlisting language="xml" linenumbering="unnumbered">\r
+<class id="clm" controller="open-ils.cstore open-ils.pcrud"\r
+ oils_obj:fieldmapper="config::language_map"\r
+ oils_persist:tablename="config.language_map"\r
+ reporter:label="Language Map" oils_persist:field_safe="true"> <co id="dmCO5-1"/> <co id="dmCO5-2"/> <co id="dmCO5-3"/> <co id="dmCO5-4"/>\r
+ <fields oils_persist:primary="code" oils_persist:sequence=""> <co id="dmCO5-5"/>\r
+ <field reporter:label="Language Code" name="code"\r
+ reporter:selector="value" reporter:datatype="text"/> <co id="dmCO5-6"/>\r
+ <field reporter:label="Language" name="value"\r
+ reporter:datatype="text" oils_persist:i18n="true"/> <co id="dmCO5-7"/>\r
+ </fields>\r
+ <links/>\r
+ <permacrud xmlns="http://open-ils.org/spec/opensrf/IDL/permacrud/v1"> <co id="dmCO5-8"/>\r
+ <actions>\r
+ <create global_required="true" permission="CREATE_MARC_CODE"> <co id="dmCO5-9"/>\r
+ <retrieve global_required="true"\r
+ permission="CREATE_MARC_CODE UPDATE_MARC_CODE DELETE_MARC_CODE">\r
+ <update global_required="true" permission="UPDATE_MARC_CODE">\r
+ <delete global_required="true" permission="DELETE_MARC_CODE">\r
+ </actions>\r
+ </permacrud>\r
+</class>\r
+</programlisting>\r
+ <calloutlist>\r
+ <callout arearefs="dmCO5-1">\r
+ <simpara>\r
+ The <literal>class</literal> element defines the attributes and permissions for classes,\r
+ and relationships between classes.\r
+ </simpara>\r
+ <indexterm><primary>Evergreen Interface Definition Language (IDL)</primary><secondary>class element</secondary></indexterm>\r
+ <itemizedlist>\r
+ <listitem>\r
+ <simpara>\r
+ The <literal>id</literal> attribute on the <literal>class</literal> element defines the class hint that is\r
+ used everywhere in Evergreen.\r
+ </simpara>\r
+ </listitem>\r
+ <listitem>\r
+ <simpara>\r
+ The <literal>controller</literal> attribute defines the OpenSRF\r
+ services that provide access to the data for the class objects.\r
+ </simpara>\r
+ </listitem>\r
+ </itemizedlist>\r
+ </callout>\r
+ <callout arearefs="dmCO5-2">\r
+ <simpara>\r
+ The <literal>oils_obj::fieldmapper</literal> attribute defines the name of the class that\r
+ is generated by <literal>OpenILS::Utils::Fieldmapper</literal>.\r
+ </simpara>\r
+ </callout>\r
+ <callout arearefs="dmCO5-3">\r
+ <simpara>\r
+ The <literal>oils_persist:tablename</literal> attribute defines the name of the table\r
+ that contains the data for the class objects.\r
+ </simpara>\r
+ </callout>\r
+ <callout arearefs="dmCO5-4">\r
+ <simpara>\r
+ The reporter interface uses <literal>reporter:label</literal> attribute values in\r
+ the source list to provide meaningful class and attribute names. The\r
+ <literal>open-ils.fielder</literal> service generates a set of methods that provide direct\r
+ access to the classes for which <literal>oils_persist:field_safe</literal> is <literal>true</literal>. For\r
+ example,\r
+ </simpara>\r
+<screen>\r
+<userinput>\r
+srfsh# request open-ils.fielder open-ils.fielder.clm.atomic \\r
+{"query":{"code":{"=":"eng"}}}\r
+\r
+Received Data: [\r
+ {\r
+ "value":"English",\r
+ "code":"eng"\r
+ }\r
+]\r
+</userinput>\r
+</screen>\r
+ </callout>\r
+ <callout arearefs="dmCO5-5">\r
+ <simpara>\r
+ The <literal>fields</literal> element defines the list of fields for the class.\r
+ </simpara>\r
+ <indexterm><primary>Evergreen Interface Definition Language (IDL)</primary><secondary>fields element</secondary></indexterm>\r
+ <itemizedlist>\r
+ <listitem>\r
+ <simpara>\r
+ The <literal>oils_persist:primary</literal> attribute defines the column that acts as\r
+ the primary key for the table.\r
+ </simpara>\r
+ </listitem>\r
+ <listitem>\r
+ <simpara>\r
+ The <literal>oils_persist:sequence</literal> attribute holds the name of the database\r
+ sequence.\r
+ </simpara>\r
+ </listitem>\r
+ </itemizedlist>\r
+ </callout>\r
+ <callout arearefs="dmCO5-6">\r
+ <simpara>\r
+ Each <literal>field</literal> element defines one property of the class.\r
+ </simpara>\r
+ <indexterm><primary>Evergreen Interface Definition Language (IDL)</primary><secondary>field element</secondary></indexterm>\r
+ <itemizedlist>\r
+ <listitem>\r
+ <simpara>\r
+ The <literal>name</literal> attribute defines the getter/setter method name for the field.\r
+ </simpara>\r
+ </listitem>\r
+ <listitem>\r
+ <simpara>\r
+ The <literal>reporter:label</literal> attribute defines the attribute name as used in\r
+ the reporter interface.\r
+ </simpara>\r
+ </listitem>\r
+ <listitem>\r
+ <simpara>\r
+ The <literal>reporter:selector</literal> attribute defines the field used in the reporter\r
+ filter interface to provide a selectable list. This gives the user a more\r
+ meaningful access point than the raw numeric ID or abstract code.\r
+ </simpara>\r
+ </listitem>\r
+ <listitem>\r
+ <simpara>\r
+ The <literal>reporter:datatype</literal> attribute defines the type of data held by\r
+ this property for the purposes of the reporter.\r
+ </simpara>\r
+ </listitem>\r
+ </itemizedlist>\r
+ </callout>\r
+ <callout arearefs="dmCO5-7">\r
+ <simpara>\r
+ The <literal>oils_persist:i18n</literal> attribute, when <literal>true</literal>, means that\r
+ translated values for the field’s contents may be accessible in\r
+ different locales.\r
+ </simpara>\r
+ </callout>\r
+ <callout arearefs="dmCO5-8">\r
+ <simpara>\r
+ <indexterm><primary>Evergreen Interface Definition Language (IDL)</primary><secondary>permacrud element</secondary></indexterm>\r
+ The <literal>permacrud</literal> element defines the permissions (if any) required\r
+ to <emphasis role="strong">c</emphasis>reate, <emphasis role="strong">r</emphasis>etrieve, <emphasis role="strong">u</emphasis>pdate, \r
+ and <emphasis role="strong">d</emphasis>elete data for this\r
+ class. <literal>open-ils.permacrud</literal> must be defined as a controller for the class\r
+ for the permissions to be applied.\r
+ </simpara>\r
+ \r
+ </callout>\r
+ <callout arearefs="dmCO5-9">\r
+ <simpara>\r
+ Each action requires one or more <literal>permission</literal> values that the\r
+ user must possess to perform the action.\r
+ </simpara>\r
+ <itemizedlist>\r
+ <listitem>\r
+ <simpara>\r
+ If the <literal>global_required</literal> attribute is <literal>true</literal>, then the user must\r
+ have been granted that permission globally (depth = 0) to perform\r
+ the action.\r
+ </simpara>\r
+ </listitem>\r
+ <listitem>\r
+ <simpara>\r
+ The <literal>context_field</literal> attribute denotes the <literal><field></literal> that identifies\r
+ the org_unit at which the user must have the pertinent permission.\r
+ </simpara>\r
+ </listitem>\r
+ <listitem>\r
+ <indexterm><primary>Evergreen Interface Definition Language (IDL)</primary><secondary>action element</secondary></indexterm>\r
+ <simpara>\r
+ An action element may contain a <literal><context_field></literal> element that\r
+ defines the linked class (identified by the <literal>link</literal> attribute) and\r
+ the field in the linked class that identifies the org_unit where\r
+ the permission must be held.\r
+ </simpara>\r
+ <itemizedlist>\r
+ <listitem>\r
+ <indexterm><primary>Evergreen Interface Definition Language (IDL)</primary><secondary>context_field element</secondary></indexterm>\r
+ <simpara>\r
+ If the <literal><context_field></literal> element contains a <literal>jump</literal> attribute,\r
+ then it defines a link to a link to a class with a field identifying\r
+ the org_unit where the permission must be held.\r
+ </simpara>\r
+ </listitem>\r
+ </itemizedlist>\r
+ </listitem>\r
+ </itemizedlist>\r
+ </callout>\r
+ </calloutlist>\r
+ </simplesect>\r
+ <simplesect id="_reporter_data_types_and_their_possible_values">\r
+ <title>Reporter data types and their possible values</title>\r
+ <itemizedlist>\r
+ <listitem>\r
+ <simpara>\r
+ <literal>bool</literal>: Boolean <literal>true</literal> or <literal>false</literal>\r
+ </simpara>\r
+ </listitem>\r
+ <listitem>\r
+ <simpara>\r
+ <literal>id</literal>: ID of the row in the database\r
+ </simpara>\r
+ </listitem>\r
+ <listitem>\r
+ <simpara>\r
+ <literal>int</literal>: integer value\r
+ </simpara>\r
+ </listitem>\r
+ <listitem>\r
+ <simpara>\r
+ <literal>interval</literal>: PostgreSQL time interval\r
+ </simpara>\r
+ </listitem>\r
+ <listitem>\r
+ <simpara>\r
+ <literal>link</literal>: link to another class, as defined in the <literal><links></literal>\r
+ element of the class definition\r
+ </simpara>\r
+ </listitem>\r
+ <listitem>\r
+ <simpara>\r
+ <literal>money</literal>: currency amount\r
+ </simpara>\r
+ </listitem>\r
+ <listitem>\r
+ <simpara>\r
+ <literal>org_unit</literal>: list of org_units\r
+ </simpara>\r
+ </listitem>\r
+ <listitem>\r
+ <simpara>\r
+ <literal>text</literal>: text value\r
+ </simpara>\r
+ </listitem>\r
+ <listitem>\r
+ <simpara>\r
+ <literal>timestamp</literal>: PostgreSQL timestamp\r
+ </simpara>\r
+ </listitem>\r
+ </itemizedlist>\r
+ </simplesect>\r
+ <simplesect id="_idl_example_with_linked_fields_actor_workstation">\r
+ <title>IDL example with linked fields (actor.workstation)</title>\r
+ <simpara>Just as tables often include columns with foreign keys that point\r
+ to values stored in the column of a different table, IDL classes\r
+ can contain fields that link to fields in other classes. The <literal><links></literal>\r
+ element defines which fields link to fields in other classes, and\r
+ the nature of the relationship:</simpara>\r
+<programlisting language="xml" linenumbering="unnumbered">\r
+<class id="aws" controller="open-ils.cstore"\r
+ oils_obj:fieldmapper="actor::workstation"\r
+ oils_persist:tablename="actor.workstation"\r
+ reporter:label="Workstation">\r
+ <fields oils_persist:primary="id"\r
+ oils_persist:sequence="actor.workstation_id_seq">\r
+ <field reporter:label="Workstation ID" name="id"\r
+ reporter:datatype="id"/>\r
+ <field reporter:label="Workstation Name" name="name"\r
+ reporter:datatype="text"/>\r
+ <field reporter:label="Owning Library" name="owning_lib"\r
+ reporter:datatype="org_unit"/>\r
+ <field reporter:label="Circulations" name="circulations"\r
+ oils_persist:virtual="true" reporter:datatype="link"/> <co id="dmCO6-1"/>\r
+ </fields>\r
+ <links> <co id="dmCO6-2"/>\r
+ <link field="owning_lib" reltype="has_a" key="id"\r
+ map="" class="aou"/> <co id="dmCO6-3"/>\r
+ <link field="circulations" reltype="has_many" key="workstation"\r
+ map="" class="circ"/>\r
+ <link field="circulation_checkins" reltype="has_many"\r
+ key="checkin_workstation" map="" class="circ"/>\r
+ </links>\r
+</class>\r
+</programlisting>\r
+ <calloutlist>\r
+ <callout arearefs="dmCO6-1">\r
+ <simpara>\r
+ This field includes an <literal>oils_persist:virtual</literal> attribute with the value of\r
+ <literal>true</literal>, meaning that the linked class <literal>circ</literal> is a virtual class.\r
+ </simpara>\r
+ </callout>\r
+ <callout arearefs="dmCO6-2">\r
+ <simpara>\r
+ The <literal><links></literal> element contains 0 or more <literal><link></literal> elements.\r
+ </simpara>\r
+ </callout>\r
+ <callout arearefs="dmCO6-3">\r
+ <simpara>\r
+ Each <literal><link></literal> element defines the field (<literal>field</literal>) that links to a different\r
+ class (<literal>class</literal>), the relationship (<literal>rel_type</literal>) between this field and the target\r
+ field (<literal>key</literal>). If the field in this class links to a virtual class, the (<literal>map</literal>)\r
+ attribute defines the field in the target class that returns a list of matching\r
+ objects for each object in this class.\r
+ </simpara>\r
+ </callout>\r
+ </calloutlist>\r
+ </simplesect>\r
+ </section>\r
+ <section id="open_ils_cstore_literal_data_access_interfaces">\r
+ <title><literal>open-ils.cstore</literal> data access interfaces</title>\r
+ <indexterm><primary>cstore</primary></indexterm>\r
+ <simpara>For each class documented in the IDL, the <literal>open-ils.cstore</literal> service\r
+ automatically generates a set of data access methods, based on the\r
+ <literal>oils_persist:tablename</literal> class attribute.</simpara>\r
+ <simpara>For example, for the class hint <literal>clm</literal>, cstore generates the following\r
+ methods with the <literal>config.language_map</literal> qualifer:</simpara>\r
+ <itemizedlist>\r
+ <listitem>\r
+ <simpara>\r
+ <literal>open-ils.cstore.direct.config.language_map.id_list {"code" { "like": "e%" } }</literal>\r
+ </simpara>\r
+ <simpara>Retrieves a list composed only of the IDs that match the query.</simpara>\r
+ </listitem>\r
+ <listitem>\r
+ <simpara>\r
+ <literal>open-ils.cstore.direct.config.language_map.retrieve "eng"</literal>\r
+ </simpara>\r
+ <simpara>Retrieves the object that matches a specific ID.</simpara>\r
+ </listitem>\r
+ <listitem>\r
+ <simpara>\r
+ <literal>open-ils.cstore.direct.config.language_map.search {"code" : "eng"}</literal>\r
+ </simpara>\r
+ <simpara>Retrieves a list of objects that match the query.</simpara>\r
+ </listitem>\r
+ <listitem>\r
+ <simpara>\r
+ <literal>open-ils.cstore.direct.config.language_map.create <_object_></literal>\r
+ </simpara>\r
+ <simpara>Creates a new object from the passed in object.</simpara>\r
+ </listitem>\r
+ <listitem>\r
+ <simpara>\r
+ <literal>open-ils.cstore.direct.config.language_map.update <_object_></literal>\r
+ </simpara>\r
+ <simpara>Updates the object that has been passed in.</simpara>\r
+ </listitem>\r
+ <listitem>\r
+ <simpara>\r
+ <literal>open-ils.cstore.direct.config.language_map.delete "eng"</literal>\r
+ </simpara>\r
+ <simpara>Deletes the object that matches the query.</simpara>\r
+ </listitem>\r
+ </itemizedlist>\r
+ </section>\r
+ <section id="_open_ils_pcrud_data_access_interfaces">\r
+ <title>open-ils.pcrud data access interfaces</title>\r
+ <indexterm><primary>pcrud</primary></indexterm>\r
+ <simpara>For each class documented in the IDL, the <literal>open-ils.pcrud</literal> service\r
+ automatically generates a set of data access methods, based on the\r
+ <literal>oils_persist:tablename</literal> class attribute.</simpara>\r
+ <simpara>For example, for the class hint <literal>clm</literal>, <literal>open-ils.pcrud</literal> generates the following\r
+ methods that parallel the <literal>open-ils.cstore</literal> interface:</simpara>\r
+ <itemizedlist>\r
+ <listitem>\r
+ <simpara>\r
+ <literal>open-ils.pcrud.id_list.clm <_authtoken_>, { "code": { "like": "e%" } }</literal>\r
+ </simpara>\r
+ </listitem>\r
+ <listitem>\r
+ <simpara>\r
+ <literal>open-ils.pcrud.retrieve.clm <_authtoken_>, "eng"</literal>\r
+ </simpara>\r
+ </listitem>\r
+ <listitem>\r
+ <simpara>\r
+ <literal>open-ils.pcrud.search.clm <_authtoken_>, { "code": "eng" }</literal>\r
+ </simpara>\r
+ </listitem>\r
+ <listitem>\r
+ <simpara>\r
+ <literal>open-ils.pcrud.create.clm <_authtoken_>, <_object_></literal>\r
+ </simpara>\r
+ </listitem>\r
+ <listitem>\r
+ <simpara>\r
+ <literal>open-ils.pcrud.update.clm <_authtoken_>, <_object_></literal>\r
+ </simpara>\r
+ </listitem>\r
+ <listitem>\r
+ <simpara>\r
+ <literal>open-ils.pcrud.delete.clm <_authtoken_>, "eng"</literal>\r
+ </simpara>\r
+ </listitem>\r
+ </itemizedlist>\r
+ </section>\r
+ <section id="_transaction_and_savepoint_control">\r
+ <title>Transaction and savepoint control</title>\r
+ <simpara>Both <literal>open-ils.cstore</literal> and <literal>open-ils.pcrud</literal> enable you to control database transactions\r
+ to ensure that a set of operations either all succeed, or all fail,\r
+ atomically:</simpara>\r
+ <itemizedlist>\r
+ <listitem>\r
+ <simpara>\r
+ <literal>open-ils.cstore.transaction.begin</literal>\r
+ </simpara>\r
+ </listitem>\r
+ <listitem>\r
+ <simpara>\r
+ <literal>open-ils.cstore.transaction.commit</literal>\r
+ </simpara>\r
+ </listitem>\r
+ <listitem>\r
+ <simpara>\r
+ <literal>open-ils.cstore.transaction.rollback</literal>\r
+ </simpara>\r
+ </listitem>\r
+ <listitem>\r
+ <simpara>\r
+ <literal>open-ils.pcrud.transaction.begin</literal>\r
+ </simpara>\r
+ </listitem>\r
+ <listitem>\r
+ <simpara>\r
+ <literal>open-ils.pcrud.transaction.commit</literal>\r
+ </simpara>\r
+ </listitem>\r
+ <listitem>\r
+ <simpara>\r
+ <literal>open-ils.pcrud.transaction.rollback</literal>\r
+ </simpara>\r
+ </listitem>\r
+ </itemizedlist>\r
+ <simpara>At a more granular level, <literal>open-ils.cstore</literal> and <literal>open-ils.pcrud</literal> enable you to set database\r
+ savepoints to ensure that a set of operations either all succeed, or all\r
+ fail, atomically, within a given transaction:</simpara>\r
+ <itemizedlist>\r
+ <listitem>\r
+ <simpara>\r
+ <literal>open-ils.cstore.savepoint.begin</literal>\r
+ </simpara>\r
+ </listitem>\r
+ <listitem>\r
+ <simpara>\r
+ <literal>open-ils.cstore.savepoint.commit</literal>\r
+ </simpara>\r
+ </listitem>\r
+ <listitem>\r
+ <simpara>\r
+ <literal>open-ils.cstore.savepoint.rollback</literal>\r
+ </simpara>\r
+ </listitem>\r
+ <listitem>\r
+ <simpara>\r
+ <literal>open-ils.pcrud.savepoint.begin</literal>\r
+ </simpara>\r
+ </listitem>\r
+ <listitem>\r
+ <simpara>\r
+ <literal>open-ils.pcrud.savepoint.commit</literal>\r
+ </simpara>\r
+ </listitem>\r
+ <listitem>\r
+ <simpara>\r
+ <literal>open-ils.pcrud.savepoint.rollback</literal>\r
+ </simpara>\r
+ </listitem>\r
+ </itemizedlist>\r
+ <simpara>Transactions and savepoints must be performed within a stateful\r
+ connection to the <literal>open-ils.cstore</literal> and <literal>open-ils.pcrud</literal> services.\r
+ In <literal>srfsh</literal>, you can open a stateful connection using the <literal>open</literal>\r
+ command, and then close the stateful connection using the <literal>close</literal>\r
+ command - for example:</simpara>\r
+ <screen>srfsh# open open-ils.cstore\r
+ ... perform various transaction-related work\r
+ srfsh# close open-ils.cstore</screen>\r
+ <simplesect id="_json_queries">\r
+ <title>JSON Queries</title>\r
+ <indexterm><primary>JSON</primary></indexterm>\r
+ <simpara>Beyond simply retrieving objects by their ID using the <literal>\*.retrieve</literal>\r
+ methods, you can issue queries against the <literal>\*.delete</literal> and <literal>\*.search</literal>\r
+ methods using JSON to filter results with simple or complex search\r
+ conditions.</simpara>\r
+ <simpara>For example, to generate a list of barcodes that are held in a\r
+ copy location that allows holds and is visible in the OPAC:</simpara>\r
+<programlisting language="sh" linenumbering="unnumbered">\r
+srfsh# request open-ils.cstore open-ils.cstore.json_query <co id="dmCO7-1"/>\r
+ {"select": {"acp":["barcode"], "acpl":["name"]}, <co id="dmCO7-2"/>\r
+ "from": {"acp":"acpl"}, <co id="dmCO7-3"/>\r
+ "where": [ <co id="dmCO7-4"/>\r
+ {"+acpl": "holdable"}, <co id="dmCO7-5"/>\r
+ {"+acpl": "opac_visible"} <co id="dmCO7-6"/>\r
+ ]}\r
+\r
+Received Data: {\r
+ "barcode":"BARCODE1",\r
+ "name":"Stacks"\r
+}\r
+\r
+Received Data: {\r
+ "barcode":"BARCODE2",\r
+ "name":"Stacks"\r
+}\r
+</programlisting>\r
+ <calloutlist>\r
+ <callout arearefs="dmCO7-1">\r
+ <simpara>\r
+ Invoke the <literal>json_query</literal> service.\r
+ </simpara>\r
+ </callout>\r
+ <callout arearefs="dmCO7-2">\r
+ <simpara>\r
+ Select the <literal>barcode</literal> field from the <literal>acp</literal> class and the <literal>name</literal>\r
+ field from the <literal>acpl</literal> class.\r
+ </simpara>\r
+ </callout>\r
+ <callout arearefs="dmCO7-3">\r
+ <simpara>\r
+ Join the <literal>acp</literal> class to the <literal>acpl</literal> class based on the linked field\r
+ defined in the IDL.\r
+ </simpara>\r
+ </callout>\r
+ <callout arearefs="dmCO7-4">\r
+ <simpara>\r
+ Add a <literal>where</literal> clause to filter the results. We have more than one\r
+ condition beginning with the same key, so we wrap the conditions inside\r
+ an array.\r
+ </simpara>\r
+ </callout>\r
+ <callout arearefs="dmCO7-5">\r
+ <simpara>\r
+ The first condition tests whether the boolean value of the <literal>holdable</literal>\r
+ field on the <literal>acpl</literal> class is true.\r
+ </simpara>\r
+ </callout>\r
+ <callout arearefs="dmCO7-6">\r
+ <simpara>\r
+ The second condition tests whether the boolean value of the\r
+ <literal>opac_visible</literal> field on the <literal>acpl</literal> class is true.\r
+ </simpara>\r
+ </callout>\r
+ </calloutlist>\r
+ <simpara>For thorough coverage of the breadth of support offered by JSON\r
+ query syntax, see <ulink url="http://open-ils.org/dokuwiki/doku.php?id=documentation:technical:jsontutorial">JSON Queries: A Tutorial</ulink>.</simpara>\r
+ </simplesect>\r
+ <simplesect id="_fleshing_linked_objects">\r
+ <title>Fleshing linked objects</title>\r
+ <simpara>A simplistic approach to retrieving a set of objects that are linked to\r
+ an object that you are retrieving - for example, a set of call numbers\r
+ linked to the barcodes that a given user has borrowed - would be to:\r
+ 1. Retrieve the list of circulation objects (<literal>circ</literal> class)\r
+ for a given user (<literal>usr</literal> class).\r
+ 2. For each circulation object, look up the target copy (<literal>target_copy</literal>\r
+ field, linked to the <literal>acp</literal> class).\r
+ 3. For each copy, look up the call number for that copy (<literal>call_number</literal>\r
+ field, linked to the <literal>acn</literal> class).</simpara>\r
+ <simpara>However, this would result in potentially hundreds of round-trip\r
+ queries from the client to the server. Even with low-latency connections,\r
+ the network overhead would be considerable. So, built into the <literal>open-ils.cstore</literal> and\r
+ <literal>open-ils.pcrud</literal> access methods is the ability to <emphasis>flesh</emphasis> linked fields -\r
+ that is, rather than return an identifier to a given linked field,\r
+ the method can return the entire object as part of the initial response.</simpara>\r
+ <simpara>Most of the interfaces that return class instances from the IDL offer the\r
+ ability to flesh returned fields. For example, the\r
+ <literal>open-ils.cstore.direct.\*.retrieve</literal> methods allow you to specify a\r
+ JSON structure defining the fields you wish to flesh in the returned object.</simpara>\r
+ <formalpara><title>Fleshing fields in objects returned by <literal>open-ils.cstore</literal></title><para>\r
+<programlisting language="sh" linenumbering="unnumbered">\r
+srfsh# request open-ils.cstore open-ils.cstore.direct.asset.copy.retrieve 1, \\r
+ {\r
+ "flesh": 1, <co id="dmCO8-1"/>\r
+ "flesh_fields": { <co id="dmCO8-2"/>\r
+ "acp": ["location"]\r
+ }\r
+ }\r
+</programlisting>\r
+ </para></formalpara>\r
+ <calloutlist>\r
+ <callout arearefs="dmCO8-1">\r
+ <simpara>\r
+ The <literal>flesh</literal> argument is the depth at which objects should be fleshed.\r
+ For example, to flesh out a field that links to another object that includes\r
+ a field that links to another object, you would specify a depth of 2.\r
+ </simpara>\r
+ </callout>\r
+ <callout arearefs="dmCO8-2">\r
+ <simpara>\r
+ The <literal>flesh_fields</literal> argument contains a list of objects with the fields\r
+ to flesh for each object.\r
+ </simpara>\r
+ </callout>\r
+ </calloutlist>\r
+ <simpara>Let’s flesh things a little deeper. In addition to the copy location,\r
+ let’s also flesh the call number attached to the copy, and then flesh\r
+ the bibliographic record attached to the call number.</simpara>\r
+ <formalpara><title>Fleshing fields in fields of objects returned by <literal>open-ils.cstore</literal></title><para>\r
+<programlisting language="java" linenumbering="unnumbered">\r
+request open-ils.cstore open-ils.cstore.direct.asset.copy.retrieve 1, \\r
+ {\r
+ "flesh": 2,\r
+ "flesh_fields": {\r
+ "acp": ["location", "call_number"],\r
+ "acn": ["record"]\r
+ }\r
+ }\r
+</programlisting>\r
+ </para></formalpara>\r
+ </simplesect>\r
+ </section>\r
+ <section id="_adding_an_idl_entry_for_resolverresolver">\r
+ <title>Adding an IDL entry for ResolverResolver</title>\r
+ <simpara>Most OpenSRF methods in Evergreen define their object interface in the\r
+ IDL. Without an entry in the IDL, the prospective caller of a given\r
+ method is forced to either call the method and inspect the returned\r
+ contents, or read the source to work out the structure of the JSON\r
+ payload. At this stage of the tutorial, we have not defined an entry\r
+ in the IDL to represent the object returned by the\r
+ <literal>open-ils.resolver.resolve_holdings</literal> method. It is time to complete\r
+ that task.</simpara>\r
+ <simpara>The <literal>open-ils.resolver</literal> service is unlike many of the other classes\r
+ defined in the IDL because its data is not stored in the Evergreen\r
+ database. Instead, the data is requested from an external Web service\r
+ and only temporarily cached in <literal>memcached</literal>. Fortunately, the IDL\r
+ enables us to represent this kind of class by setting the\r
+ <literal>oils_persist:virtual</literal> class attribute to <literal>true</literal>.</simpara>\r
+ <simpara>So, let’s add an entry to the IDL for the <literal>open-ils.resolver.resolve_holdings</literal>\r
+ service:</simpara>\r
+ <programlisting language="xml" linenumbering="unnumbered"></programlisting>\r
+ <simpara>And let’s make <literal>ResolverResolver.pm</literal> return an array composed of our new\r
+ <literal>rhr</literal> classes rather than raw JSON objects:</simpara>\r
+ <programlisting language="perl" linenumbering="unnumbered"></programlisting>\r
+ <simpara>Once we add the new entry to the IDL and copy the revised <literal>ResolverResolver.pm</literal>\r
+ Perl module to <literal>/openils/lib/perl5/OpenILS/Application/</literal>, we need to:</simpara>\r
+ <orderedlist numeration="arabic">\r
+ <listitem>\r
+ <simpara>\r
+ Copy the updated IDL to both the <literal>/openils/conf/</literal> and\r
+ <literal>/openils/var/web/reports/</literal> directories. The Dojo approach to\r
+ parsing the IDL uses the IDL stored in the reports directory.\r
+ </simpara>\r
+ </listitem>\r
+ <listitem>\r
+ <simpara>\r
+ Restart the Perl services to make the new IDL visible to the services\r
+ and refresh the <literal>open-ils.resolver</literal> implementation\r
+ </simpara>\r
+ </listitem>\r
+ <listitem>\r
+ <simpara>\r
+ Rerun <filename>/openils/bin/autogen.sh</filename> to regenerate the JavaScript versions<indexterm><primary>autogen</primary></indexterm>\r
+ of the IDL required by the HTTP translator and gateway.\r
+ </simpara>\r
+ </listitem>\r
+ </orderedlist>\r
+ <simpara>We also need to adjust our JavaScript client to use the nifty new<indexterm><primary>JavaScript</primary></indexterm>\r
+ objects that <literal>open-ils.resolver.resolve_holdings</literal> now returns.\r
+ The best approach is to use the support in Evergreen’s Dojo extensions<indexterm><primary>Dojo toolkit</primary></indexterm>\r
+ to generate the JavaScript classes directly from the IDL XML file.</simpara>\r
+ <formalpara><title>Accessing classes defined in the IDL via Fieldmapper</title><para>\r
+ <programlisting language="html" linenumbering="unnumbered"></programlisting>\r
+ </para></formalpara>\r
+ <calloutlist>\r
+ <callout arearefs="">\r
+ <simpara>\r
+ Load the Dojo core.\r
+ </simpara>\r
+ </callout>\r
+ <callout arearefs="">\r
+ <simpara>\r
+ <literal>fieldmapper.AutoIDL</literal> reads <filename>/openils/var/reports/fm_IDL.xml</filename> to\r
+ generate a list of class properties.\r
+ </simpara>\r
+ </callout>\r
+ <callout arearefs="">\r
+ <simpara>\r
+ <literal>fieldmapper.dojoData</literal> seems to provide a store for Evergreen data\r
+ accessed via Dojo.\r
+ </simpara>\r
+ </callout>\r
+ <callout arearefs="">\r
+ <simpara>\r
+ <literal>fieldmapper.Fieldmapper</literal> converts the list of class properties into\r
+ actual classes.\r
+ </simpara>\r
+ </callout>\r
+ <callout arearefs="">\r
+ <simpara>\r
+ <literal>fieldmapper.standardRequest</literal> invokes an OpenSRF method and returns\r
+ an array of objects.\r
+ </simpara>\r
+ </callout>\r
+ <callout arearefs="">\r
+ <simpara>\r
+ The first argument to <literal>fieldmapper.standardRequest</literal> is an array\r
+ containing the OpenSRF service name and method name.\r
+ </simpara>\r
+ </callout>\r
+ <callout arearefs="">\r
+ <simpara>\r
+ The second argument to <literal>fieldmapper.standardRequest</literal> is an array\r
+ containing the arguments to pass to the OpenSRF method.\r
+ </simpara>\r
+ </callout>\r
+ <callout arearefs="">\r
+ <simpara>\r
+ As Fieldmapper has instantiated the returned objects based on their\r
+ class hints, we can invoke getter/setter methods on the objects.\r
+ </simpara>\r
+ </callout>\r
+ </calloutlist>\r
+ </section>\r
+ \r
+</chapter>\r
--- /dev/null
+<?xml version="1.0" encoding="utf-8"?>\r
+<chapter xml:id="filesandvdirectories" xmlns="http://docbook.org/ns/docbook" version="5.0" xml:lang="EN"\r
+ xmlns:xi="http://www.w3.org/2001/XInclude" xmlns:xlink="http://www.w3.org/1999/xlink">\r
+ <info>\r
+ <title>Evergreen File Structure and Configuration Files</title>\r
+ <abstract><para>This section will describe the basic file structure and cover key configuration files. Understanding the directory and file structure of Evergreen will allow you \r
+ to be able to customize your Evergreen software and take full advantage of many features.</para></abstract>\r
+ </info>\r
+ \r
+ <section xml:id="EvergreenDirectoryStructure"> \r
+ <title>Evergreen Directory Structure</title>\r
+ <para>This is the top level directory structure of Evergreen located in the default installation directory <filename class="directory">/openils</filename>:</para>\r
+ <table>\r
+ <title xml:id="EvergreenDirectoryStructureTable">Evergreen Directory Structure</title>\r
+ <tgroup align="left" cols="2" colsep="1" rowsep="1">\r
+ <colspec colnum="1" colwidth="1.0*"/>\r
+ <colspec colnum="2" colwidth="3.0*"/>\r
+ <thead>\r
+ <row>\r
+ <entry>Directory</entry>\r
+ <entry>Description</entry>\r
+ </row>\r
+ </thead>\r
+ <tbody>\r
+ <row>\r
+ <entry><filename class="directory">bin</filename></entry>\r
+ <entry>Contains many critical Perl and shell scripts such as <filename class="directory">autogen.sh</filename> and \r
+ <filename class="directory">oils.ctl</filename>. </entry>\r
+ </row>\r
+ <row>\r
+ <entry><filename class="directory">conf</filename></entry>\r
+ <entry>Contains the configuration scripts including the two most important base configuration files <filename>opensrf_core.xml</filename> and <filename>opensrf.xml</filename>.</entry>\r
+ </row>\r
+ <row>\r
+ <entry><filename class="directory">include</filename></entry>\r
+ <entry>Contains the header files used by the scripts written in C.</entry>\r
+ </row>\r
+ <row>\r
+ <entry><filename class="directory">lib</filename></entry>\r
+ <entry>Contains the core code of Evergreen including the C code and perl modules. In particular, the perl modules in the \r
+ subdirectory<filename class="directory">perl5/OpenILS</filename> \r
+ are of particular interest to developers. </entry>\r
+ </row>\r
+ <row>\r
+ <entry><filename class="directory">var</filename></entry>\r
+ <entry>Largest directory and includes the web directories (<filename class="directory">web</filename>), lock pid fies \r
+ (<filename class="directory">run</filename>), circ setting files (<filename class="directory">circ</filename>) templates \r
+ (<filename class="directory">templates</filename>) and log (<filename class="directory">templates</filename> and \r
+ <filename class="directory">data</filename>) files.</entry>\r
+ </row>\r
+ </tbody>\r
+ </tgroup>\r
+ </table>\r
+ <section xml:id="EvergreenFiles"> \r
+ <title>Evergreen Configuration Files</title>\r
+ \r
+ <table>\r
+ <title xml:id="EvergreenConfigurationFilesTable">Key Evergreen Configuration Files</title>\r
+ <tgroup align="left" cols="2" colsep="1" rowsep="1">\r
+ <colspec colnum="1" colwidth="1.0*"/>\r
+ <colspec colnum="2" colwidth="3.0*"/>\r
+ <thead>\r
+ <row>\r
+ <entry>File</entry>\r
+ <entry>Description</entry>\r
+ </row>\r
+ </thead>\r
+ <tbody>\r
+ <row>\r
+ <entry><filename>/openils/conf/opensrf_core.xml</filename><indexterm><primary>configuration files</primary>\r
+ <secondary>opensrf_core.xml</secondary></indexterm></entry>\r
+ \r
+ <entry>Files which controls which Evergreen services are run on the public and private routers. For a service to run, it must be registered in this file. This file also controls the loglevel and points to the log file for the services. An Evergreen restart is required for changes to take effect.</entry>\r
+ </row>\r
+ <row>\r
+ <entry><filename>/openils/conf/opensrf.xml</filename><indexterm><primary>configuration files</primary><secondary>opensrf.xml</secondary></indexterm></entry>\r
+ <entry>Use this file to set directory locations, the default locale, default notice settings and settings for all Evergreen services. \r
+ It is critical for any administrator to understand the settings in this file. An Evergreen restart is required for changes to take effect.</entry>\r
+ </row>\r
+ <row>\r
+ <entry><filename>/openils/conf/fm_IDL.xml</filename> <indexterm><primary>configuration files</primary><secondary>fm_IDL.xml</secondary></indexterm></entry>\r
+ \r
+ <entry>Used for linking the OpenSRF/Evergreen services to the Evergreen database tables. An Evergreen restart is required for changes to take \r
+ effect. Running autogen.sh is also required.</entry>\r
+ </row>\r
+ <row>\r
+ <entry><filename>/etc/apache2/eg_vhost.conf</filename><indexterm><primary>configuration files</primary><secondary>Apache</secondary></indexterm></entry>\r
+ <entry>Controls the Evergreen virtual site. Allows to configure the skin for the OPAC or configure various directories within the Apache web server. \r
+ An Apache restart is required for changes to this file to take effect.</entry>\r
+ </row>\r
+ </tbody>\r
+ </tgroup>\r
+ </table>\r
+ <table>\r
+ <title xml:id="EvergreenScriptsTable">Useful Evergreen Scripts</title>\r
+ <tgroup align="left" cols="2" colsep="1" rowsep="1">\r
+ <colspec colnum="1" colwidth="1.0*"/>\r
+ <colspec colnum="2" colwidth="2.0*"/>\r
+ <thead>\r
+ <row>\r
+ <entry>File</entry>\r
+ <entry>Description</entry>\r
+ </row>\r
+ </thead>\r
+ <tbody>\r
+ <row>\r
+ <entry><filename>/openils/bin/autogen.sh</filename><indexterm><primary>autogen</primary></indexterm></entry>\r
+ <entry>Used to update changes to org units and the fm_IDL.xml file. Will generate web and staff client pages based on contents of files and \r
+ Evergreen database entries.</entry>\r
+ </row>\r
+ <row>\r
+ <entry><filename>/openils/bin/clark-kent.pl</filename><indexterm><primary>reports</primary><secondary>starting</secondary></indexterm></entry>\r
+ \r
+ <entry>Perl script for starting the reporter.</entry>\r
+ </row>\r
+ <row>\r
+ <entry><filename>/openils/bin/action_trigger_runner.pl</filename><indexterm><primary>action triggers</primary>\r
+ <secondary>runner</secondary></indexterm></entry>\r
+ <entry>Perl script used to trigger the actions set up in the action trigger tool in the staff client.</entry>\r
+ </row>\r
+ <row>\r
+ <entry><filename>/openils/bin/osrf_ctl.sh</filename></entry>\r
+ <entry>The start up script for OpenSRF and Evergreen. </entry>\r
+ </row>\r
+ <row>\r
+ <entry><filename>/openils/bin/reshelving_ complete.srfsh</filename></entry>\r
+ <entry>Change status from <quote>reshelving</quote> to <quote>available</quote> for items which have been in reshelving for a certain amount of time.</entry>\r
+ </row>\r
+ <row>\r
+ <entry><filename>/openils/bin/srfsh</filename><indexterm><primary>srfsh</primary></indexterm></entry>\r
+ \r
+ <entry>Used to start the OpenSRF shell.</entry>\r
+ </row>\r
+ \r
+ </tbody>\r
+ </tgroup>\r
+ </table>\r
+ </section>\r
+ </section>\r
+</chapter>\r
--- /dev/null
+<?xml version='1.0' encoding='UTF-8'?>\r
+<appendix xmlns="http://docbook.org/ns/docbook" xmlns:xi="http://www.w3.org/2001/XInclude"\r
+xmlns:xl="http://www.w3.org/1999/xlink" version="5.0" xml:id="install_checklist">\r
+ <info>\r
+ <title>Evergreen Installation Checklist</title>\r
+ <abstract>\r
+ <para>This appendix is a checklist of things to do to istall and configure Evergreen. It will refer to the necessary chapter with the specific instructions for each item.</para>\r
+ </abstract>\r
+ </info>\r
+ <orderedlist>\r
+ <listitem>\r
+ <para><link linkend="serversideinstallation-opensrf">Install OpenSRF</link></para><indexterm><primary>OpenSRF</primary></indexterm>\r
+ </listitem>\r
+ <listitem>\r
+ <para><link linkend="serversideinstallation-ubuntudebian">Install Evergreen server software</link></para>\r
+ </listitem>\r
+ <listitem>\r
+ <para><link linkend="staffclientinstallation">Install Evergreen staff client</link></para><indexterm><primary>staff client</primary></indexterm>\r
+ </listitem>\r
+ <listitem>\r
+ <para><link linkend="backingup">Establish a back up strategy for Evergreen data and files</link></para>\r
+ </listitem>\r
+ <listitem>\r
+ <para><link linkend="configuringPostgreSQL">Configure PostgreSQL for better performance</link></para><indexterm><primary>PostgreSQL</primary></indexterm>\r
+ </listitem>\r
+ <listitem>\r
+ <para><link linkend="logfiles">Configure Evergreen error logging</link></para><indexterm><primary>logs</primary></indexterm>\r
+ </listitem>\r
+ <listitem>\r
+ <para><link linkend="orgtypes">Set up organizational unit types</link></para><indexterm><primary>organizational unit types</primary></indexterm>\r
+ </listitem>\r
+ <listitem>\r
+ <para><link linkend="orgunits">Set up organizational units</link></para><indexterm><primary>organizational units</primary></indexterm>\r
+ </listitem>\r
+ <listitem>\r
+ <para><link linkend="languagesandlocalization">Customize localization and languages</link> (optional)</para><indexterm><primary>localization and languages</primary></indexterm>\r
+ </listitem>\r
+ <listitem>\r
+ <para><link linkend="circmodifiers">Add circ modifiers</link></para><indexterm><primary>circulation modifiers</primary></indexterm>\r
+ </listitem>\r
+ <listitem>\r
+ <para><link linkend="copystatus">Configure copy statuses</link></para><indexterm><primary>copy status</primary></indexterm>\r
+ </listitem>\r
+ <listitem>\r
+ <para><link linkend="catalogingtemplates">Add cataloguing templates</link></para><indexterm><primary>cataloguing </primary><secondary>templates</secondary></indexterm>\r
+ </listitem>\r
+ <listitem>\r
+ <para><link linkend="permissions">Add user groups and assign permissions</link></para><indexterm><primary>permissions</primary></indexterm>\r
+ </listitem>\r
+ <listitem>\r
+ <para><link linkend="lsa">Adjust various Local Administration Settings</link></para>\r
+ </listitem>\r
+ <listitem>\r
+ <para>Adjust circulation policies and <link linkend="lsa-group-penalty">penalty threshholds</link> for groups</para><indexterm><primary>penalty threshholds</primary></indexterm>\r
+ </listitem>\r
+ <listitem>\r
+ <para><link linkend="admin-staff_accounts">Add staff users</link></para>\r
+ </listitem>\r
+ <listitem>\r
+ <para><link linkend="Customizing_OPAC">Customize OPAC</link> as needed</para><indexterm><primary>OPAC</primary><secondary>customizing</secondary></indexterm>\r
+ </listitem>\r
+ <listitem>\r
+ <para><link linkend="migratingdata">Import data</link></para><indexterm><primary>migrating</primary><secondary>importing data</secondary></indexterm>\r
+ </listitem>\r
+ <listitem>\r
+ <para><link linkend="report-introduction">Start the reporter service and set up reports</link></para><indexterm><primary>reports</primary><secondary>starting</secondary></indexterm>\r
+ </listitem>\r
+ <listitem>\r
+ <para><link linkend="notifications">Set up email notifications for holds and overdue items</link></para><indexterm><primary>notifications</primary></indexterm>\r
+ </listitem>\r
+ <listitem>\r
+ <para><link linkend="actiontriggers">Set up action triggers</link></para><indexterm><primary>action triggers</primary></indexterm>\r
+ </listitem>\r
+ <listitem>\r
+ <para><link linkend="z3950">Set up Z39.50 server</link> (optional)</para><indexterm><primary>Z39.50</primary></indexterm>\r
+ </listitem>\r
+ <listitem>\r
+ <para><link linkend="relevancyrankings">Adjust search relevancy settings</link> if required (optional)</para><indexterm><primary>search relevancy</primary></indexterm>\r
+ </listitem>\r
+ <listitem>\r
+ <para><link linkend="sipserver">Install SIP server</link> (optional) - for communications with automated devices such as self check stations, autmated sorters and other devices using SIP</para><indexterm><primary>SIP</primary></indexterm>\r
+ </listitem>\r
+ </orderedlist>\r
+</appendix>\r
--- /dev/null
+<?xml version="1.0" encoding="UTF-8"?>\r
+<chapter xml:id="intro_to_sql" xmlns="http://docbook.org/ns/docbook" version="5.0" xml:lang="EN"\r
+ xmlns:xi="http://www.w3.org/2001/XInclude" xmlns:xlink="http://www.w3.org/1999/xlink">\r
+ <chapterinfo>\r
+ <title>Introduction to SQL for Evergreen Administrators</title>\r
+ </chapterinfo>\r
+ <abstract id="itnroSQL_abstract">\r
+ <simpara>This chapter was taken from Dan Scott's <emphasis>Introduction to SQL for Evergreen Administrators</emphasis>, February 2010.</simpara>\r
+ </abstract> \r
+ <section id="intro_to_databases">\r
+ <title>Introduction to SQL Databases</title>\r
+ <indexterm><primary>sql</primary></indexterm>\r
+ <simplesect>\r
+ <title>Introduction</title>\r
+ <simpara>Over time, the SQL database has become the standard method of storing,\r
+ retrieving, and processing raw data for applications. Ranging from embedded\r
+ databases such as SQLite and Apache Derby, to enterprise databases such as\r
+ Oracle and IBM DB2, any SQL database offers basic advantages to application\r
+ developers such as standard interfaces (Structured Query Language (SQL), Java\r
+ Database Connectivity (JDBC), Open Database Connectivity (ODBC), Perl Database\r
+ Independent Interface (DBI)), a standard conceptual model of data (tables,\r
+ fields, relationships, constraints, etc), performance in storing and retrieving\r
+ data, concurrent access, etc.</simpara>\r
+ <simpara>Evergreen is built on PostgreSQL, an open source SQL database that began as\r
+ <literal>POSTGRES</literal> at the University of California at Berkeley in 1986 as a research\r
+ project led by Professor Michael Stonebraker. A SQL interface was added to a\r
+ fork of the original POSTGRES Berkelely code in 1994, and in 1996 the project\r
+ was renamed PostgreSQL.</simpara>\r
+ </simplesect>\r
+ <simplesect id="_tables">\r
+ <title>Tables</title>\r
+ <indexterm><primary>sql</primary><secondary>tables</secondary></indexterm>\r
+ <simpara>The table is the cornerstone of a SQL database. Conceptually, a database table\r
+ is similar to a single sheet in a spreadsheet: every table has one or more\r
+ columns, with each row in the table containing values for each column. Each\r
+ column in a table defines an attribute corresponding to a particular data type.</simpara>\r
+ <simpara>We’ll insert a row into a table, then display the resulting contents. Don’t\r
+ worry if the INSERT statement is completely unfamiliar, we’ll talk more about\r
+ the syntax of the insert statement later.</simpara>\r
+ <formalpara><title><literal>actor.usr_note</literal> database table</title><para>\r
+<programlisting language="sql" linenumbering="unnumbered">\r
+evergreen=# INSERT INTO actor.usr_note (usr, creator, pub, title, value)\r
+ VALUES (1, 1, TRUE, 'Who is this guy?', 'He''s the administrator!');\r
+\r
+evergreen=# select id, usr, creator, pub, title, value from actor.usr_note;\r
+ id | usr | creator | pub | title | value\r
+----+-----+---------+-----+------------------+-------------------------\r
+ 1 | 1 | 1 | t | Who is this guy? | He's the administrator!\r
+(1 rows)\r
+</programlisting>\r
+ </para></formalpara>\r
+ <simpara>PostgreSQL supports table inheritance, which lets you define tables that\r
+ inherit the column definitions of a given parent table. A search of the data in\r
+ the parent table includes the data in the child tables. Evergreen uses table\r
+ inheritance: for example, the <literal>action.circulation</literal> table is a child of the\r
+ <literal>money.billable_xact</literal> table, and the <literal>money.*_payment</literal> tables all inherit from\r
+ the <literal>money.payment</literal> parent table.</simpara>\r
+ </simplesect>\r
+ <simplesect id="_schemas">\r
+ <title>Schemas</title>\r
+ <simpara>PostgreSQL, like most SQL databases, supports the use of schema names to group\r
+ collections of tables and other database objects together. You might think of\r
+ schemas as namespaces if you’re a programmer; or you might think of the schema\r
+ / table / column relationship like the area code / exchange / local number\r
+ structure of a telephone number.</simpara>\r
+ <table\r
+ frame="all"\r
+ rowsep="1" colsep="1"\r
+ >\r
+ <title>Examples: database object names</title>\r
+ <?dbhtml table-width="100%"?>\r
+ <?dbfo table-width="100%"?>\r
+ <tgroup cols="4">\r
+ <colspec colname="col_1" colwidth="2.0*"/>\r
+ <colspec colname="col_2" colwidth="1.0*"/>\r
+ <colspec colname="col_3" colwidth="1.0*"/>\r
+ <colspec colname="col_4" colwidth="1.0*"/>\r
+ <thead>\r
+ <row>\r
+ <entry align="left" valign="top">Full name </entry>\r
+ <entry align="left" valign="top">Schema name </entry>\r
+ <entry align="left" valign="top">Table name </entry>\r
+ <entry align="left" valign="top">Field name</entry>\r
+ </row>\r
+ </thead>\r
+ <tbody>\r
+ <row>\r
+ <entry align="left" valign="top"><simpara>actor.usr_note.title</simpara></entry>\r
+ <entry align="left" valign="top"><simpara>actor</simpara></entry>\r
+ <entry align="left" valign="top"><simpara>usr_note</simpara></entry>\r
+ <entry align="left" valign="top"><simpara>title</simpara></entry>\r
+ </row>\r
+ <row>\r
+ <entry align="left" valign="top"><simpara>biblio.record_entry.marc</simpara></entry>\r
+ <entry align="left" valign="top"><simpara>biblio</simpara></entry>\r
+ <entry align="left" valign="top"><simpara>record_entry</simpara></entry>\r
+ <entry align="left" valign="top"><simpara>marc</simpara></entry>\r
+ </row>\r
+ </tbody>\r
+ </tgroup>\r
+ </table>\r
+ <simpara>The default schema name in PostgreSQL is <literal>public</literal>, so if you do not specify a\r
+ schema name when creating or accessing a database object, PostgreSQL will use\r
+ the <literal>public</literal> schema. As a result, you might not find the object that you’re\r
+ looking for if you don’t use the appropriate schema.</simpara>\r
+ <formalpara><title>Example: Creating a table without a specific schema</title><para>\r
+<programlisting language="sql" linenumbering="unnumbered">\r
+evergreen=# CREATE TABLE foobar (foo TEXT, bar TEXT);\r
+CREATE TABLE\r
+evergreen=# \d foobar\r
+ Table "public.foobar"\r
+ Column | Type | Modifiers\r
+--------+------+-----------\r
+ foo | text |\r
+ bar | text |\r
+</programlisting>\r
+ </para></formalpara>\r
+ <formalpara><title>Example: Trying to access a unqualified table outside of the public schema</title><para>\r
+ <programlisting language="sql" linenumbering="unnumbered">evergreen=# SELECT * FROM usr_note;\r
+ ERROR: relation "usr_note" does not exist\r
+ LINE 1: SELECT * FROM usr_note;\r
+ ^</programlisting>\r
+ </para></formalpara>\r
+ <simpara>Evergreen uses schemas to organize all of its tables with mostly intuitive,\r
+ if short, schema names. Here’s the current (as of 2010-01-03) list of schemas\r
+ used by Evergreen:</simpara>\r
+ <table\r
+ frame="all"\r
+ rowsep="1" colsep="1"\r
+ >\r
+ <title>Evergreen schema names</title>\r
+ <?dbhtml table-width="80%"?>\r
+ <?dbfo table-width="80%"?>\r
+ <tgroup cols="2">\r
+ <colspec colname="col_1" colwidth="1.0*"/>\r
+ <colspec colname="col_2" colwidth="1.0*"/>\r
+ <thead>\r
+ <row>\r
+ <entry align="left" valign="top">Schema name </entry>\r
+ <entry align="left" valign="top">Description</entry>\r
+ </row>\r
+ </thead>\r
+ <tbody>\r
+ <row>\r
+ <entry align="left" valign="top"><simpara><literal>acq</literal></simpara></entry>\r
+ <entry align="left" valign="top"><simpara>Acquisitions</simpara></entry>\r
+ </row>\r
+ <row>\r
+ <entry align="left" valign="top"><simpara><literal>action</literal></simpara></entry>\r
+ <entry align="left" valign="top"><simpara>Circulation actions</simpara></entry>\r
+ </row>\r
+ <row>\r
+ <entry align="left" valign="top"><simpara><literal>action_trigger</literal></simpara></entry>\r
+ <entry align="left" valign="top"><simpara>Event mechanisms</simpara></entry>\r
+ </row>\r
+ <row>\r
+ <entry align="left" valign="top"><simpara><literal>actor</literal></simpara></entry>\r
+ <entry align="left" valign="top"><simpara>Evergreen users and organization units</simpara></entry>\r
+ </row>\r
+ <row>\r
+ <entry align="left" valign="top"><simpara><literal>asset</literal></simpara></entry>\r
+ <entry align="left" valign="top"><simpara>Call numbers and copies</simpara></entry>\r
+ </row>\r
+ <row>\r
+ <entry align="left" valign="top"><simpara><literal>auditor</literal></simpara></entry>\r
+ <entry align="left" valign="top"><simpara>Track history of changes to selected tables</simpara></entry>\r
+ </row>\r
+ <row>\r
+ <entry align="left" valign="top"><simpara><literal>authority</literal></simpara></entry>\r
+ <entry align="left" valign="top"><simpara>Authority records</simpara></entry>\r
+ </row>\r
+ <row>\r
+ <entry align="left" valign="top"><simpara><literal>biblio</literal></simpara></entry>\r
+ <entry align="left" valign="top"><simpara>Bibliographic records</simpara></entry>\r
+ </row>\r
+ <row>\r
+ <entry align="left" valign="top"><simpara><literal>booking</literal></simpara></entry>\r
+ <entry align="left" valign="top"><simpara>Resource bookings</simpara></entry>\r
+ </row>\r
+ <row>\r
+ <entry align="left" valign="top"><simpara><literal>config</literal></simpara></entry>\r
+ <entry align="left" valign="top"><simpara>Evergreen configurable options</simpara></entry>\r
+ </row>\r
+ <row>\r
+ <entry align="left" valign="top"><simpara><literal>container</literal></simpara></entry>\r
+ <entry align="left" valign="top"><simpara>Buckets for records, call numbers, copies, and users</simpara></entry>\r
+ </row>\r
+ <row>\r
+ <entry align="left" valign="top"><simpara><literal>extend_reporter</literal></simpara></entry>\r
+ <entry align="left" valign="top"><simpara>Extra views for report definitions</simpara></entry>\r
+ </row>\r
+ <row>\r
+ <entry align="left" valign="top"><simpara><literal>metabib</literal></simpara></entry>\r
+ <entry align="left" valign="top"><simpara>Metadata about bibliographic records</simpara></entry>\r
+ </row>\r
+ <row>\r
+ <entry align="left" valign="top"><simpara><literal>money</literal></simpara></entry>\r
+ <entry align="left" valign="top"><simpara>Fines and bills</simpara></entry>\r
+ </row>\r
+ <row>\r
+ <entry align="left" valign="top"><simpara><literal>offline</literal></simpara></entry>\r
+ <entry align="left" valign="top"><simpara>Offline transactions</simpara></entry>\r
+ </row>\r
+ <row>\r
+ <entry align="left" valign="top"><simpara><literal>permission</literal></simpara></entry>\r
+ <entry align="left" valign="top"><simpara>User permissions</simpara></entry>\r
+ </row>\r
+ <row>\r
+ <entry align="left" valign="top"><simpara><literal>query</literal></simpara></entry>\r
+ <entry align="left" valign="top"><simpara>Stored SQL statements</simpara></entry>\r
+ </row>\r
+ <row>\r
+ <entry align="left" valign="top"><simpara><literal>reporter</literal></simpara></entry>\r
+ <entry align="left" valign="top"><simpara>Report definitions</simpara></entry>\r
+ </row>\r
+ <row>\r
+ <entry align="left" valign="top"><simpara><literal>search</literal></simpara></entry>\r
+ <entry align="left" valign="top"><simpara>Search functions</simpara></entry>\r
+ </row>\r
+ <row>\r
+ <entry align="left" valign="top"><simpara><literal>serial</literal></simpara></entry>\r
+ <entry align="left" valign="top"><simpara>Serial MFHD records</simpara></entry>\r
+ </row>\r
+ <row>\r
+ <entry align="left" valign="top"><simpara><literal>stats</literal></simpara></entry>\r
+ <entry align="left" valign="top"><simpara>Convenient views of circulation and asset statistics</simpara></entry>\r
+ </row>\r
+ <row>\r
+ <entry align="left" valign="top"><simpara><literal>vandelay</literal></simpara></entry>\r
+ <entry align="left" valign="top"><simpara>MARC batch importer and exporter</simpara></entry>\r
+ </row>\r
+ </tbody>\r
+ </tgroup>\r
+ </table>\r
+ <note><simpara>The term <emphasis>schema</emphasis> has two meanings in the world of SQL databases. We have\r
+ discussed the schema as a conceptual grouping of tables and other database\r
+ objects within a given namespace; for example, "the <emphasis role="strong">actor</emphasis> schema contains the\r
+ tables and functions related to users and organizational units". Another common\r
+ usage of <emphasis>schema</emphasis> is to refer to the entire data model for a given database;\r
+ for example, "the Evergreen database schema".</simpara></note>\r
+ </simplesect>\r
+ <simplesect id="_columns">\r
+ <title>Columns</title>\r
+ <simpara>Each column definition consists of:</simpara>\r
+ <itemizedlist>\r
+ <listitem>\r
+ <simpara>\r
+ a data type\r
+ </simpara>\r
+ </listitem>\r
+ <listitem>\r
+ <simpara>\r
+ (optionally) a default value to be used whenever a row is inserted that\r
+ does not contain a specific value\r
+ </simpara>\r
+ </listitem>\r
+ <listitem>\r
+ <simpara>\r
+ (optionally) one or more constraints on the values beyond data type\r
+ </simpara>\r
+ </listitem>\r
+ </itemizedlist>\r
+ <simpara>Although PostgreSQL supports dozens of data types, Evergreen makes our life\r
+ easier by only using a handful.</simpara>\r
+ <table\r
+ frame="all"\r
+ rowsep="1" colsep="1"\r
+ >\r
+ <title>PostgreSQL data types used by Evergreen</title>\r
+ <?dbhtml table-width="90%"?>\r
+ <?dbfo table-width="90%"?>\r
+ <tgroup cols="3">\r
+ <colspec colname="col_1" colwidth="1.0*"/>\r
+ <colspec colname="col_2" colwidth="1.0*"/>\r
+ <colspec colname="col_3" colwidth="2.5*"/>\r
+ <thead>\r
+ <row>\r
+ <entry align="left" valign="top">Type name </entry>\r
+ <entry align="left" valign="top">Description </entry>\r
+ <entry align="left" valign="top">Limits</entry>\r
+ </row>\r
+ </thead>\r
+ <tbody>\r
+ <row>\r
+ <entry align="left" valign="top"><simpara><literal>INTEGER</literal></simpara></entry>\r
+ <entry align="left" valign="top"><simpara>Medium integer</simpara></entry>\r
+ <entry align="left" valign="top"><simpara>-2147483648 to +2147483647</simpara></entry>\r
+ </row>\r
+ <row>\r
+ <entry align="left" valign="top"><simpara><literal>BIGINT</literal></simpara></entry>\r
+ <entry align="left" valign="top"><simpara>Large integer</simpara></entry>\r
+ <entry align="left" valign="top"><simpara>-9223372036854775808 to 9223372036854775807</simpara></entry>\r
+ </row>\r
+ <row>\r
+ <entry align="left" valign="top"><simpara><literal>SERIAL</literal></simpara></entry>\r
+ <entry align="left" valign="top"><simpara>Sequential integer</simpara></entry>\r
+ <entry align="left" valign="top"><simpara>1 to 2147483647</simpara></entry>\r
+ </row>\r
+ <row>\r
+ <entry align="left" valign="top"><simpara><literal>BIGSERIAL</literal></simpara></entry>\r
+ <entry align="left" valign="top"><simpara>Large sequential integer</simpara></entry>\r
+ <entry align="left" valign="top"><simpara>1 to 9223372036854775807</simpara></entry>\r
+ </row>\r
+ <row>\r
+ <entry align="left" valign="top"><simpara><literal>TEXT</literal></simpara></entry>\r
+ <entry align="left" valign="top"><simpara>Variable length character data</simpara></entry>\r
+ <entry align="left" valign="top"><simpara>Unlimited length</simpara></entry>\r
+ </row>\r
+ <row>\r
+ <entry align="left" valign="top"><simpara><literal>BOOL</literal></simpara></entry>\r
+ <entry align="left" valign="top"><simpara>Boolean</simpara></entry>\r
+ <entry align="left" valign="top"><simpara>TRUE or FALSE</simpara></entry>\r
+ </row>\r
+ <row>\r
+ <entry align="left" valign="top"><simpara><literal>TIMESTAMP WITH TIME ZONE</literal></simpara></entry>\r
+ <entry align="left" valign="top"><simpara>Timestamp</simpara></entry>\r
+ <entry align="left" valign="top"><simpara>4713 BC to 294276 AD</simpara></entry>\r
+ </row>\r
+ <row>\r
+ <entry align="left" valign="top"><simpara><literal>TIME</literal></simpara></entry>\r
+ <entry align="left" valign="top"><simpara>Time</simpara></entry>\r
+ <entry align="left" valign="top"><simpara>Expressed in HH:MM:SS</simpara></entry>\r
+ </row>\r
+ <row>\r
+ <entry align="left" valign="top"><simpara><literal>NUMERIC</literal>(precision, scale)</simpara></entry>\r
+ <entry align="left" valign="top"><simpara>Decimal</simpara></entry>\r
+ <entry align="left" valign="top"><simpara>Up to 1000 digits of precision. In Evergreen mostly used for money\r
+ values, with a precision of 6 and a scale of 2 (<literal>####.##</literal>).</simpara></entry>\r
+ </row>\r
+ </tbody>\r
+ </tgroup>\r
+ </table>\r
+ <simpara>Full details about these data types are available from the\r
+ <ulink url="http://www.postgresql.org/docs/8.4/static/datatype.html">data types section of\r
+ the PostgreSQL manual</ulink>.</simpara>\r
+ </simplesect>\r
+ <simplesect id="_constraints">\r
+ <title>Constraints</title>\r
+ <simplesect id="_prevent_null_values">\r
+ <title>Prevent NULL values</title>\r
+ <simpara>A column definition may include the constraint <literal>NOT NULL</literal> to prevent NULL\r
+ values. In PostgreSQL, a NULL value is not the equivalent of zero or false or\r
+ an empty string; it is an explicit non-value with special properties. We’ll\r
+ talk more about how to work with NULL values when we get to queries.</simpara>\r
+ </simplesect>\r
+ <simplesect id="_primary_key">\r
+ <title>Primary key</title>\r
+ <simpara>Every table can have at most one primary key. A primary key consists of one or\r
+ more columns which together uniquely identify each row in a table. If you\r
+ attempt to insert a row into a table that would create a duplicate or NULL\r
+ primary key entry, the database rejects the row and returns an error.</simpara>\r
+ <simpara>Natural primary keys are drawn from the intrinsic properties of the data being\r
+ modelled. For example, some potential natural primary keys for a table that\r
+ contains people would be:</simpara>\r
+ <table\r
+ frame="all"\r
+ rowsep="1" colsep="1"\r
+ >\r
+ <title>Example: Some potential natural primary keys for a table of people</title>\r
+ <?dbhtml table-width="90%"?>\r
+ <?dbfo table-width="90%"?>\r
+ <tgroup cols="3">\r
+ <colspec colname="col_1" colwidth="1.0*"/>\r
+ <colspec colname="col_2" colwidth="2.0*"/>\r
+ <colspec colname="col_3" colwidth="2.0*"/>\r
+ <thead>\r
+ <row>\r
+ <entry align="left" valign="top">Natural key </entry>\r
+ <entry align="left" valign="top">Pros </entry>\r
+ <entry align="left" valign="top">Cons</entry>\r
+ </row>\r
+ </thead>\r
+ <tbody>\r
+ <row>\r
+ <entry align="left" valign="top"><simpara>First name, last name, address</simpara></entry>\r
+ <entry align="left" valign="top"><simpara>No two people with the same name would ever live at the same address, right?</simpara></entry>\r
+ <entry align="left" valign="top"><simpara>Lots of columns force data duplication in referencing tables</simpara></entry>\r
+ </row>\r
+ <row>\r
+ <entry align="left" valign="top"><simpara>SSN or driver’s license</simpara></entry>\r
+ <entry align="left" valign="top"><simpara>These are guaranteed to be unique</simpara></entry>\r
+ <entry align="left" valign="top"><simpara>Lots of people don’t have an SSN or a driver’s license</simpara></entry>\r
+ </row>\r
+ </tbody>\r
+ </tgroup>\r
+ </table>\r
+ <simpara>To avoid problems with natural keys, many applications instead define surrogate\r
+ primary keys. A surrogate primary keys is a column with an autoincrementing\r
+ integer value added to a table definition that ensures uniqueness.</simpara>\r
+ <simpara>Evergreen uses surrogate keys (a column named <literal>id</literal> with a <literal>SERIAL</literal> data type)\r
+ for most of its tables.</simpara>\r
+ </simplesect>\r
+ <simplesect id="_foreign_keys">\r
+ <title>Foreign keys</title>\r
+ <simpara>Every table can contain zero or more foreign keys: one or more columns that\r
+ refer to the primary key of another table.</simpara>\r
+ <simpara>For example, let’s consider Evergreen’s modelling of the basic relationship\r
+ between copies, call numbers, and bibliographic records. Bibliographic records\r
+ contained in the <literal>biblio.record_entry</literal> table can have call numbers attached to\r
+ them. Call numbers are contained in the <literal>asset.call_number</literal> table, and they can\r
+ have copies attached to them. Copies are contained in the <literal>asset.copy</literal> table.</simpara>\r
+ <table\r
+ frame="all"\r
+ rowsep="1" colsep="1"\r
+ >\r
+ <title>Example: Evergreen’s copy / call number / bibliographic record relationships</title>\r
+ <?dbhtml table-width="100%"?>\r
+ <?dbfo table-width="100%"?>\r
+ <tgroup cols="4">\r
+ <colspec colname="col_1" colwidth="1.0*"/>\r
+ <colspec colname="col_2" colwidth="1.0*"/>\r
+ <colspec colname="col_3" colwidth="1.0*"/>\r
+ <colspec colname="col_4" colwidth="1.0*"/>\r
+ <thead>\r
+ <row>\r
+ <entry align="left" valign="top">Table </entry>\r
+ <entry align="left" valign="top">Primary key </entry>\r
+ <entry align="left" valign="top">Column with a foreign key </entry>\r
+ <entry align="left" valign="top">Points to</entry>\r
+ </row>\r
+ </thead>\r
+ <tbody>\r
+ <row>\r
+ <entry align="left" valign="top"><simpara>asset.copy</simpara></entry>\r
+ <entry align="left" valign="top"><simpara>asset.copy.id</simpara></entry>\r
+ <entry align="left" valign="top"><simpara>asset.copy.call_number</simpara></entry>\r
+ <entry align="left" valign="top"><simpara>asset.call_number.id</simpara></entry>\r
+ </row>\r
+ <row>\r
+ <entry align="left" valign="top"><simpara>asset.call_number</simpara></entry>\r
+ <entry align="left" valign="top"><simpara>asset.call_number.id</simpara></entry>\r
+ <entry align="left" valign="top"><simpara>asset.call_number.record</simpara></entry>\r
+ <entry align="left" valign="top"><simpara>biblio.record_entry.id</simpara></entry>\r
+ </row>\r
+ <row>\r
+ <entry align="left" valign="top"><simpara>biblio.record_entry</simpara></entry>\r
+ <entry align="left" valign="top"><simpara>biblio.record_entry.id</simpara></entry>\r
+ <entry align="left" valign="top"><simpara></simpara></entry>\r
+ <entry align="left" valign="top"><simpara></simpara></entry>\r
+ </row>\r
+ </tbody>\r
+ </tgroup>\r
+ </table>\r
+ </simplesect>\r
+ <simplesect id="_check_constraints">\r
+ <title>Check constraints</title>\r
+ <simpara>PostgreSQL enables you to define rules to ensure that the value to be inserted\r
+ or updated meets certain conditions. For example, you can ensure that an\r
+ incoming integer value is within a specific range, or that a ZIP code matches a\r
+ particular pattern.</simpara>\r
+ </simplesect>\r
+ </simplesect>\r
+ <simplesect id="_deconstructing_a_table_definition_statement">\r
+ <title>Deconstructing a table definition statement</title>\r
+ <simpara>The <literal>actor.org_address</literal> table is a simple table in the Evergreen schema that\r
+ we can use as a concrete example of many of the properties of databases that\r
+ we have discussed so far.</simpara>\r
+<programlisting language="sql" linenumbering="unnumbered">\r
+CREATE TABLE actor.org_address (\r
+ id SERIAL PRIMARY KEY, <co id="sqlCO1-1"/>\r
+ valid BOOL NOT NULL DEFAULT TRUE, <co id="sqlCO1-2"/>\r
+ address_type TEXT NOT NULL DEFAULT 'MAILING', <co id="sqlCO1-3"/>\r
+ org_unit INT NOT NULL REFERENCES actor.org_unit (id) <co id="sqlCO1-4"/>\r
+ DEFERRABLE INITIALLY DEFERRED,\r
+ street1 TEXT NOT NULL,\r
+ street2 TEXT, <co id="sqlCO1-5"/>\r
+ city TEXT NOT NULL,\r
+ county TEXT,\r
+ state TEXT NOT NULL,\r
+ country TEXT NOT NULL,\r
+ post_code TEXT NOT NULL\r
+);\r
+</programlisting>\r
+ <calloutlist>\r
+ <callout arearefs="sqlCO1-1">\r
+ <simpara>\r
+ The column named <literal>id</literal> is defined with a special data type of <literal>SERIAL</literal>; if\r
+ given no value when a row is inserted into a table, the database automatically\r
+ generates the next sequential integer value for the column. <literal>SERIAL</literal> is a\r
+ popular data type for a primary key because it is guaranteed to be unique - and\r
+ indeed, the constraint for this column identifies it as the <literal>PRIMARY KEY</literal>.\r
+ </simpara>\r
+ </callout>\r
+ <callout arearefs="sqlCO1-2">\r
+ <simpara>\r
+ The data type <literal>BOOL</literal> defines a boolean value: <literal>TRUE</literal> or <literal>FALSE</literal> are the only\r
+ acceptable values for the column. The constraint <literal>NOT NULL</literal> instructs the\r
+ database to prevent the column from ever containing a NULL value. The column\r
+ property <literal>DEFAULT TRUE</literal> instructs the database to automatically set the value\r
+ of the column to <literal>TRUE</literal> if no value is provided.\r
+ </simpara>\r
+ </callout>\r
+ <callout arearefs="sqlCO1-3">\r
+ <simpara>\r
+ The data type <literal>TEXT</literal> defines a text column of practically unlimited length.\r
+ As with the previous column, there is a <literal>NOT NULL</literal> constraint, and a default\r
+ value of <literal>'MAILING'</literal> will result if no other value is supplied.\r
+ </simpara>\r
+ </callout>\r
+ <callout arearefs="sqlCO1-4">\r
+ <simpara>\r
+ The <literal>REFERENCES actor.org_unit (id)</literal> clause indicates that this column has a\r
+ foreign key relationship to the <literal>actor.org_unit</literal> table, and that the value of\r
+ this column in every row in this table must have a corresponding value in the\r
+ <literal>id</literal> column in the referenced table (<literal>actor.org_unit</literal>).\r
+ </simpara>\r
+ </callout>\r
+ <callout arearefs="sqlCO1-5">\r
+ <simpara>\r
+ The column named <literal>street2</literal> demonstrates that not all columns have constraints\r
+ beyond data type. In this case, the column is allowed to be NULL or to contain a\r
+ <literal>TEXT</literal> value.\r
+ </simpara>\r
+ </callout>\r
+ </calloutlist>\r
+ </simplesect>\r
+ <simplesect id="_displaying_a_table_definition_using_literal_psql_literal">\r
+ <title>Displaying a table definition using <literal>psql</literal></title>\r
+ <simpara>The <literal>psql</literal> command-line interface is the preferred method for accessing\r
+ PostgreSQL databases. It offers features like tab-completion, readline support\r
+ for recalling previous commands, flexible input and output formats, and\r
+ is accessible via a standard SSH session.</simpara>\r
+ <simpara>If you press the <literal>Tab</literal> key once after typing one or more characters of the\r
+ database object name, <literal>psql</literal> automatically completes the name if there are no\r
+ other matches. If there are other matches for your current input, nothing\r
+ happens until you press the <literal>Tab</literal> key a second time, at which point <literal>psql</literal>\r
+ displays all of the matches for your current input.</simpara>\r
+ <simpara>To display the definition of a database object such as a table, issue the\r
+ command <literal>\d _object-name_</literal>. For example, to display the definition of the\r
+ actor.usr_note table:</simpara>\r
+<programlisting language="sh" linenumbering="unnumbered">\r
+$ psql evergreen <co id="sqlCO2-1"/>\r
+psql (8.4.1)\r
+Type "help" for help.\r
+\r
+evergreen=# \d actor.usr_note <co id="sqlCO2-2"/>\r
+ Table "actor.usr_note"\r
+ Column | Type | Modifiers\r
+-------------+--------------------------+-------------------------------------------------------------\r
+ id | bigint | not null default nextval('actor.usr_note_id_seq'::regclass)\r
+ usr | bigint | not null\r
+ creator | bigint | not null\r
+ create_date | timestamp with time zone | default now()\r
+ pub | boolean | not null default false\r
+ title | text | not null\r
+ value | text | not null\r
+Indexes:\r
+ "usr_note_pkey" PRIMARY KEY, btree (id)\r
+ "actor_usr_note_creator_idx" btree (creator)\r
+ "actor_usr_note_usr_idx" btree (usr)\r
+Foreign-key constraints:\r
+ "usr_note_creator_fkey" FOREIGN KEY (creator) REFERENCES actor.usr(id) ON ...\r
+ "usr_note_usr_fkey" FOREIGN KEY (usr) REFERENCES actor.usr(id) ON DELETE ....\r
+\r
+evergreen=# \q <co id="sqlCO2-3"/>\r
+$\r
+</programlisting>\r
+ <calloutlist>\r
+ <callout arearefs="sqlCO2-1">\r
+ <simpara>\r
+ This is the most basic connection to a PostgreSQL database. You can use a\r
+ number of other flags to specify user name, hostname, port, and other options.\r
+ </simpara>\r
+ </callout>\r
+ <callout arearefs="sqlCO2-2">\r
+ <simpara>\r
+ The <literal>\d</literal> command displays the definition of a database object.\r
+ </simpara>\r
+ </callout>\r
+ <callout arearefs="sqlCO2-3">\r
+ <simpara>\r
+ The <literal>\q</literal> command quits the <literal>psql</literal> session and returns you to the shell prompt.\r
+ </simpara>\r
+ </callout>\r
+ </calloutlist>\r
+ </simplesect>\r
+ </section>\r
+ <section id="basic_sql_queries">\r
+ <title>Basic SQL queries</title>\r
+ <simplesect id="_the_select_statement">\r
+ <title>The SELECT statement</title>\r
+ <simpara>The SELECT statement is the basic tool for retrieving information from a\r
+ database. The syntax for most SELECT statements is:</simpara>\r
+ <blockquote>\r
+ <literallayout><literal>SELECT</literal> [<emphasis>columns(s)</emphasis>]\r
+ <literal>FROM</literal> [<emphasis>table(s)</emphasis>]\r
+ [<literal>WHERE</literal> <emphasis>condition(s)</emphasis>]\r
+ [<literal>GROUP BY</literal> <emphasis>columns(s)</emphasis>]\r
+ [<literal>HAVING</literal> <emphasis>grouping-condition(s)</emphasis>]\r
+ [<literal>ORDER BY</literal> <emphasis>column(s)</emphasis>]\r
+ [<literal>LIMIT</literal> <emphasis>maximum-results</emphasis>]\r
+ [<literal>OFFSET</literal> <emphasis>start-at-result-#</emphasis>]\r
+ ;</literallayout>\r
+ </blockquote>\r
+ <simpara>For example, to select all of the columns for each row in the\r
+ <literal>actor.usr_address</literal> table, issue the following query:</simpara>\r
+ <programlisting language="sql" linenumbering="unnumbered">SELECT *\r
+ FROM actor.usr_address\r
+ ;</programlisting>\r
+ </simplesect>\r
+ <simplesect id="_selecting_particular_columns_from_a_table">\r
+ <title>Selecting particular columns from a table</title>\r
+ <simpara><literal>SELECT *</literal> returns all columns from all of the tables included in your query.\r
+ However, quite often you will want to return only a subset of the possible\r
+ columns. You can retrieve specific columns by listing the names of the columns\r
+ you want after the <literal>SELECT</literal> keyword. Separate each column name with a comma.</simpara>\r
+ <simpara>For example, to select just the city, county, and state from the\r
+ actor.usr_address table, issue the following query:</simpara>\r
+ <programlisting language="sql" linenumbering="unnumbered">SELECT city, county, state\r
+ FROM actor.usr_address\r
+ ;</programlisting>\r
+ </simplesect>\r
+ <simplesect id="_sorting_results_with_the_order_by_clause">\r
+ <title>Sorting results with the ORDER BY clause</title>\r
+ <simpara>By default, a SELECT statement returns rows matching your query with no\r
+ guarantee of any particular order in which they are returned. To force\r
+ the rows to be returned in a particular order, use the ORDER BY clause\r
+ to specify one or more columns to determine the sorting priority of the\r
+ rows.</simpara>\r
+ <simpara>For example, to sort the rows returned from your <literal>actor.usr_address</literal> query by\r
+ city, with county and then zip code as the tie breakers, issue the\r
+ following query:</simpara>\r
+<programlisting language="sql" linenumbering="unnumbered">\r
+SELECT city, county, state\r
+ FROM actor.usr_address\r
+ ORDER BY city, county, post_code\r
+;\r
+</programlisting>\r
+ </simplesect>\r
+ <simplesect id="_filtering_results_with_the_where_clause">\r
+ <title>Filtering results with the WHERE clause</title>\r
+ <simpara>Thus far, your results have been returning all of the rows in the table.\r
+ Normally, however, you would want to restrict the rows that are returned to the\r
+ subset of rows that match one or more conditions of your search. The <literal>WHERE</literal>\r
+ clause enables you to specify a set of conditions that filter your query\r
+ results. Each condition in the <literal>WHERE</literal> clause is an SQL expression that returns\r
+ a boolean (true or false) value.</simpara>\r
+ <simpara>For example, to restrict the results returned from your <literal>actor.usr_address</literal>\r
+ query to only those rows containing a state value of <emphasis>Connecticut</emphasis>, issue the\r
+ following query:</simpara>\r
+<programlisting language="sql" linenumbering="unnumbered">\r
+SELECT city, county, state\r
+ FROM actor.usr_address\r
+ WHERE state = 'Connecticut'\r
+ ORDER BY city, county, post_code\r
+;\r
+</programlisting>\r
+ <simpara>You can include more conditions in the <literal>WHERE</literal> clause with the <literal>OR</literal> and <literal>AND</literal>\r
+ operators. For example, to further restrict the results returned from your\r
+ <literal>actor.usr_address</literal> query to only those rows where the state column contains a\r
+ value of <emphasis>Connecticut</emphasis> and the city column contains a value of <emphasis>Hartford</emphasis>,\r
+ issue the following query:</simpara>\r
+<programlisting language="sql" linenumbering="unnumbered">\r
+SELECT city, county, state\r
+ FROM actor.usr_address\r
+ WHERE state = 'Connecticut'\r
+ AND city = 'Hartford'\r
+ ORDER BY city, county, post_code\r
+;\r
+</programlisting>\r
+ <note><simpara>To return rows where the state is <emphasis>Connecticut</emphasis> and the city is <emphasis>Hartford</emphasis> or\r
+ <emphasis>New Haven</emphasis>, you must use parentheses to explicitly group the city value\r
+ conditions together, or else the database will evaluate the <literal>OR city = 'New\r
+ Haven'</literal> clause entirely on its own and match all rows where the city column is\r
+ <emphasis>New Haven</emphasis>, even though the state might not be <emphasis>Connecticut</emphasis>.</simpara></note>\r
+ <formalpara><title>Trouble with OR</title><para>\r
+<programlisting language="sql" linenumbering="unnumbered">\r
+SELECT city, county, state\r
+ FROM actor.usr_address\r
+ WHERE state = 'Connecticut'\r
+ AND city = 'Hartford' OR city = 'New Haven'\r
+ ORDER BY city, county, post_code\r
+;\r
+\r
+-- Can return unwanted rows because the OR is not grouped!\r
+</programlisting>\r
+ </para></formalpara>\r
+ <formalpara><title>Grouped OR’ed conditions</title><para>\r
+<programlisting language="sql" linenumbering="unnumbered">\r
+SELECT city, county, state\r
+ FROM actor.usr_address\r
+ WHERE state = 'Connecticut'\r
+ AND (city = 'Hartford' OR city = 'New Haven')\r
+ ORDER BY city, county, post_code\r
+;\r
+\r
+-- The parentheses ensure that the OR is applied to the cities, and the\r
+-- state in either case must be 'Connecticut'\r
+</programlisting>\r
+ </para></formalpara>\r
+ <simplesect id="_comparison_operators">\r
+ <title>Comparison operators</title>\r
+ <simpara>Here is a partial list of comparison operators that are commonly used in\r
+ <literal>WHERE</literal> clauses:</simpara>\r
+ <simplesect id="_comparing_two_scalar_values">\r
+ <title>Comparing two scalar values</title>\r
+ <itemizedlist>\r
+ <listitem>\r
+ <simpara>\r
+ <literal>x = y</literal> (equal to)\r
+ </simpara>\r
+ </listitem>\r
+ <listitem>\r
+ <simpara>\r
+ <literal>x != y</literal> (not equal to)\r
+ </simpara>\r
+ </listitem>\r
+ <listitem>\r
+ <simpara>\r
+ <literal>x < y</literal> (less than)\r
+ </simpara>\r
+ </listitem>\r
+ <listitem>\r
+ <simpara>\r
+ <literal>x > y</literal> (greater than)\r
+ </simpara>\r
+ </listitem>\r
+ <listitem>\r
+ <simpara>\r
+ <literal>x LIKE y</literal> (TEXT value x matches a subset of TEXT y, where y is a string that\r
+ can contain <emphasis>%</emphasis> as a wildcard for 0 or more characters, and <emphasis>_</emphasis> as a wildcard\r
+ for a single character. For example, <literal>WHERE 'all you can eat fish and chips\r
+ and a big stick' LIKE '%fish%stick'</literal> would return TRUE)\r
+ </simpara>\r
+ </listitem>\r
+ <listitem>\r
+ <simpara>\r
+ <literal>x ILIKE y</literal> (like LIKE, but the comparison ignores upper-case / lower-case)\r
+ </simpara>\r
+ </listitem>\r
+ <listitem>\r
+ <simpara>\r
+ <literal>x IN y</literal> (x is in the list of values y, where y can be a list or a SELECT\r
+ statement that returns a list)\r
+ </simpara>\r
+ </listitem>\r
+ </itemizedlist>\r
+ </simplesect>\r
+ </simplesect>\r
+ </simplesect>\r
+ <simplesect id="_null_values">\r
+ <title>NULL values</title>\r
+ <simpara>SQL databases have a special way of representing the value of a column that has\r
+ no value: <literal>NULL</literal>. A <literal>NULL</literal> value is not equal to zero, and is not an empty\r
+ string; it is equal to nothing, not even another <literal>NULL</literal>, because it has no value\r
+ that can be compared.</simpara>\r
+ <simpara>To return rows from a table where a given column is not <literal>NULL</literal>, use the\r
+ <literal>IS NOT NULL</literal> comparison operator.</simpara>\r
+ <formalpara><title>Retrieving rows where a column is not <literal>NULL</literal></title><para>\r
+<programlisting language="sql" linenumbering="unnumbered">\r
+SELECT id, first_given_name, family_name\r
+ FROM actor.usr\r
+ WHERE second_given_name IS NOT NULL\r
+;\r
+</programlisting>\r
+ </para></formalpara>\r
+ <simpara>Similarly, to return rows from a table where a given column is <literal>NULL</literal>, use\r
+ the <literal>IS NULL</literal> comparison operator.</simpara>\r
+ <formalpara><title>Retrieving rows where a column is <literal>NULL</literal></title><para>\r
+<programlisting language="sql" linenumbering="unnumbered">\r
+SELECT id, first_given_name, second_given_name, family_name\r
+ FROM actor.usr\r
+ WHERE second_given_name IS NULL\r
+;\r
+\r
+ id | first_given_name | second_given_name | family_name\r
+----+------------------+-------------------+----------------\r
+ 1 | Administrator | | System Account\r
+(1 row)\r
+</programlisting>\r
+ </para></formalpara>\r
+ <simpara>Notice that the <literal>NULL</literal> value in the output is displayed as empty space,\r
+ indistinguishable from an empty string; this is the default display method in\r
+ <literal>psql</literal>. You can change the behaviour of <literal>psql</literal> using the <literal>pset</literal> command:</simpara>\r
+ <formalpara><title>Changing the way <literal>NULL</literal> values are displayed in <literal>psql</literal></title><para>\r
+<programlisting language="sql" linenumbering="unnumbered">\r
+evergreen=# \pset null '(null)'\r
+Null display is '(null)'.\r
+\r
+SELECT id, first_given_name, second_given_name, family_name\r
+ FROM actor.usr\r
+ WHERE second_given_name IS NULL\r
+;\r
+\r
+ id | first_given_name | second_given_name | family_name\r
+----+------------------+-------------------+----------------\r
+ 1 | Administrator | (null) | System Account\r
+(1 row)\r
+</programlisting>\r
+ </para></formalpara>\r
+ <simpara>Database queries within programming languages such as Perl and C have\r
+ special methods of checking for <literal>NULL</literal> values in returned results.</simpara>\r
+ </simplesect>\r
+ <simplesect id="_text_delimiter">\r
+ <title>Text delimiter: '</title>\r
+ <simpara>You might have noticed that we have been using the <literal>'</literal> character to delimit\r
+ TEXT values and values such as dates and times that are TEXT values. Sometimes,\r
+ however, your TEXT value itself contains a <literal>'</literal> character, such as the word\r
+ <literal>you’re</literal>. To prevent the database from prematurely ending the TEXT value at the\r
+ first <literal>'</literal> character and returning a syntax error, use another <literal>'</literal> character to\r
+ escape the following <literal>'</literal> character.</simpara>\r
+ <simpara>For example, to change the last name of a user in the <literal>actor.usr</literal> table to\r
+ <literal>L’estat</literal>, issue the following SQL:</simpara>\r
+ <formalpara><title>Escaping <literal>'</literal> in TEXT values</title><para>\r
+<programlisting language="sql" linenumbering="unnumbered">\r
+UPDATE actor.usr\r
+ SET family_name = 'L''estat'\r
+ WHERE profile IN (\r
+ SELECT id\r
+ FROM permission.grp_tree\r
+ WHERE name = 'Vampire'\r
+ )\r
+ ;</programlisting>\r
+ </para></formalpara>\r
+ <simpara>When you retrieve the row from the database, the value is displayed with just\r
+ a single <literal>'</literal> character:</simpara>\r
+<programlisting language="sql" linenumbering="unnumbered">\r
+SELECT id, family_name\r
+ FROM actor.usr\r
+ WHERE family_name = 'L''estat'\r
+;\r
+\r
+ id | family_name\r
+----+-------------\r
+ 1 | L'estat\r
+(1 row)\r
+</programlisting>\r
+ </simplesect>\r
+ <simplesect id="_grouping_and_eliminating_results_with_the_group_by_and_having_clauses">\r
+ <title>Grouping and eliminating results with the GROUP BY and HAVING clauses</title>\r
+ <simpara>The GROUP BY clause returns a unique set of results for the desired columns.\r
+ This is most often used in conjunction with an aggregate function to present\r
+ results for a range of values in a single query, rather than requiring you to\r
+ issue one query per target value.</simpara>\r
+ <formalpara><title>Returning unique results of a single column with <literal>GROUP BY</literal></title><para>\r
+<programlisting language="sql" linenumbering="unnumbered">\r
+SELECT grp\r
+ FROM permission.grp_perm_map\r
+ GROUP BY grp\r
+ ORDER BY grp;\r
+\r
+ grp\r
+-----+\r
+ 1\r
+ 2\r
+ 3\r
+ 4\r
+ 5\r
+ 6\r
+ 7\r
+ 10\r
+(8 rows)\r
+</programlisting>\r
+ </para></formalpara>\r
+ <simpara>While <literal>GROUP BY</literal> can be useful for a single column, it is more often used\r
+ to return the distinct results across multiple columns. For example, the\r
+ following query shows us which groups have permissions at each depth in\r
+ the library hierarchy:</simpara>\r
+ <formalpara><title>Returning unique results of multiple columns with <literal>GROUP BY</literal></title><para>\r
+<programlisting language="sql" linenumbering="unnumbered">\r
+SELECT grp, depth\r
+ FROM permission.grp_perm_map\r
+ GROUP BY grp, depth\r
+ ORDER BY depth, grp;\r
+\r
+ grp | depth\r
+-----+-------\r
+ 1 | 0\r
+ 2 | 0\r
+ 3 | 0\r
+ 4 | 0\r
+ 5 | 0\r
+ 10 | 0\r
+ 3 | 1\r
+ 4 | 1\r
+ 5 | 1\r
+ 6 | 1\r
+ 7 | 1\r
+ 10 | 1\r
+ 3 | 2\r
+ 4 | 2\r
+ 10 | 2\r
+(15 rows)\r
+</programlisting>\r
+ </para></formalpara>\r
+ <simpara>Extending this further, you can use the <literal>COUNT()</literal> aggregate function to\r
+ also return the number of times each unique combination of <literal>grp</literal> and <literal>depth</literal>\r
+ appears in the table. <emphasis>Yes, this is a sneak peek at the use of aggregate\r
+ functions! Keeners.</emphasis></simpara>\r
+ <formalpara><title>Counting unique column combinations with <literal>GROUP BY</literal></title><para>\r
+<programlisting language="sql" linenumbering="unnumbered">\r
+SELECT grp, depth, COUNT(grp)\r
+ FROM permission.grp_perm_map\r
+ GROUP BY grp, depth\r
+ ORDER BY depth, grp;\r
+\r
+ grp | depth | count\r
+-----+-------+-------\r
+ 1 | 0 | 6\r
+ 2 | 0 | 2\r
+ 3 | 0 | 45\r
+ 4 | 0 | 3\r
+ 5 | 0 | 5\r
+ 10 | 0 | 1\r
+ 3 | 1 | 3\r
+ 4 | 1 | 4\r
+ 5 | 1 | 1\r
+ 6 | 1 | 9\r
+ 7 | 1 | 5\r
+ 10 | 1 | 10\r
+ 3 | 2 | 24\r
+ 4 | 2 | 8\r
+ 10 | 2 | 7\r
+(15 rows)\r
+</programlisting>\r
+ </para></formalpara>\r
+ <simpara>You can use the <literal>WHERE</literal> clause to restrict the returned results before grouping\r
+ is applied to the results. The following query restricts the results to those\r
+ rows that have a depth of 0.</simpara>\r
+ <formalpara><title>Using the <literal>WHERE</literal> clause with <literal>GROUP BY</literal></title><para>\r
+<programlisting language="sql" linenumbering="unnumbered">\r
+SELECT grp, COUNT(grp)\r
+ FROM permission.grp_perm_map\r
+ WHERE depth = 0\r
+ GROUP BY grp\r
+ ORDER BY 2 DESC\r
+;\r
+\r
+ grp | count\r
+-----+-------\r
+ 3 | 45\r
+ 1 | 6\r
+ 5 | 5\r
+ 4 | 3\r
+ 2 | 2\r
+ 10 | 1\r
+(6 rows)\r
+</programlisting>\r
+ </para></formalpara>\r
+ <simpara>To restrict results after grouping has been applied to the rows, use the\r
+ <literal>HAVING</literal> clause; this is typically used to restrict results based on\r
+ a comparison to the value returned by an aggregate function. For example,\r
+ the following query restricts the returned rows to those that have more than\r
+ 5 occurrences of the same value for <literal>grp</literal> in the table.</simpara>\r
+ <formalpara><title><literal>GROUP BY</literal> restricted by a <literal>HAVING</literal> clause</title><para>\r
+<programlisting language="sql" linenumbering="unnumbered">\r
+SELECT grp, COUNT(grp)\r
+ FROM permission.grp_perm_map\r
+ GROUP BY grp\r
+ HAVING COUNT(grp) > 5\r
+;\r
+\r
+ grp | count\r
+-----+-------\r
+ 6 | 9\r
+ 4 | 15\r
+ 5 | 6\r
+ 1 | 6\r
+ 3 | 72\r
+ 10 | 18\r
+(6 rows)\r
+</programlisting>\r
+ </para></formalpara>\r
+ </simplesect>\r
+ <simplesect id="_eliminating_duplicate_results_with_the_distinct_keyword">\r
+ <title>Eliminating duplicate results with the DISTINCT keyword</title>\r
+ <simpara><literal>GROUP BY</literal> is one way of eliminating duplicate results from the rows returned\r
+ by your query. The purpose of the <literal>DISTINCT</literal> keyword is to remove duplicate\r
+ rows from the results of your query. However, it works, and it is easy - so if\r
+ you just want a quick list of the unique set of values for a column or set of\r
+ columns, the <literal>DISTINCT</literal> keyword might be appropriate.</simpara>\r
+ <simpara>On the other hand, if you are getting duplicate rows back when you don’t expect\r
+ them, then applying the <literal>DISTINCT</literal> keyword might be a sign that you are\r
+ papering over a real problem.</simpara>\r
+ <formalpara><title>Returning unique results of multiple columns with <literal>DISTINCT</literal></title><para>\r
+<programlisting language="sql" linenumbering="unnumbered">\r
+SELECT DISTINCT grp, depth\r
+ FROM permission.grp_perm_map\r
+ ORDER BY depth, grp\r
+;\r
+\r
+ grp | depth\r
+-----+-------\r
+ 1 | 0\r
+ 2 | 0\r
+ 3 | 0\r
+ 4 | 0\r
+ 5 | 0\r
+ 10 | 0\r
+ 3 | 1\r
+ 4 | 1\r
+ 5 | 1\r
+ 6 | 1\r
+ 7 | 1\r
+ 10 | 1\r
+ 3 | 2\r
+ 4 | 2\r
+ 10 | 2\r
+(15 rows)\r
+</programlisting>\r
+ </para></formalpara>\r
+ </simplesect>\r
+ <simplesect id="_paging_through_results_with_the_limit_and_offset_clauses">\r
+ <title>Paging through results with the LIMIT and OFFSET clauses</title>\r
+ <simpara>The <literal>LIMIT</literal> clause restricts the total number of rows returned from your query\r
+ and is useful if you just want to list a subset of a large number of rows. For\r
+ example, in the following query we list the five most frequently used\r
+ circulation modifiers:</simpara>\r
+ <formalpara><title>Using the <literal>LIMIT</literal> clause to restrict results</title><para>\r
+<programlisting language="sql" linenumbering="unnumbered">\r
+SELECT circ_modifier, COUNT(circ_modifier)\r
+ FROM asset.copy\r
+ GROUP BY circ_modifier\r
+ ORDER BY 2 DESC\r
+ LIMIT 5\r
+;\r
+\r
+ circ_modifier | count\r
+---------------+--------\r
+ CIRC | 741995\r
+ BOOK | 636199\r
+ SER | 265906\r
+ DOC | 191598\r
+ LAW MONO | 126627\r
+(5 rows)\r
+</programlisting>\r
+ </para></formalpara>\r
+ <simpara>When you use the <literal>LIMIT</literal> clause to restrict the total number of rows returned\r
+ by your query, you can also use the <literal>OFFSET</literal> clause to determine which subset\r
+ of the rows will be returned. The use of the <literal>OFFSET</literal> clause assumes that\r
+ you’ve used the <literal>ORDER BY</literal> clause to impose order on the results.</simpara>\r
+ <simpara>In the following example, we use the <literal>OFFSET</literal> clause to get results 6 through\r
+ 10 from the same query that we prevously executed.</simpara>\r
+ <formalpara><title>Using the <literal>OFFSET</literal> clause to return a specific subset of rows</title><para>\r
+<programlisting language="sql" linenumbering="unnumbered">\r
+SELECT circ_modifier, COUNT(circ_modifier)\r
+ FROM asset.copy\r
+ GROUP BY circ_modifier\r
+ ORDER BY 2 DESC\r
+ LIMIT 5\r
+ OFFSET 5\r
+;\r
+\r
+ circ_modifier | count\r
+---------------+--------\r
+ LAW SERIAL | 102758\r
+ DOCUMENTS | 86215\r
+ BOOK_WEB | 63786\r
+ MFORM SER | 39917\r
+ REF | 34380\r
+(5 rows)\r
+</programlisting>\r
+ </para></formalpara>\r
+ </simplesect>\r
+ </section>\r
+ <section id="advanced_sql_queries">\r
+ <title>Advanced SQL queries</title>\r
+ <simplesect id="_transforming_column_values_with_functions">\r
+ <title>Transforming column values with functions</title>\r
+ <simpara>PostgreSQL includes many built-in functions for manipulating column data.\r
+ You can also create your own functions (and Evergreen does make use of\r
+ many custom functions). There are two types of functions used in\r
+ databases: scalar functions and aggregate functions.</simpara>\r
+ <simplesect id="_scalar_functions">\r
+ <title>Scalar functions</title>\r
+ <simpara>Scalar functions transform each value of the target column. If your query\r
+ would return 50 values for a column in a given query, and you modify your\r
+ query to apply a scalar function to the values returned for that column,\r
+ it will still return 50 values. For example, the UPPER() function,\r
+ used to convert text values to upper-case, modifies the results in the\r
+ following set of queries:</simpara>\r
+ <formalpara><title>Using the UPPER() scalar function to convert text values to upper-case</title><para>\r
+<programlisting language="sql" linenumbering="unnumbered">\r
+-- First, without the UPPER() function for comparison\r
+SELECT shortname, name\r
+ FROM actor.org_unit\r
+ WHERE id < 4\r
+;\r
+\r
+ shortname | name\r
+-----------+-----------------------\r
+ CONS | Example Consortium\r
+ SYS1 | Example System 1\r
+ SYS2 | Example System 2\r
+(3 rows)\r
+\r
+-- Now apply the UPPER() function to the name column\r
+SELECT shortname, UPPER(name)\r
+ FROM actor.org_unit\r
+ WHERE id < 4\r
+;\r
+\r
+ shortname | upper\r
+-----------+--------------------\r
+ CONS | EXAMPLE CONSORTIUM\r
+ SYS1 | EXAMPLE SYSTEM 1\r
+ SYS2 | EXAMPLE SYSTEM 2\r
+(3 rows)\r
+</programlisting>\r
+ </para></formalpara>\r
+ <simpara>There are so many scalar functions in PostgreSQL that we cannot cover them\r
+ all here, but we can list some of the most commonly used functions:</simpara>\r
+ <itemizedlist>\r
+ <listitem>\r
+ <simpara>\r
+ || - concatenates two text values together\r
+ </simpara>\r
+ </listitem>\r
+ <listitem>\r
+ <simpara>\r
+ COALESCE() - returns the first non-NULL value from the list of arguments\r
+ </simpara>\r
+ </listitem>\r
+ <listitem>\r
+ <simpara>\r
+ LOWER() - returns a text value converted to lower-case\r
+ </simpara>\r
+ </listitem>\r
+ <listitem>\r
+ <simpara>\r
+ REPLACE() - returns a text value after replacing all occurrences of a given text value with a different text value\r
+ </simpara>\r
+ </listitem>\r
+ <listitem>\r
+ <simpara>\r
+ REGEXP_REPLACE() - returns a text value after being transformed by a regular expression\r
+ </simpara>\r
+ </listitem>\r
+ <listitem>\r
+ <simpara>\r
+ UPPER() - returns a text value converted to upper-case\r
+ </simpara>\r
+ </listitem>\r
+ </itemizedlist>\r
+ <simpara>For a complete list of scalar functions, see\r
+ <ulink url="http://www.postgresql.org/docs/8.3/interactive/functions.html">the PostgreSQL function documentation</ulink>.</simpara>\r
+ </simplesect>\r
+ <simplesect id="_aggregate_functions">\r
+ <title>Aggregate functions</title>\r
+ <simpara>Aggregate functions return a single value computed from the the complete set of\r
+ values returned for the specified column.</simpara>\r
+ <itemizedlist>\r
+ <listitem>\r
+ <simpara>\r
+ AVG()\r
+ </simpara>\r
+ </listitem>\r
+ <listitem>\r
+ <simpara>\r
+ COUNT()\r
+ </simpara>\r
+ </listitem>\r
+ <listitem>\r
+ <simpara>\r
+ MAX()\r
+ </simpara>\r
+ </listitem>\r
+ <listitem>\r
+ <simpara>\r
+ MIN()\r
+ </simpara>\r
+ </listitem>\r
+ <listitem>\r
+ <simpara>\r
+ SUM()\r
+ </simpara>\r
+ </listitem>\r
+ </itemizedlist>\r
+ </simplesect>\r
+ </simplesect>\r
+ <simplesect id="_sub_selects">\r
+ <title>Sub-selects</title>\r
+ <simpara>A sub-select is the technique of using the results of one query to feed\r
+ into another query. You can, for example, return a set of values from\r
+ one column in a SELECT statement to be used to satisfy the IN() condition\r
+ of another SELECT statement; or you could return the MAX() value of a\r
+ column in a SELECT statement to match the = condition of another SELECT\r
+ statement.</simpara>\r
+ <simpara>For example, in the following query we use a sub-select to restrict the copies\r
+ returned by the main SELECT statement to only those locations that have an\r
+ <literal>opac_visible</literal> value of <literal>TRUE</literal>:</simpara>\r
+ <formalpara><title>Sub-select example</title><para>\r
+<programlisting language="sql" linenumbering="unnumbered">\r
+SELECT call_number\r
+ FROM asset.copy\r
+ WHERE deleted IS FALSE\r
+ AND location IN (\r
+ SELECT id\r
+ FROM asset.copy_location\r
+ WHERE opac_visible IS TRUE\r
+ )\r
+;\r
+</programlisting>\r
+ </para></formalpara>\r
+ <simpara>Sub-selects can be an approachable way to breaking down a problem that\r
+ requires matching values between different tables, and often result in\r
+ a clearly expressed solution to a problem. However, if you start writing\r
+ sub-selects within sub-selects, you should consider tackling the problem\r
+ with joins instead.</simpara>\r
+ </simplesect>\r
+ <simplesect id="_joins">\r
+ <title>Joins</title>\r
+ <simpara>Joins enable you to access the values from multiple tables in your query\r
+ results and comparison operators. For example, joins are what enable you to\r
+ relate a bibliographic record to a barcoded copy via the <literal>biblio.record_entry</literal>,\r
+ <literal>asset.call_number</literal>, and <literal>asset.copy</literal> tables. In this section, we discuss the\r
+ most common kind of join—the inner join—as well as the less common outer join\r
+ and some set operations which can compare and contrast the values returned by\r
+ separate queries.</simpara>\r
+ <simpara>When we talk about joins, we are going to talk about the left-hand table and\r
+ the right-hand table that participate in the join. Every join brings together\r
+ just two tables - but you can use an unlimited (for our purposes) number\r
+ of joins in a single SQL statement. Each time you use a join, you effectively\r
+ create a new table, so when you add a second join clause to a statement,\r
+ table 1 and table 2 (which were the left-hand table and the right-hand table\r
+ for the first join) now act as a merged left-hand table and the new table\r
+ in the second join clause is the right-hand table.</simpara>\r
+ <simpara>Clear as mud? Okay, let’s look at some examples.</simpara>\r
+ <simplesect id="_inner_joins">\r
+ <title>Inner joins</title>\r
+ <simpara>An inner join returns all of the columns from the left-hand table in the join\r
+ with all of the columns from the right-hand table in the joins that match a\r
+ condition in the ON clause. Typically, you use the <literal>=</literal> operator to match the\r
+ foreign key of the left-hand table with the primary key of the right-hand\r
+ table to follow the natural relationship between the tables.</simpara>\r
+ <simpara>In the following example, we return all of columns from the <literal>actor.usr</literal> and\r
+ <literal>actor.org_unit</literal> tables, joined on the relationship between the user’s home\r
+ library and the library’s ID. Notice in the results that some columns, like\r
+ <literal>id</literal> and <literal>mailing_address</literal>, appear twice; this is because both the <literal>actor.usr</literal>\r
+ and <literal>actor.org_unit</literal> tables include columns with these names. This is also why\r
+ we have to fully qualify the column names in our queries with the schema and\r
+ table names.</simpara>\r
+ <formalpara><title>A simple inner join</title><para>\r
+<programlisting language="sql" linenumbering="unnumbered">\r
+SELECT *\r
+ FROM actor.usr\r
+ INNER JOIN actor.org_unit ON actor.usr.home_ou = actor.org_unit.id\r
+ WHERE actor.org_unit.shortname = 'CONS'\r
+;\r
+\r
+-[ RECORD 1 ]------------------+---------------------------------\r
+id | 1\r
+card | 1\r
+profile | 1\r
+usrname | admin\r
+email |\r
+...\r
+mailing_address |\r
+billing_address |\r
+home_ou | 1\r
+...\r
+claims_never_checked_out_count | 0\r
+id | 1\r
+parent_ou |\r
+ou_type | 1\r
+ill_address | 1\r
+holds_address | 1\r
+mailing_address | 1\r
+billing_address | 1\r
+shortname | CONS\r
+name | Example Consortium\r
+email |\r
+phone |\r
+opac_visible | t\r
+fiscal_calendar | 1\r
+</programlisting>\r
+ </para></formalpara>\r
+ <simpara>Of course, you do not have to return every column from the joined tables;\r
+ you can (and should) continue to specify only the columns that you want to\r
+ return. In the following example, we count the number of borrowers for\r
+ every user profile in a given library by joining the <literal>permission.grp_tree</literal>\r
+ table where profiles are defined against the <literal>actor.usr</literal> table, and then\r
+ joining the <literal>actor.org_unit</literal> table to give us access to the user’s home\r
+ library:</simpara>\r
+ <formalpara><title>Borrower Count by Profile (Adult, Child, etc)/Library</title><para>\r
+<programlisting language="sql" linenumbering="unnumbered">\r
+SELECT permission.grp_tree.name, actor.org_unit.name, COUNT(permission.grp_tree.name)\r
+ FROM actor.usr\r
+ INNER JOIN permission.grp_tree\r
+ ON actor.usr.profile = permission.grp_tree.id\r
+ INNER JOIN actor.org_unit\r
+ ON actor.org_unit.id = actor.usr.home_ou\r
+ WHERE actor.usr.deleted IS FALSE\r
+ GROUP BY permission.grp_tree.name, actor.org_unit.name\r
+ ORDER BY actor.org_unit.name, permission.grp_tree.name\r
+;\r
+\r
+ name | name | count\r
+-------+--------------------+-------\r
+ Users | Example Consortium | 1\r
+(1 row)\r
+</programlisting>\r
+ </para></formalpara>\r
+ </simplesect>\r
+ <simplesect id="_aliases">\r
+ <title>Aliases</title>\r
+ <simpara>So far we have been fully-qualifying all of our table names and column names to\r
+ prevent any confusion. This quickly gets tiring with lengthy qualified\r
+ table names like <literal>permission.grp_tree</literal>, so the SQL syntax enables us to assign\r
+ aliases to table names and column names. When you define an alias for a table\r
+ name, you can access its column throughout the rest of the statement by simply\r
+ appending the column name to the alias with a period; for example, if you assign\r
+ the alias <literal>au</literal> to the <literal>actor.usr</literal> table, you can access the <literal>actor.usr.id</literal>\r
+ column through the alias as <literal>au.id</literal>.</simpara>\r
+ <simpara>The formal syntax for declaring an alias for a column is to follow the column\r
+ name in the result columns clause with <literal>AS</literal> <emphasis>alias</emphasis>. To declare an alias for a table name,\r
+ follow the table name in the FROM clause (including any JOIN statements) with\r
+ <literal>AS</literal> <emphasis>alias</emphasis>. However, the <literal>AS</literal> keyword is optional for tables (and columns as\r
+ of PostgreSQL 8.4), and in practice most SQL statements leave it out. For\r
+ example, we can write the previous INNER JOIN statement example using aliases\r
+ instead of fully-qualified identifiers:</simpara>\r
+ <formalpara><title>Borrower Count by Profile (using aliases)</title><para>\r
+<programlisting language="sql" linenumbering="unnumbered">\r
+SELECT pgt.name AS "Profile", aou.name AS "Library", COUNT(pgt.name) AS "Count"\r
+ FROM actor.usr au\r
+ INNER JOIN permission.grp_tree pgt\r
+ ON au.profile = pgt.id\r
+ INNER JOIN actor.org_unit aou\r
+ ON aou.id = au.home_ou\r
+ WHERE au.deleted IS FALSE\r
+ GROUP BY pgt.name, aou.name\r
+ ORDER BY aou.name, pgt.name\r
+;\r
+\r
+ Profile | Library | Count\r
+---------+--------------------+-------\r
+ Users | Example Consortium | 1\r
+(1 row)\r
+</programlisting>\r
+ </para></formalpara>\r
+ <simpara>A nice side effect of declaring an alias for your columns is that the alias\r
+ is used as the column header in the results table. The previous version of\r
+ the query, which didn’t use aliased column names, had two columns named\r
+ <literal>name</literal>; this version of the query with aliases results in a clearer\r
+ categorization.</simpara>\r
+ </simplesect>\r
+ <simplesect id="_outer_joins">\r
+ <title>Outer joins</title>\r
+ <simpara>An outer join returns all of the rows from one or both of the tables\r
+ participating in the join.</simpara>\r
+ <itemizedlist>\r
+ <listitem>\r
+ <simpara>\r
+ For a LEFT OUTER JOIN, the join returns all of the rows from the left-hand\r
+ table and the rows matching the join condition from the right-hand table, with\r
+ NULL values for the rows with no match in the right-hand table.\r
+ </simpara>\r
+ </listitem>\r
+ <listitem>\r
+ <simpara>\r
+ A RIGHT OUTER JOIN behaves in the same way as a LEFT OUTER JOIN, with the\r
+ exception that all rows are returned from the right-hand table participating in\r
+ the join.\r
+ </simpara>\r
+ </listitem>\r
+ <listitem>\r
+ <simpara>\r
+ For a FULL OUTER JOIN, the join returns all the rows from both the left-hand\r
+ and right-hand tables, with NULL values for the rows with no match in either\r
+ the left-hand or right-hand table.\r
+ </simpara>\r
+ </listitem>\r
+ </itemizedlist>\r
+ <formalpara><title>Base tables for the OUTER JOIN examples</title><para>\r
+<programlisting language="sql" linenumbering="unnumbered">\r
+SELECT * FROM aaa;\r
+\r
+ id | stuff\r
+----+-------\r
+ 1 | one\r
+ 2 | two\r
+ 3 | three\r
+ 4 | four\r
+ 5 | five\r
+(5 rows)\r
+\r
+SELECT * FROM bbb;\r
+\r
+ id | stuff | foo\r
+----+-------+----------\r
+ 1 | one | oneone\r
+ 2 | two | twotwo\r
+ 5 | five | fivefive\r
+ 6 | six | sixsix\r
+(4 rows)\r
+</programlisting>\r
+ </para></formalpara>\r
+ <formalpara><title>Example of a LEFT OUTER JOIN</title><para>\r
+<programlisting language="sql" linenumbering="unnumbered">\r
+SELECT * FROM aaa\r
+ LEFT OUTER JOIN bbb ON aaa.id = bbb.id\r
+;\r
+ id | stuff | id | stuff | foo\r
+----+-------+----+-------+----------\r
+ 1 | one | 1 | one | oneone\r
+ 2 | two | 2 | two | twotwo\r
+ 3 | three | | |\r
+ 4 | four | | |\r
+ 5 | five | 5 | five | fivefive\r
+(5 rows)\r
+</programlisting>\r
+ </para></formalpara>\r
+ <formalpara><title>Example of a RIGHT OUTER JOIN</title><para>\r
+<programlisting language="sql" linenumbering="unnumbered">\r
+SELECT * FROM aaa\r
+ RIGHT OUTER JOIN bbb ON aaa.id = bbb.id\r
+;\r
+ id | stuff | id | stuff | foo\r
+----+-------+----+-------+----------\r
+ 1 | one | 1 | one | oneone\r
+ 2 | two | 2 | two | twotwo\r
+ 5 | five | 5 | five | fivefive\r
+ | | 6 | six | sixsix\r
+(4 rows)\r
+</programlisting>\r
+ </para></formalpara>\r
+ <formalpara><title>Example of a FULL OUTER JOIN</title><para>\r
+<programlisting language="sql" linenumbering="unnumbered">\r
+SELECT * FROM aaa\r
+ FULL OUTER JOIN bbb ON aaa.id = bbb.id\r
+;\r
+ id | stuff | id | stuff | foo\r
+----+-------+----+-------+----------\r
+ 1 | one | 1 | one | oneone\r
+ 2 | two | 2 | two | twotwo\r
+ 3 | three | | |\r
+ 4 | four | | |\r
+ 5 | five | 5 | five | fivefive\r
+ | | 6 | six | sixsix\r
+(6 rows)\r
+</programlisting>\r
+ </para></formalpara>\r
+ </simplesect>\r
+ <simplesect id="_self_joins">\r
+ <title>Self joins</title>\r
+ <simpara>It is possible to join a table to itself. You can, in fact you must, use\r
+ aliases to disambiguate the references to the table.</simpara>\r
+ </simplesect>\r
+ </simplesect>\r
+ <simplesect id="_set_operations">\r
+ <title>Set operations</title>\r
+ <simpara>Relational databases are effectively just an efficient mechanism for\r
+ manipulating sets of values; they are implementations of set theory. There are\r
+ three operators for sets (tables) in which each set must have the same number\r
+ of columns with compatible data types: the union, intersection, and difference\r
+ operators.</simpara>\r
+ <formalpara><title>Base tables for the set operation examples</title><para>\r
+<programlisting language="sql" linenumbering="unnumbered">\r
+SELECT * FROM aaa;\r
+\r
+ id | stuff\r
+ ----+-------\r
+ 1 | one\r
+ 2 | two\r
+ 3 | three\r
+ 4 | four\r
+ 5 | five\r
+ (5 rows)\r
+\r
+SELECT * FROM bbb;\r
+\r
+ id | stuff | foo\r
+ ----+-------+----------\r
+ 1 | one | oneone\r
+ 2 | two | twotwo\r
+ 5 | five | fivefive\r
+ 6 | six | sixsix\r
+(4 rows)\r
+</programlisting>\r
+ </para></formalpara>\r
+ <simplesect id="_union">\r
+ <title>Union</title>\r
+ <simpara>The <literal>UNION</literal> operator returns the distinct set of rows that are members of\r
+ either or both of the left-hand and right-hand tables. The <literal>UNION</literal> operator\r
+ does not return any duplicate rows. To return duplicate rows, use the\r
+ <literal>UNION ALL</literal> operator.</simpara>\r
+ <formalpara><title>Example of a UNION set operation</title><para>\r
+<programlisting language="sql" linenumbering="unnumbered">\r
+-- The parentheses are not required, but are intended to help\r
+-- illustrate the sets participating in the set operation\r
+(\r
+ SELECT id, stuff\r
+ FROM aaa\r
+)\r
+UNION\r
+(\r
+ SELECT id, stuff\r
+ FROM bbb\r
+)\r
+ORDER BY 1\r
+;\r
+\r
+ id | stuff\r
+----+-------\r
+ 1 | one\r
+ 2 | two\r
+ 3 | three\r
+ 4 | four\r
+ 5 | five\r
+ 6 | six\r
+(6 rows)\r
+</programlisting>\r
+ </para></formalpara>\r
+ </simplesect>\r
+ <simplesect id="_intersection">\r
+ <title>Intersection</title>\r
+ <simpara>The <literal>INTERSECT</literal> operator returns the distinct set of rows that are common to\r
+ both the left-hand and right-hand tables. To return duplicate rows, use the\r
+ <literal>INTERSECT ALL</literal> operator.</simpara>\r
+ <formalpara><title>Example of an INTERSECT set operation</title><para>\r
+<programlisting language="sql" linenumbering="unnumbered">\r
+(\r
+ SELECT id, stuff\r
+ FROM aaa\r
+)\r
+INTERSECT\r
+(\r
+ SELECT id, stuff\r
+ FROM bbb\r
+)\r
+ORDER BY 1\r
+;\r
+\r
+ id | stuff\r
+----+-------\r
+ 1 | one\r
+ 2 | two\r
+ 5 | five\r
+(3 rows)\r
+</programlisting>\r
+ </para></formalpara>\r
+ </simplesect>\r
+ <simplesect id="_difference">\r
+ <title>Difference</title>\r
+ <simpara>The <literal>EXCEPT</literal> operator returns the rows in the left-hand table that do not\r
+ exist in the right-hand table. You are effectively subtracting the common\r
+ rows from the left-hand table.</simpara>\r
+ <formalpara><title>Example of an EXCEPT set operation</title><para>\r
+<programlisting language="sql" linenumbering="unnumbered">\r
+(\r
+ SELECT id, stuff\r
+ FROM aaa\r
+)\r
+EXCEPT\r
+(\r
+ SELECT id, stuff\r
+ FROM bbb\r
+)\r
+ORDER BY 1\r
+;\r
+\r
+ id | stuff\r
+----+-------\r
+ 3 | three\r
+ 4 | four\r
+(2 rows)\r
+\r
+-- Order matters: switch the left-hand and right-hand tables\r
+-- and you get a different result\r
+(\r
+ SELECT id, stuff\r
+ FROM bbb\r
+)\r
+EXCEPT\r
+(\r
+ SELECT id, stuff\r
+ FROM aaa\r
+)\r
+ORDER BY 1\r
+;\r
+\r
+ id | stuff\r
+----+-------\r
+ 6 | six\r
+(1 row)\r
+</programlisting>\r
+ </para></formalpara>\r
+ </simplesect>\r
+ </simplesect>\r
+ <simplesect id="_views">\r
+ <title>Views</title>\r
+ <simpara>A view is a persistent <literal>SELECT</literal> statement that acts like a read-only table.\r
+ To create a view, issue the <literal>CREATE VIEW</literal> statement, giving the view a name\r
+ and a <literal>SELECT</literal> statement on which the view is built.</simpara>\r
+ <simpara>The following example creates a view based on our borrower profile count:</simpara>\r
+ <formalpara><title>Creating a view</title><para>\r
+<programlisting language="sql" linenumbering="unnumbered">\r
+CREATE VIEW actor.borrower_profile_count AS\r
+ SELECT pgt.name AS "Profile", aou.name AS "Library", COUNT(pgt.name) AS "Count"\r
+ FROM actor.usr au\r
+ INNER JOIN permission.grp_tree pgt\r
+ ON au.profile = pgt.id\r
+ INNER JOIN actor.org_unit aou\r
+ ON aou.id = au.home_ou\r
+ WHERE au.deleted IS FALSE\r
+ GROUP BY pgt.name, aou.name\r
+ ORDER BY aou.name, pgt.name\r
+;\r
+</programlisting>\r
+ </para></formalpara>\r
+ <simpara>When you subsequently select results from the view, you can apply additional\r
+ <literal>WHERE</literal> clauses to filter the results, or <literal>ORDER BY</literal> clauses to change the\r
+ order of the returned rows. In the following examples, we issue a simple\r
+ <literal>SELECT *</literal> statement to show that the default results are returned in the\r
+ same order from the view as the equivalent SELECT statement would be returned.\r
+ Then we issue a <literal>SELECT</literal> statement with a <literal>WHERE</literal> clause to further filter the\r
+ results.</simpara>\r
+ <formalpara><title>Selecting results from a view</title><para>\r
+<programlisting language="sql" linenumbering="unnumbered">\r
+SELECT * FROM actor.borrower_profile_count;\r
+\r
+ Profile | Library | Count\r
+----------------------------+----------------------------+-------\r
+ Faculty | University Library | 208\r
+ Graduate | University Library | 16\r
+ Patrons | University Library | 62\r
+...\r
+\r
+-- You can still filter your results with WHERE clauses\r
+SELECT *\r
+ FROM actor.borrower_profile_count\r
+ WHERE "Profile" = 'Faculty';\r
+\r
+ Profile | Library | Count\r
+---------+----------------------------+-------\r
+ Faculty | University Library | 208\r
+ Faculty | College Library | 64\r
+ Faculty | College Library 2 | 102\r
+ Faculty | University Library 2 | 776\r
+(4 rows)\r
+</programlisting>\r
+ </para></formalpara>\r
+ </simplesect>\r
+ <simplesect id="_inheritance">\r
+ <title>Inheritance</title>\r
+ <simpara>PostgreSQL supports table inheritance: that is, a child table inherits its\r
+ base definition from a parent table, but can add additional columns to its\r
+ own definition. The data from any child tables is visible in queries against\r
+ the parent table.</simpara>\r
+ <simpara>Evergreen uses table inheritance in several areas:\r
+ * In the Vandelay MARC batch importer / exporter, Evergreen defines base\r
+ tables for generic queues and queued records for which authority record and\r
+ bibliographic record child tables\r
+ * Billable transactions are based on the <literal>money.billable_xact</literal> table;\r
+ child tables include <literal>action.circulation</literal> for circulation transactions\r
+ and <literal>money.grocery</literal> for general bills.\r
+ * Payments are based on the <literal>money.payment</literal> table; its child table is\r
+ <literal>money.bnm_payment</literal> (for brick-and-mortar payments), which in turn has child\r
+ tables of <literal>money.forgive_payment</literal>, <literal>money.work_payment</literal>, <literal>money.credit_payment</literal>,\r
+ <literal>money.goods_payment</literal>, and <literal>money.bnm_desk_payment</literal>. The\r
+ <literal>money.bnm_desk_payment</literal> table in turn has child tables of <literal>money.cash_payment</literal>,\r
+ <literal>money.check_payment</literal>, and <literal>money.credit_card_payment</literal>.\r
+ * Transits are based on the <literal>action.transit_copy</literal> table, which has a child\r
+ table of <literal>action.hold_transit_copy</literal> for transits initiated by holds.\r
+ * Generic acquisition line items are defined by the\r
+ <literal>acq.lineitem_attr_definition</literal> table, which in turn has a number of child\r
+ tables to define MARC attributes, generated attributes, user attributes, and\r
+ provider attributes.</simpara>\r
+ </simplesect>\r
+ </section>\r
+ <section id="understanding_query_performance_with_explain">\r
+ <title>Understanding query performance with EXPLAIN</title>\r
+ <simpara>Some queries run for a long, long time. This can be the result of a poorly\r
+ written query—a query with a join condition that joins every\r
+ row in the <literal>biblio.record_entry</literal> table with every row in the <literal>metabib.full_rec</literal>\r
+ view would consume a massive amount of memory and disk space and CPU time—or\r
+ a symptom of a schema that needs some additional indexes. PostgreSQL provides\r
+ the <literal>EXPLAIN</literal> tool to estimate how long it will take to run a given query and\r
+ show you the <emphasis>query plan</emphasis> (how it plans to retrieve the results from the\r
+ database).</simpara>\r
+ <simpara>To generate the query plan without actually running the statement, simply\r
+ prepend the <literal>EXPLAIN</literal> keyword to your query. In the following example, we\r
+ generate the query plan for the poorly written query that would join every\r
+ row in the <literal>biblio.record_entry</literal> table with every row in the <literal>metabib.full_rec</literal>\r
+ view:</simpara>\r
+ <formalpara><title>Query plan for a terrible query</title><para>\r
+<programlisting language="sql" linenumbering="unnumbered">\r
+EXPLAIN SELECT *\r
+ FROM biblio.record_entry\r
+ FULL OUTER JOIN metabib.full_rec ON 1=1\r
+;\r
+\r
+ QUERY PLAN\r
+-------------------------------------------------------------------------------//\r
+ Merge Full Join (cost=0.00..4959156437783.60 rows=132415734100864 width=1379)\r
+ -> Seq Scan on record_entry (cost=0.00..400634.16 rows=2013416 width=1292)\r
+ -> Seq Scan on real_full_rec (cost=0.00..1640972.04 rows=65766704 width=87)\r
+(3 rows)\r
+</programlisting>\r
+ </para></formalpara>\r
+ <simpara>This query plan shows that the query would return 132415734100864 rows, and it\r
+ plans to accomplish what you asked for by sequentially scanning (<emphasis>Seq Scan</emphasis>)\r
+ every row in each of the tables participating in the join.</simpara>\r
+ <simpara>In the following example, we have realized our mistake in joining every row of\r
+ the left-hand table with every row in the right-hand table and take the saner\r
+ approach of using an <literal>INNER JOIN</literal> where the join condition is on the record ID.</simpara>\r
+ <formalpara><title>Query plan for a less terrible query</title><para>\r
+<programlisting language="sql" linenumbering="unnumbered">\r
+EXPLAIN SELECT *\r
+ FROM biblio.record_entry bre\r
+ INNER JOIN metabib.full_rec mfr ON mfr.record = bre.id;\r
+ QUERY PLAN\r
+----------------------------------------------------------------------------------------//\r
+ Hash Join (cost=750229.86..5829273.98 rows=65766704 width=1379)\r
+ Hash Cond: (real_full_rec.record = bre.id)\r
+ -> Seq Scan on real_full_rec (cost=0.00..1640972.04 rows=65766704 width=87)\r
+ -> Hash (cost=400634.16..400634.16 rows=2013416 width=1292)\r
+ -> Seq Scan on record_entry bre (cost=0.00..400634.16 rows=2013416 width=1292)\r
+(5 rows)\r
+</programlisting>\r
+ </para></formalpara>\r
+ <simpara>This time, we will return 65766704 rows - still way too many rows. We forgot\r
+ to include a <literal>WHERE</literal> clause to limit the results to something meaningful. In\r
+ the following example, we will limit the results to deleted records that were\r
+ modified in the last month.</simpara>\r
+ <formalpara><title>Query plan for a realistic query</title><para>\r
+<programlisting language="sql" linenumbering="unnumbered">\r
+EXPLAIN SELECT *\r
+ FROM biblio.record_entry bre\r
+ INNER JOIN metabib.full_rec mfr ON mfr.record = bre.id\r
+ WHERE bre.deleted IS TRUE\r
+ AND DATE_TRUNC('MONTH', bre.edit_date) >\r
+ DATE_TRUNC ('MONTH', NOW() - '1 MONTH'::INTERVAL)\r
+;\r
+\r
+ QUERY PLAN\r
+----------------------------------------------------------------------------------------//\r
+ Hash Join (cost=5058.86..2306218.81 rows=201669 width=1379)\r
+ Hash Cond: (real_full_rec.record = bre.id)\r
+ -> Seq Scan on real_full_rec (cost=0.00..1640972.04 rows=65766704 width=87)\r
+ -> Hash (cost=4981.69..4981.69 rows=6174 width=1292)\r
+ -> Index Scan using biblio_record_entry_deleted on record_entry bre\r
+ (cost=0.00..4981.69 rows=6174 width=1292)\r
+ Index Cond: (deleted = true)\r
+ Filter: ((deleted IS TRUE) AND (date_trunc('MONTH'::text, edit_date)\r
+ > date_trunc('MONTH'::text, (now() - '1 mon'::interval))))\r
+(7 rows)\r
+</programlisting>\r
+ </para></formalpara>\r
+ <simpara>We can see that the number of rows returned is now only 201669; that’s\r
+ something we can work with. Also, the overall cost of the query is 2306218,\r
+ compared to 4959156437783 in the original query. The <literal>Index Scan</literal> tells us\r
+ that the query planner will use the index that was defined on the <literal>deleted</literal>\r
+ column to avoid having to check every row in the <literal>biblio.record_entry</literal> table.</simpara>\r
+ <simpara>However, we are still running a sequential scan over the\r
+ <literal>metabib.real_full_rec</literal> table (the table on which the <literal>metabib.full_rec</literal>\r
+ view is based). Given that linking from the bibliographic records to the\r
+ flattened MARC subfields is a fairly common operation, we could create a\r
+ new index and see if that speeds up our query plan.</simpara>\r
+ <formalpara><title>Query plan with optimized access via a new index</title><para>\r
+<programlisting language="sql" linenumbering="unnumbered">\r
+-- This index will take a long time to create on a large database\r
+-- of bibliographic records\r
+CREATE INDEX bib_record_idx ON metabib.real_full_rec (record);\r
+\r
+EXPLAIN SELECT *\r
+ FROM biblio.record_entry bre\r
+ INNER JOIN metabib.full_rec mfr ON mfr.record = bre.id\r
+ WHERE bre.deleted IS TRUE\r
+ AND DATE_TRUNC('MONTH', bre.edit_date) >\r
+ DATE_TRUNC ('MONTH', NOW() - '1 MONTH'::INTERVAL)\r
+;\r
+\r
+ QUERY PLAN\r
+----------------------------------------------------------------------------------------//\r
+ Nested Loop (cost=0.00..1558330.46 rows=201669 width=1379)\r
+ -> Index Scan using biblio_record_entry_deleted on record_entry bre\r
+ (cost=0.00..4981.69 rows=6174 width=1292)\r
+ Index Cond: (deleted = true)\r
+ Filter: ((deleted IS TRUE) AND (date_trunc('MONTH'::text, edit_date) >\r
+ date_trunc('MONTH'::text, (now() - '1 mon'::interval))))\r
+ -> Index Scan using bib_record_idx on real_full_rec\r
+ (cost=0.00..240.89 rows=850 width=87)\r
+ Index Cond: (real_full_rec.record = bre.id)\r
+(6 rows)\r
+</programlisting>\r
+ </para></formalpara>\r
+ <simpara>We can see that the resulting number of rows is still the same (201669), but\r
+ the execution estimate has dropped to 1558330 because the query planner can\r
+ use the new index (<literal>bib_record_idx</literal>) rather than scanning the entire table.\r
+ Success!</simpara>\r
+ <note><simpara>While indexes can significantly speed up read access to tables for common\r
+ filtering conditions, every time a row is created or updated the corresponding\r
+ indexes also need to be maintained - which can decrease the performance of\r
+ writes to the database. Be careful to keep the balance of read performance\r
+ versus write performance in mind if you plan to create custom indexes in your\r
+ Evergreen database.</simpara></note>\r
+ </section>\r
+ <section id="inserting_updating_and_deleting_data">\r
+ <title>Inserting, updating, and deleting data</title>\r
+ <simplesect id="_inserting_data">\r
+ <title>Inserting data</title>\r
+ <simpara>To insert one or more rows into a table, use the INSERT statement to identify\r
+ the target table and list the columns in the table for which you are going to\r
+ provide values for each row. If you do not list one or more columns contained\r
+ in the table, the database will automatically supply a <literal>NULL</literal> value for those\r
+ columns. The values for each row follow the <literal>VALUES</literal> clause and are grouped in\r
+ parentheses and delimited by commas. Each row, in turn, is delimited by commas\r
+ (<emphasis>this multiple row syntax requires PostgreSQL 8.2 or higher</emphasis>).</simpara>\r
+ <simpara>For example, to insert two rows into the <literal>permission.usr_grp_map</literal> table:</simpara>\r
+ <formalpara><title>Inserting rows into the <literal>permission.usr_grp_map</literal> table</title><para>\r
+ <programlisting language="sql" linenumbering="unnumbered">INSERT INTO permission.usr_grp_map (usr, grp)\r
+ VALUES (2, 10), (2, 4)\r
+ ;</programlisting>\r
+ </para></formalpara>\r
+ <simpara>Of course, as with the rest of SQL, you can replace individual column values\r
+ with one or more use sub-selects:</simpara>\r
+ <formalpara><title>Inserting rows using sub-selects instead of integers</title><para>\r
+<programlisting language="sql" linenumbering="unnumbered">\r
+INSERT INTO permission.usr_grp_map (usr, grp)\r
+ VALUES (\r
+ (SELECT id FROM actor.usr\r
+ WHERE family_name = 'Scott' AND first_given_name = 'Daniel'),\r
+ (SELECT id FROM permission.grp_tree\r
+ WHERE name = 'Local System Administrator')\r
+ ), (\r
+ (SELECT id FROM actor.usr\r
+ WHERE family_name = 'Scott' AND first_given_name = 'Daniel'),\r
+ (SELECT id FROM permission.grp_tree\r
+ WHERE name = 'Circulator')\r
+ )\r
+;\r
+</programlisting>\r
+ </para></formalpara>\r
+ </simplesect>\r
+ <simplesect id="_inserting_data_using_a_select_statement">\r
+ <title>Inserting data using a SELECT statement</title>\r
+ <simpara>Sometimes you want to insert a bulk set of data into a new table based on\r
+ a query result. Rather than a <literal>VALUES</literal> clause, you can use a <literal>SELECT</literal>\r
+ statement to insert one or more rows matching the column definitions. This\r
+ is a good time to point out that you can include explicit values, instead\r
+ of just column identifiers, in the return columns of the <literal>SELECT</literal> statement.\r
+ The explicit values are returned in every row of the result set.</simpara>\r
+ <simpara>In the following example, we insert 6 rows into the <literal>permission.usr_grp_map</literal>\r
+ table; each row will have a <literal>usr</literal> column value of 1, with varying values for\r
+ the <literal>grp</literal> column value based on the <literal>id</literal> column values returned from\r
+ <literal>permission.grp_tree</literal>:</simpara>\r
+ <formalpara><title>Inserting rows via a <literal>SELECT</literal> statement</title><para>\r
+<programlisting language="sql" linenumbering="unnumbered">\r
+INSERT INTO permission.usr_grp_map (usr, grp)\r
+ SELECT 1, id\r
+ FROM permission.grp_tree\r
+ WHERE id > 2\r
+;\r
+\r
+INSERT 0 6\r
+</programlisting>\r
+ </para></formalpara>\r
+ </simplesect>\r
+ <simplesect id="_deleting_rows">\r
+ <title>Deleting rows</title>\r
+ <simpara>Deleting data from a table is normally fairly easy. To delete rows from a table,\r
+ issue a <literal>DELETE</literal> statement identifying the table from which you want to delete\r
+ rows and a <literal>WHERE</literal> clause identifying the row or rows that should be deleted.</simpara>\r
+ <simpara>In the following example, we delete all of the rows from the\r
+ <literal>permission.grp_perm_map</literal> table where the permission maps to\r
+ <literal>UPDATE_ORG_UNIT_CLOSING</literal> and the group is anything other than administrators:</simpara>\r
+ <formalpara><title>Deleting rows from a table</title><para>\r
+<programlisting language="sql" linenumbering="unnumbered">\r
+DELETE FROM permission.grp_perm_map\r
+ WHERE grp IN (\r
+ SELECT id\r
+ FROM permission.grp_tree\r
+ WHERE name != 'Local System Administrator'\r
+ ) AND perm = (\r
+ SELECT id\r
+ FROM permission.perm_list\r
+ WHERE code = 'UPDATE_ORG_UNIT_CLOSING'\r
+ )\r
+;\r
+</programlisting>\r
+ </para></formalpara>\r
+ <note><simpara>There are two main reasons that a <literal>DELETE</literal> statement may not actually\r
+ delete rows from a table, even when the rows meet the conditional clause.</simpara></note>\r
+ <orderedlist numeration="arabic">\r
+ <listitem>\r
+ <simpara>\r
+ If the row contains a value that is the target of a relational constraint,\r
+ for example, if another table has a foreign key pointing at your target\r
+ table, you will be prevented from deleting a row with a value corresponding\r
+ to a row in the dependent table.\r
+ </simpara>\r
+ </listitem>\r
+ <listitem>\r
+ <simpara>\r
+ If the table has a rule that substitutes a different action for a <literal>DELETE</literal>\r
+ statement, the deletion will not take place. In Evergreen it is common for a\r
+ table to have a rule that substitutes the action of setting a <literal>deleted</literal> column\r
+ to <literal>TRUE</literal>. For example, if a book is discarded, deleting the row representing\r
+ the copy from the <literal>asset.copy</literal> table would severely affect circulation statistics,\r
+ bills, borrowing histories, and their corresponding tables in the database that\r
+ have foreign keys pointing at the <literal>asset.copy</literal> table (<literal>action.circulation</literal> and\r
+ <literal>money.billing</literal> and its children respectively). Instead, the <literal>deleted</literal> column\r
+ value is set to <literal>TRUE</literal> and Evergreen’s application logic skips over these rows\r
+ in most cases.\r
+ </simpara>\r
+ </listitem>\r
+ </orderedlist>\r
+ </simplesect>\r
+ <simplesect id="_updating_rows">\r
+ <title>Updating rows</title>\r
+ <simpara>To update rows in a table, issue an <literal>UPDATE</literal> statement identifying the table\r
+ you want to update, the column or columns that you want to set with their\r
+ respective new values, and (optionally) a <literal>WHERE</literal> clause identifying the row or\r
+ rows that should be updated.</simpara>\r
+ <simpara>Following is the syntax for the <literal>UPDATE</literal> statement:</simpara>\r
+ <blockquote>\r
+ <literallayout><literal>UPDATE</literal> [<emphasis>table-name</emphasis>]\r
+ <literal>SET</literal> [<emphasis>column</emphasis>] <literal>TO</literal> [<emphasis>new-value</emphasis>]\r
+ <literal>WHERE</literal> [<emphasis>condition</emphasis>]\r
+ ;</literallayout>\r
+ </blockquote>\r
+ </simplesect>\r
+ </section>\r
+ <section id="query_requests">\r
+ <title>Query requests</title>\r
+ <simpara>The following queries were requested by Bibliomation, but might be reusable\r
+ by other libraries.</simpara>\r
+ <simplesect id="_monthly_circulation_stats_by_collection_code_library">\r
+ <title>Monthly circulation stats by collection code / library</title>\r
+ <formalpara><title>Monthly Circulation Stats by Collection Code/Library</title><para>\r
+<programlisting language="sql" linenumbering="unnumbered">\r
+SELECT COUNT(acirc.id) AS "COUNT", aou.name AS "Library", acl.name AS "Copy Location"\r
+ FROM asset.copy ac\r
+ INNER JOIN asset.copy_location acl ON ac.location = acl.id\r
+ INNER JOIN action.circulation acirc ON acirc.target_copy = ac.id\r
+ INNER JOIN actor.org_unit aou ON acirc.circ_lib = aou.id\r
+ WHERE DATE_TRUNC('MONTH', acirc.create_time) = DATE_TRUNC('MONTH', NOW() - INTERVAL '3 month')\r
+ AND acirc.desk_renewal IS FALSE\r
+ AND acirc.opac_renewal IS FALSE\r
+ AND acirc.phone_renewal IS FALSE\r
+ GROUP BY aou.name, acl.name\r
+ ORDER BY aou.name, acl.name, 1\r
+;\r
+</programlisting>\r
+ </para></formalpara>\r
+ </simplesect>\r
+ <simplesect id="_monthly_circulation_stats_by_borrower_stat_library">\r
+ <title>Monthly circulation stats by borrower stat / library</title>\r
+ <formalpara><title>Monthly Circulation Stats by Borrower Stat/Library</title><para>\r
+<programlisting language="sql" linenumbering="unnumbered">\r
+SELECT COUNT(acirc.id) AS "COUNT", aou.name AS "Library", asceum.stat_cat_entry AS "Borrower Stat"\r
+ FROM action.circulation acirc\r
+ INNER JOIN actor.org_unit aou ON acirc.circ_lib = aou.id\r
+ INNER JOIN actor.stat_cat_entry_usr_map asceum ON asceum.target_usr = acirc.usr\r
+ INNER JOIN actor.stat_cat astat ON asceum.stat_cat = astat.id\r
+ WHERE DATE_TRUNC('MONTH', acirc.create_time) = DATE_TRUNC('MONTH', NOW() - INTERVAL '3 month')\r
+ AND astat.name = 'Preferred language'\r
+ AND acirc.desk_renewal IS FALSE\r
+ AND acirc.opac_renewal IS FALSE\r
+ AND acirc.phone_renewal IS FALSE\r
+ GROUP BY aou.name, asceum.stat_cat_entry\r
+ ORDER BY aou.name, asceum.stat_cat_entry, 1\r
+;\r
+</programlisting>\r
+ </para></formalpara>\r
+ </simplesect>\r
+ <simplesect id="_monthly_intralibrary_loan_stats_by_library">\r
+ <title>Monthly intralibrary loan stats by library</title>\r
+ <formalpara><title>Monthly Intralibrary Loan Stats by Library</title><para>\r
+<programlisting language="sql" linenumbering="unnumbered">\r
+SELECT aou.name AS "Library", COUNT(acirc.id)\r
+ FROM action.circulation acirc\r
+ INNER JOIN actor.org_unit aou ON acirc.circ_lib = aou.id\r
+ INNER JOIN asset.copy ac ON acirc.target_copy = ac.id\r
+ INNER JOIN asset.call_number acn ON ac.call_number = acn.id\r
+ WHERE acirc.circ_lib != acn.owning_lib\r
+ AND DATE_TRUNC('MONTH', acirc.create_time) = DATE_TRUNC('MONTH', NOW() - INTERVAL '3 month')\r
+ AND acirc.desk_renewal IS FALSE\r
+ AND acirc.opac_renewal IS FALSE\r
+ AND acirc.phone_renewal IS FALSE\r
+ GROUP by aou.name\r
+ ORDER BY aou.name, 2\r
+;\r
+</programlisting>\r
+ </para></formalpara>\r
+ </simplesect>\r
+ <simplesect id="_monthly_borrowers_added_by_profile_adult_child_etc_library">\r
+ <title>Monthly borrowers added by profile (adult, child, etc) / library</title>\r
+ <formalpara><title>Monthly Borrowers Added by Profile (Adult, Child, etc)/Library</title><para>\r
+<programlisting language="sql" linenumbering="unnumbered">\r
+SELECT pgt.name AS "Profile", aou.name AS "Library", COUNT(pgt.name) AS "Count"\r
+ FROM actor.usr au\r
+ INNER JOIN permission.grp_tree pgt\r
+ ON au.profile = pgt.id\r
+ INNER JOIN actor.org_unit aou\r
+ ON aou.id = au.home_ou\r
+ WHERE au.deleted IS FALSE\r
+ AND DATE_TRUNC('MONTH', au.create_date) = DATE_TRUNC('MONTH', NOW() - '3 months'::interval)\r
+ GROUP BY pgt.name, aou.name\r
+ ORDER BY aou.name, pgt.name\r
+;\r
+</programlisting>\r
+ </para></formalpara>\r
+ </simplesect>\r
+ <simplesect id="_borrower_count_by_profile_adult_child_etc_library">\r
+ <title>Borrower count by profile (adult, child, etc) / library</title>\r
+ <formalpara><title>Borrower Count by Profile (Adult, Child, etc)/Library</title><para>\r
+<programlisting language="sql" linenumbering="unnumbered">\r
+SELECT pgt.name AS "Profile", aou.name AS "Library", COUNT(pgt.name) AS "Count"\r
+ FROM actor.usr au\r
+ INNER JOIN permission.grp_tree pgt\r
+ ON au.profile = pgt.id\r
+ INNER JOIN actor.org_unit aou\r
+ ON aou.id = au.home_ou\r
+ WHERE au.deleted IS FALSE\r
+ GROUP BY pgt.name, aou.name\r
+ ORDER BY aou.name, pgt.name\r
+;\r
+</programlisting>\r
+ </para></formalpara>\r
+ </simplesect>\r
+ <simplesect id="_monthly_items_added_by_collection_library">\r
+ <title>Monthly items added by collection / library</title>\r
+ <simpara>We define a <quote>collection</quote> as a shelving location in Evergreen.</simpara>\r
+ <formalpara><title>Monthly Items Added by Collection/Library</title><para>\r
+<programlisting language="sql" linenumbering="unnumbered">\r
+SELECT aou.name AS "Library", acl.name, COUNT(ac.barcode)\r
+ FROM actor.org_unit aou\r
+ INNER JOIN asset.call_number acn ON acn.owning_lib = aou.id\r
+ INNER JOIN asset.copy ac ON ac.call_number = acn.id\r
+ INNER JOIN asset.copy_location acl ON ac.location = acl.id\r
+ WHERE ac.deleted IS FALSE\r
+ AND acn.deleted IS FALSE\r
+ AND DATE_TRUNC('MONTH', ac.create_date) = DATE_TRUNC('MONTH', NOW() - '1 month'::interval)\r
+ GROUP BY aou.name, acl.name\r
+ ORDER BY aou.name, acl.name\r
+;\r
+</programlisting>\r
+ </para></formalpara>\r
+ </simplesect>\r
+ <simplesect id="_hold_purchase_alert_by_library">\r
+ <title>Hold purchase alert by library</title>\r
+ <simpara>in the following set of queries, we bring together the active title, volume,\r
+ and copy holds and display those that have more than a certain number of holds\r
+ per title. The goal is to UNION ALL the three queries, then group by the\r
+ bibliographic record ID and display the title / author information for those\r
+ records that have more than a given threshold of holds.</simpara>\r
+ <formalpara><title>Hold Purchase Alert by Library</title><para>\r
+<programlisting language="sql" linenumbering="unnumbered">\r
+-- Title holds\r
+SELECT all_holds.bib_id, aou.name, rmsr.title, rmsr.author, COUNT(all_holds.bib_id)\r
+ FROM\r
+ (\r
+ (\r
+ SELECT target, request_lib\r
+ FROM action.hold_request\r
+ WHERE hold_type = 'T'\r
+ AND fulfillment_time IS NULL\r
+ AND cancel_time IS NULL\r
+ )\r
+ UNION ALL\r
+ -- Volume holds\r
+ (\r
+ SELECT bre.id, request_lib\r
+ FROM action.hold_request ahr\r
+ INNER JOIN asset.call_number acn ON ahr.target = acn.id\r
+ INNER JOIN biblio.record_entry bre ON acn.record = bre.id\r
+ WHERE ahr.hold_type = 'V'\r
+ AND ahr.fulfillment_time IS NULL\r
+ AND ahr.cancel_time IS NULL\r
+ )\r
+ UNION ALL\r
+ -- Copy holds\r
+ (\r
+ SELECT bre.id, request_lib\r
+ FROM action.hold_request ahr\r
+ INNER JOIN asset.copy ac ON ahr.target = ac.id\r
+ INNER JOIN asset.call_number acn ON ac.call_number = acn.id\r
+ INNER JOIN biblio.record_entry bre ON acn.record = bre.id\r
+ WHERE ahr.hold_type = 'C'\r
+ AND ahr.fulfillment_time IS NULL\r
+ AND ahr.cancel_time IS NULL\r
+ )\r
+ ) AS all_holds(bib_id, request_lib)\r
+ INNER JOIN reporter.materialized_simple_record rmsr\r
+ INNER JOIN actor.org_unit aou ON aou.id = all_holds.request_lib\r
+ ON rmsr.id = all_holds.bib_id\r
+ GROUP BY all_holds.bib_id, aou.name, rmsr.id, rmsr.title, rmsr.author\r
+ HAVING COUNT(all_holds.bib_id) > 2\r
+ ORDER BY aou.name\r
+;\r
+</programlisting>\r
+ </para></formalpara>\r
+ </simplesect>\r
+ <simplesect id="_update_borrower_records_with_a_different_home_library">\r
+ <title>Update borrower records with a different home library</title>\r
+ <simpara>In this example, the library has opened a new branch in a growing area,\r
+ and wants to reassign the home library for the patrons in the vicinity of\r
+ the new branch to the new branch. To accomplish this, we create a staging table\r
+ that holds a set of city names and the corresponding branch shortname for the home\r
+ library for each city.</simpara>\r
+ <simpara>Then we issue an <literal>UPDATE</literal> statement to set the home library for patrons with a\r
+ physical address with a city that matches the city names in our staging table.</simpara>\r
+ <formalpara><title>Update borrower records with a different home library</title><para>\r
+<programlisting language="sql" linenumbering="unnumbered">\r
+CREATE SCHEMA staging;\r
+CREATE TABLE staging.city_home_ou_map (city TEXT, ou_shortname TEXT,\r
+ FOREIGN KEY (ou_shortname) REFERENCES actor.org_unit (shortname));\r
+INSERT INTO staging.city_home_ou_map (city, ou_shortname)\r
+ VALUES ('Southbury', 'BR1'), ('Middlebury', 'BR2'), ('Hartford', 'BR3');\r
+BEGIN;\r
+\r
+UPDATE actor.usr au SET home_ou = COALESCE(\r
+ (\r
+ SELECT aou.id\r
+ FROM actor.org_unit aou\r
+ INNER JOIN staging.city_home_ou_map schom ON schom.ou_shortname = aou.shortname\r
+ INNER JOIN actor.usr_address aua ON aua.city = schom.city\r
+ WHERE au.id = aua.usr\r
+ GROUP BY aou.id\r
+ ), home_ou)\r
+WHERE (\r
+ SELECT aou.id\r
+ FROM actor.org_unit aou\r
+ INNER JOIN staging.city_home_ou_map schom ON schom.ou_shortname = aou.shortname\r
+ INNER JOIN actor.usr_address aua ON aua.city = schom.city\r
+ WHERE au.id = aua.usr\r
+ GROUP BY aou.id\r
+) IS NOT NULL;\r
+</programlisting>\r
+ </para></formalpara>\r
+ </simplesect>\r
+ </section>\r
+ \r
+</chapter>\r
--- /dev/null
+<?xml version="1.0" encoding="utf-8"?>\r
+<chapter xml:id="JSON_Queries" xmlns="http://docbook.org/ns/docbook" version="5.0" xml:lang="EN"\r
+ xmlns:xi="http://www.w3.org/2001/XInclude" xmlns:xlink="http://www.w3.org/1999/xlink">\r
+ <info>\r
+ <title>JSON Queries</title>\r
+ <indexterm><primary>JSON</primary></indexterm>\r
+ </info>\r
+ <para>The json_query facility provides a way for client applications to query the database over the network. Instead of constructing its own SQL, the application encodes a query in the \r
+ form of a JSON string and passes it to the json_query service. Then the json_query service parses the JSON, constructs and executes the corresponding SQL, and returns the results to \r
+ the client application.</para>\r
+ <para>This arrangement enables the json_query service to act as a gatekeeper, protecting the database from potentially damaging SQL commands. In particular, the generated SQL is \r
+ confined to SELECT statements, which will not change the contents of the database.</para>\r
+\r
+ <para>In addition, the json_query service sometimes uses its knowledge of the database structure to supply column names and join conditions so that the client application doesn't \r
+ have to.</para>\r
+\r
+ <para>Nevertheless, the need to encode a query in a JSON string adds complications, because the client needs to know how to build the right JSON. JSON queries are also somewhat \r
+ limiting -- they can't do all of the things that you can do with raw SQL.</para>\r
+ <simplesect>\r
+ <title>The IDL</title>\r
+\r
+ <para>A JSON query does not refer to tables and columns. Instead, it refers to classes and fields, which the IDL maps to the corresponding database entities.</para>\r
+\r
+ <para>The IDL (Interface Definition Language) is an XML file, typically <filename>/openils/conf/fm_IDL.xml</filename>. It maps each class to a table, view, or subquery, and \r
+ each field to a column. It also includes information about foreign key relationships.</para>\r
+\r
+ <para>(The IDL also defines virtual classes and virtual fields, which don't correspond to database entities. We won't discuss them here, because json_query ignores them.)</para>\r
+\r
+ <para>When it first starts up, json_query loads a relevant subset of the IDL into memory. Thereafter, it consults its copy of the IDL whenever it needs to know about the database \r
+ structure. It uses the IDL to validate the JSON queries, and to translate classes and fields to the corresponding tables and columns. In some cases it uses the IDL to supply information \r
+ that the queries don't provide.\r
+ Definitions</para>\r
+\r
+ <para>You should also be familiar with JSON. However it is worth defining a couple of terms that have other meanings in other contexts:</para>\r
+\r
+ <itemizedlist>\r
+ <listitem><para>An "object" is a JSON object, i.e. a comma-separated list of name:value pairs, enclosed in curly braces, like this:</para>\r
+ <screen>{ "a":"frobozz", "b":24, "c":null }</screen>\r
+ </listitem>\r
+ <listitem><para>An "array" is a JSON array, i.e. a comma-separated list of values, enclosed in square brackets, like this:</para>\r
+ <screen>[ "Goober", 629, null, false, "glub" ]</screen>\r
+ </listitem> \r
+ </itemizedlist> \r
+ </simplesect>\r
+ <simplesect>\r
+ <title>The Examples</title>\r
+ <para>The test_json_query utility generated the SQL for all of the sample queries in this tutorial. Newlines and indentation were then inserted manually for readability.</para>\r
+ <para>All examples involve the actor.org_unit table, sometimes in combination with a few related tables. The queries themselves are designed to illustrate the syntax, not \r
+ to do anything useful at the application level. For example, it's not meaningful to take the square root of an org_unit id, except to illustrate how to code a function call. \r
+ The examples are like department store mannequins -- they have no brains, they're only for display.</para>\r
+ <para>The simplest kind of query defines nothing but a FROM clause. For example:</para>\r
+ <programlisting>\r
+ {\r
+ "from":"aou"\r
+ }\r
+ </programlisting>\r
+ <para>In this minimal example we select from only one table. Later we will see how to join multiple tables.</para>\r
+ <para>Since we don't supply a WHERE clause, json_query constructs a default WHERE clause for us, including all the available columns. The resulting SQL looks like this:</para>\r
+<programlisting language="sql">\r
+SELECT\r
+ "aou".billing_address AS "billing_address",\r
+ "aou".holds_address AS "holds_address",\r
+ "aou".id AS "id",\r
+ "aou".ill_address AS "ill_address",\r
+ "aou".mailing_address AS "mailing_address",\r
+ "aou".name AS "name",\r
+ "aou".ou_type AS "ou_type",\r
+ "aou".parent_ou AS "parent_ou",\r
+ "aou".shortname AS "shortname",\r
+ "aou".email AS "email",\r
+ "aou".phone AS "phone",\r
+ "aou".opac_visible AS "opac_visible"\r
+FROM\r
+ actor.org_unit AS "aou" ; \r
+</programlisting>\r
+ </simplesect>\r
+ <simplesect>\r
+ <title>Default SELECT Clauses</title>\r
+ <indexterm><primary>JSON</primary><secondary>SELECT clauses</secondary></indexterm>\r
+ <para>The default SELECT clause includes every column that the IDL defines it as a non-virtual field for the class in question. If a column is present in the database but \r
+ not defined in the IDL, json_query doesn't know about it. In the case of the example shown above, all the columns are defined in the IDL, so they all show up in the default \r
+ SELECT clause.</para>\r
+ <para>If the FROM clause joins two or more tables, the default SELECT clause includes columns only from the core table, not from any of the joined tables.</para>\r
+ <para>The default SELECT clause has almost the same effect as "SELECT *", but not exactly. If you were to "SELECT * from actor.org_unit_type in psql, the output would \r
+ include all the same columns as in the example above, but not in the same order. A default SELECT clause includes the columns in the order in which the IDL defines them, \r
+ which may be different from the order in which the database defines them.</para> \r
+ <para>In practice, the sequencing of columns in the SELECT clause is not significant. The result set is returned to the client program in the form of a data structure, which \r
+ the client program can navigate however it chooses.</para>\r
+ </simplesect> \r
+ <simplesect>\r
+ <title>Other Lessons</title>\r
+ <para>There are other ways to get a default SELECT clause. However, default SELECT clauses are a distraction at this point, because most of the time you'll specify your \r
+ own SELECT clause explicitly, as we will discuss later.</para>\r
+ <para>Let's consider some more important aspects of this simple example -- more important because they apply to more complex queries as well.</para>\r
+ <itemizedlist>\r
+ <listitem>\r
+ <para> The entire JSON query is an object. In this simple case the object includes only one entry, for the FROM clause. Typically you'll also have entries \r
+ for the SELECT clause and the WHERE clause, and possibly for HAVING, ORDER BY, LIMIT, or OFFSET clauses. There is no separate entry for a GROUP BY clause, which you \r
+ can specify by other means.</para> \r
+ </listitem>\r
+ <listitem>\r
+ <para>Although all the other entries are optional, you must include an entry for the FROM clause. You cannot, for example, do a SELECT USER the way \r
+ you can in psql.</para> \r
+ </listitem>\r
+ <listitem>\r
+ <para>Every column is qualified by an alias for the table. This alias is always the class name for the table, as defined in the IDL.</para> \r
+ </listitem>\r
+ <listitem>\r
+ <para>Every column is aliased with the column name. There is a way to choose a different column alias (not shown here).</para> \r
+ </listitem> \r
+ </itemizedlist>\r
+ </simplesect>\r
+ <simplesect>\r
+ <title>The SELECT Clause</title>\r
+ <para>The following variation also produces a default SELECT clause:</para>\r
+<programlisting>\r
+{\r
+ "from":"aou",\r
+ "select": {\r
+ "aou":"*"\r
+ }\r
+}\r
+</programlisting>\r
+ <para>...and so does this one:</para>\r
+<programlisting>\r
+{\r
+ "select": {\r
+ "aou":null\r
+ },\r
+ "from":"aou"\r
+}\r
+</programlisting>\r
+ <para>While this syntax may not be terribly useful, it does illustrate the minimal structure of a SELECT clause in a JSON query: an entry in the outermost JSON object, \r
+ with a key of <quote>select</quote>. The value associated with this key is another JSON object, whose keys are class names.</para>\r
+ <para>(These two examples also illustrate another point: unlike SQL, a JSON query doesn't care whether the FROM clause or the SELECT clause comes first.)</para>\r
+ <para>Usually you don't want the default SELECT clause. Here's how to select only some of the columns:</para>\r
+<programlisting>\r
+{\r
+ "from":"aou",\r
+ "select": {\r
+ "aou":[ "id", "name" ]\r
+ }\r
+}\r
+</programlisting>\r
+ <para>The value associated with the class name is an array of column names. If you select columns from multiple tables (not shown here), you'll need a separate entry for each table, \r
+ and a separate column list for each entry.</para>\r
+ <para>The previous example results in the following SQL:</para>\r
+<programlisting>\r
+SELECT\r
+ "aou".id AS "id",\r
+ "aou".name AS "name"\r
+FROM\r
+ actor.org_unit AS "aou" ;\r
+</programlisting>\r
+ </simplesect>\r
+ <simplesect>\r
+ <title>Fancier SELECT Clauses</title>\r
+ <para>The previous example featured an array of column names. More generally, it featured an array of field specifications, and one kind of field specification is a column name. \r
+ The other kind is a JSON object, with some combination of the following keys:</para>\r
+ <itemizedlist>\r
+ <listitem>\r
+ <para><quote>column</quote> -- the column name (required).</para> \r
+ </listitem>\r
+ <listitem>\r
+ <para><quote>alias</quote> -- used to define a column alias, which otherwise defaults to the column name.</para> \r
+ </listitem>\r
+ <listitem>\r
+ <para><quote>aggregate</quote> -- takes a value of true or false. Don't worry about this one yet. It concerns the use of GROUP BY clauses, which we will examine \r
+ later.</para> \r
+ </listitem>\r
+ <listitem>\r
+ <para><quote>transform</quote> -- the name of an SQL function to be called.</para> \r
+ </listitem>\r
+ <listitem>\r
+ <para><quote>result_field</quote> -- used with "transform"; specifies an output column of a function that returns multiple columns at a time.</para> \r
+ </listitem>\r
+ <listitem>\r
+ <para><quote>params</quote> -- used with "transform"; provides a list of parameters for the function. They may be strings, numbers, or nulls.</para> \r
+ </listitem> \r
+ </itemizedlist> \r
+ <para>This example assigns a different column alias:</para>\r
+<programlisting>\r
+{\r
+ "from":"aou",\r
+ "select": {\r
+ "aou": [\r
+ "id",\r
+ { "column":"name", "alias":"org_name" }\r
+ ]\r
+ }\r
+}\r
+\r
+SELECT\r
+ "aou".id AS "id",\r
+ "aou".name AS "org_name"\r
+FROM\r
+ actor.org_unit AS "aou" ;\r
+</programlisting>\r
+ <para>In this case, changing the column alias doesn't accomplish much. But if we were joining to the actor.org_unit_type table, which also has a "name" column, we could \r
+ use different aliases to distinguish them.</para>\r
+ <para>The following example uses a function to raise a column to upper case:</para>\r
+<programlisting>\r
+{\r
+ "from":"aou",\r
+ "select": {\r
+ "aou": [\r
+ "id",\r
+ { "column":"name", "transform":"upper" }\r
+ ]\r
+ }\r
+}\r
+ \r
+SELECT\r
+ "aou".id AS "id",\r
+ upper("aou".name ) AS "name"\r
+FROM\r
+ actor.org_unit AS "aou" ;\r
+</programlisting>\r
+ <para>Here we take a substring of the name, using the <emphasis role="italics">params</emphasis> element to pass parameters:</para>\r
+ <programlisting>\r
+ {\r
+ "from":"aou",\r
+ "select": {\r
+ "aou": [\r
+ "id", {\r
+ "column":"name",\r
+ "transform":"substr",\r
+ "params":[ 3, 5 ]\r
+ }\r
+ ]\r
+ }\r
+ }\r
+ \r
+ SELECT\r
+ "aou".id AS "id",\r
+ substr("aou".name,'3','5' ) AS "name"\r
+ FROM\r
+ actor.org_unit AS "aou" ;\r
+ </programlisting>\r
+ <para>The parameters specified with <emphasis role="italics">params</emphasis> are inserted after the applicable column (<emphasis role="italics">name</emphasis> in this case), \r
+ which is always the first parameter. They are always passed as strings, i.e. enclosed in quotes, even if the JSON expresses them as numbers. PostgreSQL will ordinarily \r
+ coerce them to the right type. However if the function name is overloaded to accept different types, PostgreSQL may invoke a function other than the one intended.</para>\r
+ <para>Finally we call a fictitious function "frobozz" that returns multiple columns, where we want only one of them:</para>\r
+<programlisting>\r
+{\r
+ "from":"aou",\r
+ "select": {\r
+ "aou": [\r
+ "id", {\r
+ "column":"name",\r
+ "transform":"frobozz",\r
+ "result_field":"zamzam"\r
+ }\r
+ ]\r
+ }\r
+}\r
+ \r
+SELECT\r
+ "aou".id AS "id",\r
+ (frobozz("aou".name ))."zamzam" AS "name"\r
+FROM\r
+ actor.org_unit AS "aou" ;\r
+</programlisting>\r
+ <para>The <emphasis role="italics">frobozz</emphasis> function doesn't actually exist, but json_query doesn't know that. The query won't fail until json_query tries to execute it in \r
+ the database.</para>\r
+ </simplesect>\r
+ <simplesect>\r
+ <title>Things You Can't Do</title>\r
+ <para>You can do some things in a SELECT clause with raw SQL (with psql, for example) that you can't do with a JSON query. Some of them matter and some of them don't.</para>\r
+ <para>When you do a JOIN, you can't arrange the selected columns in any arbitrary sequence, because all of the columns from a given table must be grouped together. \r
+ This limitation doesn't matter. The results are returned in the form of a data structure, which the client program can navigate however it likes.</para>\r
+ <para>You can't select an arbitrary expression, such as "percentage / 100" or "last_name || ', ' || first_name". Most of the time this limitation doesn't matter either, because \r
+ the client program can do these kinds of manipulations for itself. However, function calls may be a problem. You can't nest them, and you can't pass more than one column value \r
+ to them (and it has to be the first parameter).</para> \r
+ <para>You can't use a CASE expression. Instead, the client application can do the equivalent branching for itself.</para>\r
+ <para>You can't select a subquery. In raw SQL you can do something like the following:</para>\r
+<programlisting>\r
+SELECT\r
+ id,\r
+ name,\r
+ (\r
+ SELECT name\r
+ FROM actor.org_unit_type AS aout\r
+ WHERE aout.id = aou.ou_type\r
+ ) AS type_name\r
+FROM\r
+ actor.org_unit AS aou;\r
+</programlisting>\r
+ <para>This contrived example is not very realistic. Normally you would use a JOIN in this case, and that's what you should do in a JSON query. Other cases may not be so \r
+ easy to solve.</para>\r
+ </simplesect>\r
+ <simplesect>\r
+ <title>The WHERE Clause</title>\r
+ <para>Most queries need a WHERE clause, as in this simple example:</para>\r
+<programlisting>\r
+{\r
+ "from":"aou",\r
+ "select": { "aou":[ "id", "name" ] },\r
+ "where": {\r
+ "parent_ou":"3"\r
+ }\r
+}\r
+</programlisting>\r
+ <para>Like the SELECT clause, the WHERE clause gets its own entry in the top-level object of a JSON query. The key is <quote>where</quote>, and the associated value is either \r
+ an object (as shown here) or an array (to be discussed a bit later). Each entry in the object is a separate condition.</para>\r
+ <para>In this case, we use a special shortcut for expressing an equality condition. The column name is on the left of the colon, and the value to which we are equating it is on \r
+ the right.</para>\r
+ <para>Here's the resulting SQL:</para>\r
+<programlisting>\r
+SELECT\r
+ "aou".id AS "id",\r
+ "aou".name AS "name"\r
+FROM\r
+ actor.org_unit AS "aou"\r
+WHERE\r
+ "aou".parent_ou = 3;\r
+</programlisting>\r
+ <para>Like the SELECT clause, the generated WHERE clause qualifies each column name with the alias of the relevant table.</para>\r
+ <para>If you want to compare a column to NULL, put <quote>null</quote> (without quotation marks) to the right of the colon instead of a literal value. The \r
+ resulting SQL will include <quote>IS NULL</quote> instead of an equals sign.</para>\r
+ </simplesect>\r
+ <simplesect>\r
+ <title>Other Kinds of Comparisons</title>\r
+ <para>Here's the same query (which generates the same SQL) without the special shortcut:</para>\r
+<programlisting>\r
+{\r
+ "from":"aou",\r
+ "select": { "aou":[ "id", "name" ] },\r
+ "where": {\r
+ "parent_ou":{ "=":3 }\r
+ }\r
+}\r
+</programlisting>\r
+ <para>We still have an entry whose key is the column name, but this time the associated value is another JSON object. It must contain exactly one entry, \r
+ with the comparison operator on the left of the colon, and the value to be compared on the right.</para>\r
+ <para>The same syntax works for other kinds of comparison operators. For example:</para>\r
+<programlisting>\r
+{\r
+ "from":"aou",\r
+ "select": { "aou":[ "id", "name" ] },\r
+ "where": {\r
+ "parent_ou":{ ">":3 }\r
+ }\r
+}\r
+</programlisting>\r
+ <para>...turns into:</para>\r
+<programlisting>\r
+SELECT\r
+ "aou".id AS "id",\r
+ "aou".name AS "name"\r
+FROM\r
+ actor.org_unit AS "aou"\r
+WHERE\r
+ "aou".parent_ou > 3 ;\r
+</programlisting>\r
+ <para>The condition '<quote>=</quote>:null' turns into IS NULL. Any other operator used with <quote>null</quote> turns into IS NOT NULL.</para>\r
+ <para>You can use most of the comparison operators recognized by PostgreSQL:</para>\r
+ <literallayout> \r
+ = <> !=\r
+ < > <= >=\r
+ ~ ~* !~ !~*\r
+ like ilike\r
+ similar to\r
+ </literallayout> \r
+ <para>The only ones you can't use are <quote>is distinct from</quote> and <quote>is not distinct from</quote>.</para> \r
+ </simplesect>\r
+ <simplesect>\r
+ <title>Custom Comparisons</title>\r
+ <para>Here's a dirty little secret: json_query doesn't really pay much attention to the operator you supply. It merely checks to make sure that the operator doesn't contain \r
+ any semicolons or white space, in order to prevent certain kinds of SQL injection. It also allows "similar to" as a special exception.</para>\r
+ <para>As a result, you can slip an operator of your own devising into the SQL, so long as it doesn't contain any semicolons or white space, and doesn't create invalid syntax. \r
+ Here's a contrived and rather silly example:</para>\r
+<programlisting>\r
+{\r
+ "from":"aou",\r
+ "select": { "aou":[ "id", "name" ] },\r
+ "where": {\r
+ "parent_ou":{ "<2+":3 }\r
+ }\r
+}\r
+</programlisting>\r
+ <para>...which results in the following SQL:</para>\r
+<programlisting>\r
+SELECT\r
+ "aou".id AS "id",\r
+ "aou".name AS "name"\r
+FROM\r
+ actor.org_unit AS "aou"\r
+WHERE\r
+ "aou".parent_ou <2+ 3;\r
+</programlisting>\r
+ <para>It's hard to come up with a realistic case where this hack would be useful, but it could happen.</para>\r
+ </simplesect>\r
+ <simplesect>\r
+ <title>Comparing One Column to Another</title>\r
+ <para>Here's how to put another column on the right hand side of a comparison:</para>\r
+<programlisting>\r
+{\r
+ "from":"aou",\r
+ "select": { "aou":[ "id", "name" ] },\r
+ "where": {\r
+ "id": { ">": { "+aou":"parent_ou" } }\r
+ }\r
+};\r
+</programlisting>\r
+ <para>This syntax is similar to the previous examples, except that instead of comparing to a literal value, we compare to an object. This object has only a single entry, \r
+ whose key is a table alias preceded by a leading plus sign. The associated value is the name of the column.</para>\r
+ <para>Here's the resulting SQL:</para>\r
+<programlisting>\r
+SELECT\r
+ "aou".id AS "id",\r
+ "aou".name AS "name"\r
+FROM\r
+ actor.org_unit AS "aou"\r
+WHERE\r
+(\r
+ "aou".id > ( "aou".parent_ou )\r
+);\r
+</programlisting>\r
+ <para>The table alias must correspond to the appropriate table. Since json_query doesn't validate the choice of alias, it won't detect an invalid alias until it tries to \r
+ execute the query. In this simple example there's only one table to choose from. The choice of alias is more important in a subquery or join.</para>\r
+ <para>The leading plus sign, combined with a table alias, can be used in other situations to designate the table to which a column belongs. We shall defer a discussion of \r
+ this usage to the section on joins.</para>\r
+ </simplesect> \r
+ <simplesect>\r
+ <title>Testing Boolean Columns</title>\r
+ <para>In SQL, there are several ways to test a boolean column such as actor.org_unit.opac_visible. The most obvious way is to compare it to true or false:</para>\r
+<programlisting>\r
+SELECT\r
+ id\r
+FROM\r
+ actor.org_unit\r
+WHERE\r
+ opac_visible = true;\r
+</programlisting>\r
+ <para>In a JSON query this approach doesn't work. If you try it, the "= true" test will turn into IS NULL. Don't do that. Instead, use a leading plus sign, as described in \r
+ the preceding section, to treat the boolean column as a stand-alone condition:</para>\r
+<programlisting>\r
+{\r
+ "from":"aou",\r
+ "select": { "aou":[ "id" ] },\r
+ "where": {\r
+ "+aou":"opac_visible"\r
+ }\r
+}\r
+</programlisting>\r
+ <para>Result:</para>\r
+<programlisting>\r
+SELECT\r
+ "aou".id AS "id"\r
+FROM\r
+ actor.org_unit AS "aou"\r
+WHERE\r
+ "aou".opac_visible ;\r
+</programlisting>\r
+ <para>If you need to test for falsity, then write a test for truth and negate it with the "-not" operator. We will discuss the "-not" operator later, but here's a preview:</para>\r
+<programlisting>\r
+{\r
+ "from":"aou",\r
+ "select": { "aou":[ "id" ] },\r
+ "where": {\r
+ "-not": {\r
+ "+aou":"opac_visible"\r
+ }\r
+ }\r
+} \r
+\r
+SELECT\r
+ "aou".id AS "id"\r
+FROM\r
+ actor.org_unit AS "aou"\r
+WHERE\r
+ NOT ( "aou".opac_visible ); \r
+</programlisting>\r
+ <para>You can also compare a boolean column directly to a more complex condition:</para>\r
+<programlisting>\r
+{\r
+ "from":"aou",\r
+ "select": { "aou":[ "id" ] },\r
+ "where": {\r
+ "opac_visible": {\r
+ "=": { "parent_ou":{ ">":3 } }\r
+ }\r
+ }\r
+}\r
+</programlisting>\r
+ <para>Here we compare a boolean column, not to a literal value, but to a boolean expression. The resulting SQL looks a little goofy, but it works:</para>\r
+<programlisting>\r
+SELECT\r
+ "aou".id AS "id"\r
+FROM\r
+ actor.org_unit AS "aou"\r
+WHERE\r
+ (\r
+ "aou".opac_visible = ( "aou".parent_ou > 3 )\r
+ );\r
+</programlisting>\r
+ <para>In this case we compare the boolean column to a single simple condition. However you can include additional complications -- multiple conditions, IN lists, \r
+ BETWEEN clauses, and other features as described below.</para>\r
+ </simplesect>\r
+ <simplesect>\r
+ <title>Multiple Conditions</title>\r
+ <para>If you need multiple conditions, just add them to the "where" object, separated by commas:</para>\r
+<programlisting>\r
+{\r
+ "from":"aou",\r
+ "select": { "aou":[ "id", "name" ] },\r
+ "where": {\r
+ "parent_ou":{ ">":3 },\r
+ "id":{ "<>":7 }\r
+ }\r
+}\r
+</programlisting>\r
+ <para>The generated SQL connects the conditions with AND:</para>\r
+<programlisting>\r
+SELECT\r
+ "aou".id AS "id",\r
+ "aou".name AS "name"\r
+FROM\r
+ actor.org_unit AS "aou"\r
+WHERE\r
+ "aou".parent_ou g 3\r
+ AND "aou".id <> 7;\r
+</programlisting>\r
+ <para>Later we will see how to use OR instead of AND.</para>\r
+ </simplesect>\r
+ <simplesect>\r
+ <title>Using Arrays</title>\r
+ <para>Here's a puzzler. Suppose you need two conditions for the same column. How do you code them in the same WHERE clause? For example, suppose you want something like this:</para>\r
+<programlisting>\r
+SELECT\r
+ id,\r
+ name\r
+FROM\r
+ actor.org_unit\r
+WHERE\r
+ parent_ou > 3\r
+ AND parent_ou <> 7;\r
+</programlisting>\r
+ <para>You might try a WHERE clause like this:</para>\r
+<programlisting>\r
+"where": {\r
+ "parent_ou":{ ">":3 },\r
+ "parent_ou":{ "<>":7 }\r
+ }\r
+</programlisting>\r
+ <para>Nope. Won't work. According to JSON rules, two entries in the same object can't have the same key.</para>\r
+ <para>After slapping yourself in the forehead, you try something a little smarter:</para>\r
+<programlisting>\r
+"where": {\r
+ "parent_ou": {\r
+ ">":3,\r
+ "<>":7\r
+ }\r
+}\r
+</programlisting>\r
+ <para>Nice try, but that doesn't work either. Maybe it ought to work -- at least it's legal JSON -- but, no.</para>\r
+ <para>Here's what works:</para>\r
+<programlisting>\r
+{\r
+ "from":"aou",\r
+ "select": { "aou":[ "id", "name" ] },\r
+ "where": [\r
+ { "parent_ou":{ ">":3 } },\r
+ { "parent_ou":{ "<>":7 } }\r
+ ]\r
+}\r
+</programlisting>\r
+ <para>We wrapped the two conditions into two separate JSON objects, and then wrapped those objects together into a JSON array. The resulting SQL looks like this:</para>\r
+<programlisting>\r
+SELECT\r
+ "aou".id AS "id",\r
+ "aou".name AS "name\r
+FROM\r
+ actor.org_unit AS "aou"\r
+WHERE\r
+ ( "aou".parent_ou > 3 )\r
+AND\r
+ ( "aou".parent_ou <> 7 );\r
+</programlisting>\r
+ <para>That's not quite what we were hoping for, because the extra parentheses are so ugly. But they're harmless. This will do.</para>\r
+ <para>If you're in the mood, you can use arrays to as many parentheses as you like, even if there is only one condition inside:</para>\r
+<programlisting>\r
+{\r
+ "from":"aou",\r
+ "select": { "aou":[ "id", "name" ] },\r
+ "where":\r
+ [[[[[[\r
+ {\r
+ "parent_ou":{ ">":3 }\r
+ },\r
+ ]]]]]]\r
+} \r
+</programlisting>\r
+ <para>...yields:</para>\r
+<programlisting>\r
+SELECT\r
+ "aou".id AS "id",\r
+ "aou".name AS "name"\r
+FROM\r
+ actor.org_unit AS "aou"\r
+WHERE\r
+ ( ( ( ( ( ( "aou".parent_ou > 3 ) ) ) ) ) );\r
+</programlisting>\r
+ </simplesect>\r
+ <simplesect>\r
+ <title>How to OR</title>\r
+ <para>By default, json_query combines conditions with AND. When you need OR, here's how to do it:</para> \r
+<programlisting>\r
+{\r
+ "from":"aou",\r
+ "select": { "aou":[ "id", "name" ] },\r
+ "where": {\r
+ "-or": {\r
+ "id":2,\r
+ "parent_ou":3\r
+ }\r
+ }\r
+}\r
+</programlisting>\r
+ <para>We use <quote>-or</quote> as the key, with the conditions to be ORed in an associated object. The leading minus sign is there to make sure that the operator isn't confused with a \r
+ column name. Later we'll see some other operators with leading minus signs. In a couple of spots we even use plus signs.</para>\r
+ <para>Here are the results from the above example:</para>\r
+<programlisting>\r
+SELECT\r
+ "aou".id AS "id",\r
+ "aou".name AS "name"\r
+FROM\r
+ actor.org_unit AS "aou"\r
+WHERE\r
+ (\r
+ "aou".id = 2\r
+ OR "aou".parent_ou = 3\r
+ );\r
+</programlisting>\r
+ <para>The conditions paired with <quote>-or</quote> are linked by OR and enclosed in parentheses.</para>\r
+ <para>Here's how to do the same thing using an array, except that it produces an extra layer of parentheses:</para>\r
+<programlisting>\r
+{\r
+ "from":"aou",\r
+ "select": { "aou":[ "id", "name" ] },\r
+ "where": {\r
+ "-or": [\r
+ { "id":2 },\r
+ { "parent_ou":3 }\r
+ ]\r
+ }\r
+}\r
+SELECT\r
+ "aou".id AS "id",\r
+ "aou".name AS "name"\r
+FROM\r
+ actor.org_unit AS "aou"\r
+WHERE\r
+ (\r
+ ( "aou".id = 2 )\r
+ OR ( "aou".parent_ou = 3 )\r
+ );\r
+</programlisting>\r
+ <para>It's possible, though not very useful, to have only a single condition subject to the <quote>-or</quote> operator. In that case, the condition appears by itself, since there's nothing \r
+ to OR it to. This trick is another way to add an extraneous layer of parentheses.</para>\r
+ </simplesect>\r
+ <simplesect>\r
+ <title>Another way to AND</title>\r
+ <para>You can also use the <quote>-and</quote> operator. It works just like <quote>-or</quote>, except that it combines conditions with AND instead of OR. Since AND is the default, we don't usually \r
+ need a separate operator for it, but it's available.</para>\r
+ <para>In rare cases, nothing else will do -- you can't include two conditions in the same list because of the duplicate key problem, but you can't combine them with \r
+ arrays either. In particular, you might need to combine them within an expression that you're comparing to a boolean column (see the subsection above on Testing Boolean Columns).</para>\r
+ </simplesect>\r
+ <simplesect>\r
+ <title>Negation with NOT</title>\r
+ <para>The <quote>-not</quote> operator negates a condition or set of conditions. For example:</para>\r
+<programlisting>\r
+{\r
+ "from":"aou",\r
+ "select": { "aou":[ "id", "name" ] },\r
+ "where": {\r
+ "-not": {\r
+ "id":{ ">":2 },\r
+ "parent_ou":3\r
+ }\r
+ }\r
+}\r
+ \r
+SELECT\r
+ "aou".id AS "id",\r
+ "aou".name AS "name"\r
+FROM\r
+ actor.org_unit AS "aou"\r
+WHERE\r
+ NOT\r
+ (\r
+ "aou".id > 2\r
+ AND "aou".parent_ou = 3\r
+ );\r
+</programlisting>\r
+ <para>In this example we merely negate a combination of two comparisons. However the condition to be negated may be as complicated as it needs to be. Anything that can be \r
+ subject to <quote>where</quote> can be subject to <quote>-not</quote>.</para>\r
+ <para>In most cases you can achieve the same result by other means. However the <quote>-not</quote> operator is the only way to represent NOT BETWEEN \r
+ (to be discussed later).</para> \r
+ </simplesect>\r
+ <simplesect>\r
+ <title>EXISTS with Subqueries</title>\r
+ <para>Two other operators carry a leading minus sign: <quote>-exists</quote> and its negation <quote>-not-exists</quote>. These operators apply to subqueries, which have the \r
+ same format as a full query. For example:</para>\r
+<programlisting> \r
+{\r
+ "from":"aou",\r
+ "select": { "aou":[ "id", "name" ] },\r
+ "where": {\r
+ "-exists": {\r
+ "from":"asv",\r
+ "select":{ "asv":[ "id" ] },\r
+ "where": {\r
+ "owner":7\r
+ }\r
+ }\r
+ }\r
+}\r
+ \r
+SELECT\r
+ "aou".id AS "id",\r
+ "aou".name AS "name"\r
+FROM\r
+ actor.org_unit AS "aou"\r
+WHERE\r
+EXISTS\r
+ (\r
+ SELECT "asv".id AS "id"\r
+ FROM action.survey AS "asv"\r
+ WHERE "asv".owner = 7\r
+ );\r
+</programlisting>\r
+ <para>This kind of subquery is of limited use, because its WHERE clause doesn't have anything to do with the main query. It just shuts down the main query altogether \r
+ if it isn't satisfied.</para>\r
+ <para>More typical is a correlated subquery, whose WHERE clause refers to a row from the main query. For example:</para>\r
+<programlisting> \r
+{\r
+ "from":"aou",\r
+ "select": { "aou":[ "id", "name" ] },\r
+ "where": {\r
+ "-exists": {\r
+ "from":"asv",\r
+ "select":{ "asv":[ "id" ] },\r
+ "where": {\r
+ "owner":{ "=":{ "+aou":"id" }}\r
+ }\r
+ }\r
+ }\r
+} \r
+</programlisting>\r
+ <para>Note the use of <quote>+aou</quote> to qualify the id column in the inner WHERE clause.</para>\r
+<programlisting> \r
+SELECT\r
+ "aou".id AS "id",\r
+ "aou".name AS "name"\r
+FROM\r
+ actor.org_unit AS "aou"\r
+WHERE\r
+ EXISTS\r
+ (\r
+ SELECT "asv".id AS "id"\r
+ FROM action.survey AS "asv"\r
+ WHERE ("asv".owner = ( "aou".id ))\r
+ );\r
+</programlisting>\r
+ <para>This latter example illustrates the syntax, but in practice, it would probably be more natural to use an IN clause with a subquery (to be discussed later).</para>\r
+ </simplesect>\r
+ <simplesect>\r
+ <title>BETWEEN Clauses</title>\r
+ <para>Here's how to express a BETWEEN clause:</para>\r
+<programlisting> \r
+{\r
+ "from":"aou",\r
+ "select": { "aou":[ "id" ] },\r
+ "where": {\r
+ "parent_ou": { "between":[ 3, 7 ] }\r
+ }\r
+}\r
+</programlisting>\r
+ <para>The value associated with the column name is an object with a single entry, whose key is "between". The corresponding value is an array with exactly two values, defining the \r
+ range to be tested.</para>\r
+ <para>The range bounds must be either numbers or string literals. Although SQL allows them to be null, a null doesn't make sense in this context, because a null never matches \r
+ anything. Consequently json_query doesn't allow them.</para>\r
+ <para>The resulting SQL is just what you would expect:</para>\r
+<programlisting> \r
+SELECT\r
+ "aou".id AS "id"\r
+FROM\r
+ actor.org_unit AS "aou"\r
+WHERE\r
+ parent_ou BETWEEN '3' AND '7';\r
+</programlisting>\r
+ </simplesect>\r
+ <simplesect>\r
+ <title>IN and NOT IN Lists</title>\r
+ <para>There are two ways to code an IN list. One way is simply to include the list of values in an array:</para>\r
+<programlisting> \r
+{\r
+ "from":"aou",\r
+ "select": { "aou":[ "id", "name" ] },\r
+ "where": {\r
+ "parent_ou": [ 3, 5, 7 ]\r
+ }\r
+}\r
+</programlisting>\r
+ <para>As with a BETWEEN clause, the values in the array must be numbers or string literals. Nulls aren't allowed. Here's the resulting SQL, which again is just what \r
+ you would expect:</para>\r
+<programlisting> \r
+SELECT\r
+ "aou".id AS "id",\r
+ "aou".name AS "name"\r
+FROM\r
+ actor.org_unit AS "aou"\r
+WHERE\r
+ "aou".parent_ou IN (3, 5, 7);\r
+</programlisting>\r
+ <para>The other way is similar to the syntax shown above for a BETWEEN clause, except that the array may include any non-zero number of values:</para>\r
+<programlisting> \r
+{\r
+ "from":"aou",\r
+ "select": { "aou":[ "id", "name" ] },\r
+ "where": {\r
+ "parent_ou": { "in": [ 3, 5, 7 ] }\r
+ }\r
+}\r
+</programlisting>\r
+ <para>This version results in the same SQL as the first one.</para>\r
+ <para>For a NOT IN list, you can use the latter format, using the <quote>not in</quote> operator instead of <quote>in</quote>. Alternatively, you can use either format together with \r
+ the <quote>-not</quote> operator.</para> \r
+ </simplesect>\r
+ <simplesect>\r
+ <title>IN and NOT IN Clauses with Subqueries</title>\r
+ <para>For an IN clause with a subquery, the syntax is similar to the second of the two formats for an IN list (see the previous subsection). The "in" or "not in" operator \r
+ is paired, not with an array of values, but with an object representing the subquery. For example:</para>\r
+<programlisting> \r
+{\r
+ "from":"aou",\r
+ "select": { "aou":[ "id", "name" ] },\r
+ "where": {\r
+ "id": {\r
+ "in": {\r
+ "from":"asv",\r
+ "select":{ "asv":[ "owner" ] },\r
+ "where":{ "name":"Voter Registration" }\r
+ }\r
+ }\r
+ }\r
+}\r
+</programlisting>\r
+ <para>The results:</para>\r
+<programlisting> \r
+SELECT\r
+ "aou".id AS "id",\r
+ "aou".name AS "name"\r
+FROM\r
+ actor.org_unit AS "aou"\r
+WHERE\r
+ "aou".id IN\r
+ (\r
+ SELECT\r
+ "asv".owner AS "owner"\r
+ FROM\r
+ action.survey AS "asv"\r
+ WHERE\r
+ "asv".name = 'Voter Registration'\r
+ );\r
+</programlisting>\r
+ <para>In SQL the subquery may select multiple columns, but in a JSON query it can select only a single column.</para>\r
+ <para>For a NOT IN clause with a subquery, use the <quote>not in</quote> operator instead of <quote>in</quote>.</para>\r
+ </simplesect>\r
+ <simplesect>\r
+ <title>Comparing to a Function</title>\r
+ <para>Here's how to compare a column to a function call:</para>\r
+<programlisting> \r
+{\r
+ "from":"aou",\r
+ "select": { "aou":[ "id", "name" ] },\r
+ "where": {\r
+ "id":{ ">":[ "sqrt", 16 ] }\r
+ }\r
+}\r
+</programlisting>\r
+ <para>A comparison operator (<quote>></quote> in this case) is paired with an array. The first entry in the array must be a string giving the name of the function. The remaining parameters, \r
+ if any, are the parameters. They may be strings, numbers, or nulls. The resulting SQL for this example:</para>\r
+<programlisting> \r
+SELECT\r
+ "aou".id AS "id",\r
+ "aou".name AS "name"\r
+FROM\r
+ actor.org_unit AS "aou"\r
+WHERE\r
+ "aou".id > sqrt( '16' );\r
+</programlisting>\r
+ <para>All parameters are passed as quoted strings -- even if, as in this case, they are really numbers.</para>\r
+ <para>This syntax is somewhat limited in that the function parameters must be constants (hence the use of a silly example).</para>\r
+ </simplesect>\r
+ <simplesect>\r
+ <title>Putting a Function Call on the Left</title>\r
+ <para>In the discussion of the SELECT clause, we saw how you could transform the value of a selected column by passing it to a function. In the WHERE clause, you can \r
+ use similar syntax to transform the value of a column before comparing it to something else.</para>\r
+ <para>For example:</para>\r
+<programlisting> \r
+{\r
+ "from":"aou",\r
+ "select": { "aou":[ "id", "name" ] },\r
+ "where": {\r
+ "name": {\r
+ "=": {\r
+ "transform":"upper",\r
+ "value":"CARTER BRANCH"\r
+ }\r
+ }\r
+ }\r
+}\r
+</programlisting>\r
+ <para>The "transform" entry gives the name of the function that we will use on the left side of the comparison. The "value" entry designates the value on the right side \r
+ of the comparison.</para>\r
+<programlisting> \r
+SELECT\r
+ "aou".id AS "id",\r
+ "aou".name AS "name"\r
+FROM\r
+ actor.org_unit AS "aou"\r
+WHERE\r
+ upper("aou".name ) = 'CARTER BRANCH' ;\r
+</programlisting>\r
+ <para>As in the SELECT clause, you can pass literal values or nulls to the function as additional parameters by using an array tagged as <quote>params</quote>:</para>\r
+<programlisting> \r
+{\r
+ "from":"aou",\r
+ "select": { "aou":[ "id", "name" ] },\r
+ "where": {\r
+ "name": {\r
+ "=": {\r
+ "transform":"substr",\r
+ "params":[ 1, 6 ],\r
+ "value":"CARTER"\r
+ }\r
+ }\r
+ }\r
+}\r
+ \r
+SELECT\r
+ "aou".id AS "id",\r
+ "aou".name AS "name"\r
+FROM\r
+ actor.org_unit AS "aou"\r
+WHERE\r
+ substr("aou".name,'1','6' ) = 'CARTER' ;\r
+</programlisting>\r
+ <para>The first parameter is always the column name, qualified by the class name, followed by any additional parameters (which are always enclosed in quotes even if they \r
+ are numeric).</para>\r
+ <para>As in the SELECT clause: if the function returns multiple columns, you can specify the one you want by using a "result_field" entry (not shown here).</para>\r
+ <para>If you leave out the "transform" entry (or misspell it), the column name will appear on the left without any function call. This syntax works, but it's more \r
+ complicated than it needs to be.</para>\r
+ </simplesect>\r
+\r
+ <simplesect>\r
+ <title>Putting Function Calls on Both Sides</title>\r
+ <para>If you want to compare one function call to another, you can use the same syntax shown in the previous subsection -- except that the <quote>value</quote> entry carries an \r
+ array instead of a literal value. For example:</para>\r
+<programlisting> \r
+{\r
+ "from":"aou",\r
+ "select": { "aou":[ "id", "name" ] },\r
+ "where": {\r
+ "id": {\r
+ ">": {\r
+ "transform":"factorial",\r
+ "value":[ "sqrt", 1000 ]\r
+ }\r
+ }\r
+ }\r
+} \r
+SELECT\r
+ "aou".id AS "id",\r
+ "aou".name AS "name"\r
+FROM\r
+ actor.org_unit AS "aou"\r
+WHERE\r
+ factorial("aou".id ) > sqrt( '1000' ) ;\r
+</programlisting>\r
+ <para>The format for the right side function is similar to what we saw earlier, in the subsection Comparing to a Function. Note that there are two different formats \r
+ for defining function calls:</para>\r
+ <itemizedlist>\r
+ <listitem>\r
+ <para>For a function call to the left of the comparison, the function name is tagged as <quote>transform</quote>. The first parameter is always the relevant \r
+ column name; additional parameters, if any, are in an array tagged as "params". The entry for <quote>result_field</quote>, if present, specifies a subcolumn.</para> \r
+ </listitem>\r
+ <listitem>\r
+ <para>For a function call to the right of the comparison, the function name is the first entry in an array, together with any parameters. \r
+ There's no way to specify a subcolumn.</para> \r
+ </listitem>\r
+ </itemizedlist> \r
+ </simplesect>\r
+ <simplesect>\r
+ <title>Comparing a Function to a Condition</title>\r
+ <para>So far we have seen two kinds of data for the <quote>value</quote> tag. A string or number translates to a literal value, and an array translates to a function call. \r
+ The third possibility is a JSON object, which translates to a condition. For example:</para>\r
+<programlisting> \r
+{\r
+ "from":"aou",\r
+ "select": { "aou":[ "id", "name" ] },\r
+ "where": {\r
+ "id": {\r
+ "=": {\r
+ "value":{ "parent_ou":{ ">":3 } },\r
+ "transform":"is_prime"\r
+ }\r
+ }\r
+ }\r
+}\r
+</programlisting>\r
+ <para>The function tagged as <quote>transform</quote> must return boolean, or else json_query will generate invalid SQL. The function used here, <quote>is_prime</quote>, \r
+ is fictitious.</para>\r
+<programlisting> \r
+SELECT\r
+ "aou".id AS "id",\r
+ "aou".name AS "name"\r
+FROM\r
+ actor.org_unit AS "aou"\r
+WHERE\r
+(\r
+ is_prime("aou".id ) = ( "aou".parent_ou > 3 )\r
+);\r
+</programlisting>\r
+ <para>If we left out the <quote>transform</quote> entry, json_query would compare the column on the left (which would to be boolean) to the condition on the right. The results are similar \r
+ to those for a simpler format described earlier (see the subsection Testing Boolean Columns).</para>\r
+ <para>In the example above we compared the boolean to a simple condition. However the expression on the right may include multiple conditions, IN lists, subqueries, \r
+ and whatever other complications are necessary.</para>\r
+ </simplesect>\r
+ <simplesect>\r
+ <title>Things You Can't Do</title>\r
+ <para>The WHERE clause is subject to some of the same limitations as the SELECT clause. However, in the WHERE clause these limitations are more limiting, because \r
+ the client program can't compensate by doing some of the work for itself.</para>\r
+ <para>You can't use arbitrary expressions in a WHERE condition, such as "WHERE id > parent_ou -- 3". In some cases you may be able to contrive a custom operator in order to \r
+ fake such an expression. However this mechanism is neither very general nor very aesthetic.</para>\r
+ <para>To the right of a comparison operator, all function parameters must be literals or null. You can't pass a column value, nor can you nest function calls.</para>\r
+ <para>Likewise you can't include column values or arbitrary expressions in an IN list or a BETWEEN clause.</para>\r
+ <para>You can't include null values in an IN list or a BETWEEN list, not that you should ever want to.</para>\r
+ <para>As noted earlier: you can't use the comparison operators <quote>is distinct from</quote> or <quote>is not distinct from</quote>.</para>\r
+ <para>Also as noted earlier: a subquery in an IN clause cannot select more than one column.</para>\r
+ </simplesect>\r
+ <simplesect>\r
+ <title>JOIN clauses</title>\r
+ <para>Until now, our examples have selected from only one table at a time. As a result, the FROM clause has been very simple -- just a single string containing \r
+ the class name of the relevant table.</para>\r
+ <para>When the FROM clause joins multiple tables, the corresponding JSON naturally gets more complicated.</para>\r
+ <para>SQL provides two ways to define a join. One way is to list both tables in the FROM clause, and put the join conditions in the WHERE clause:</para>\r
+<programlisting> \r
+SELECT\r
+ aou.id,\r
+ aout.name\r
+FROM\r
+ actor.org_unit aou,\r
+ actor.org_unit_type aout\r
+WHERE\r
+ aout.id = aou.ou_type;\r
+</programlisting>\r
+ <para>The other way is to use an explicit JOIN clause:</para>\r
+<programlisting> \r
+SELECT\r
+ aou.id,\r
+ aout.name\r
+FROM\r
+ actor.org_unit aou\r
+ JOIN actor.org_unit_type aout\r
+ ON ( aout.id = aou.ou_type );\r
+</programlisting>\r
+ <para>JSON queries use only the second of these methods. The following example expresses the same query in JSON:</para>\r
+<programlisting> \r
+{\r
+ "select": { "aou":[ "id" ], "aout":[ "name" ] },\r
+ "from": {\r
+ "aou":"aout"\r
+ }\r
+}\r
+</programlisting>\r
+ <para>First, let's review the SELECT clause. Since it selects rows from two different tables, the data for <quote>select</quote> includes two entries, one for each table.</para>\r
+ <para>As for the FROM clause, it's no longer just a string. It's a JSON object, with exactly one entry. The key of this entry is the class name of the core table, i.e. \r
+ the table named immediately after the FROM keyword. The data associated with this key contains the rest of the information about the join. In this simple example, \r
+ that information consists entirely of a string containing the class name of the other table.</para>\r
+ <para>So where is the join condition?</para>\r
+ <para>It's in the IDL. Upon reading the IDL, json_query knows that actor.org_unit has a foreign key pointing to actor.org_unit_type, and builds a join condition accordingly:</para>\r
+<programlisting> \r
+SELECT\r
+ "aou".id AS "id",\r
+ "aout".name AS "name"\r
+FROM\r
+ actor.org_unit AS "aou"\r
+ INNER JOIN actor.org_unit_type AS "aout"\r
+ ON ( "aout".id = "aou".ou_type ) ;\r
+</programlisting>\r
+ <para>In this case the core table is the child table, and the joined table is the parent table. We could just as well have written it the other way around:</para>\r
+<programlisting> \r
+{\r
+ "select": { "aou":[ "id" ], "aout":[ "name" ] },\r
+ "from": {\r
+ "aout":"aou"\r
+ }\r
+}\r
+ \r
+SELECT\r
+ "aou".id AS "id",\r
+ "aout".name AS "name"\r
+FROM\r
+ actor.org_unit_type AS "aout"\r
+ INNER JOIN actor.org_unit AS "aou"\r
+ ON ( "aou".ou_type = "aout".id ) ;\r
+</programlisting>\r
+ </simplesect>\r
+ <simplesect>\r
+ <title>Specifying The Join Columns Explicitly</title>\r
+ <para>While it's convenient to let json_query pick the join columns, it doesn't always work.</para>\r
+ <para>For example, the actor.org_unit table has four different address ids, for four different kinds of addresses. Each of them is a foreign key to the actor.org_address table. \r
+ Json_query can't guess which one you want if you don't tell it.</para>\r
+ <para>(Actually it will try to guess. It will pick the first matching link that it finds in the IDL, which may or may not be the one you want.)</para>\r
+ <para>Here's how to define exactly which columns you want for the join:</para>\r
+<programlisting> \r
+{\r
+ "select": { "aou":[ "id" ], "aoa":[ "street1" ] },\r
+ "from": {\r
+ "aou": {\r
+ "aoa": {\r
+ "fkey":"holds_address",\r
+ "field":"id"\r
+ }\r
+ }\r
+ }\r
+}\r
+</programlisting>\r
+ <para>Before, the table we were joining was represented merely by its class name. Now it's represented by an entry in a JSON object. The key of that entry is the \r
+ class name, and the associated data is another layer of JSON object containing the attributes of the join.</para>\r
+ <para>Later we'll encounter other kinds of join attributes. For now, the only attributes that we're looking at are the ones that identify the join columns: \r
+ <quote>fkey</quote> and <quote>field</quote>. The hard part is remembering which is which:</para>\r
+ <itemizedlist>\r
+ <listitem>\r
+ <para><quote>fkey</quote> identifies the join column from the left table;</para> \r
+ </listitem>\r
+ <listitem>\r
+ <para><quote>field</quote> identifies the join column from the right table. </para> \r
+ </listitem>\r
+ </itemizedlist> \r
+ <para>When there are only two tables involved, the core table is on the left, and the non-core table is on the right. In more complex queries neither table may be the \r
+ core table.</para>\r
+ <para>Here is the result of the preceding JSON:</para>\r
+<programlisting> \r
+SELECT\r
+ "aou".id AS "id",\r
+ "aoa".street1 AS "street1"\r
+FROM\r
+ actor.org_unit AS "aou"\r
+ INNER JOIN actor.org_address AS "aoa"\r
+ ON ( "aoa".id = "aou".holds_address ) ;\r
+</programlisting>\r
+ <para>In this example the child table is on the left and the parent table is on the right. We can swap the tables if we swap the join columns as well:</para>\r
+<programlisting> \r
+{\r
+ "select": { "aou":[ "id" ], "aoa":[ "street1" ] },\r
+ "from": {\r
+ "aoa": {\r
+ "aou": {\r
+ "fkey":"id",\r
+ "field":"holds_address"\r
+ }\r
+ }\r
+ }\r
+}\r
+ \r
+SELECT\r
+ "aou".id AS "id",\r
+ "aoa".street1 AS "street1"\r
+FROM\r
+ actor.org_address AS "aoa"\r
+ INNER JOIN actor.org_unit AS "aou"\r
+ ON ( "aou".holds_address = "aoa".id ) ;\r
+</programlisting>\r
+ <para>When you specify both of the join columns, json_query assumes that you know what you're doing. It doesn't check the IDL to confirm that the join makes sense. \r
+ The burden is on you to avoid absurdities.</para>\r
+ </simplesect>\r
+ <simplesect>\r
+ <title>Specifying Only One Join Column</title>\r
+ <para>We just saw how to specify both ends of a join. It turns out that there's a shortcut -- most of the time you only need to specify one end. Consider \r
+ the following variation on the previous example:</para>\r
+<programlisting> \r
+{\r
+ "select": { "aou":[ "id" ], "aoa":[ "street1" ] },\r
+ "from": {\r
+ "aoa": {\r
+ "aou": {\r
+ "field":"holds_address"\r
+ }\r
+ }\r
+ }\r
+}\r
+</programlisting>\r
+ <para>..which results in exactly the same SQL as before.</para>\r
+ <para>Here we specified the join column from the child table, the column that is a foreign key pointing to another table. As long as that linkage is defined in the IDL, \r
+ json_query can look it up and figure out what the corresponding column is in the parent table.</para>\r
+ <para>However this shortcut doesn't work if you specify only the column in the parent table, because it would lead to ambiguities. Suppose we had specified the id \r
+ column of actor.org_address. As noted earlier, there are four different foreign keys from actor.org_unit to actor.org_address, and json_query would have no way to guess \r
+ which one we wanted.</para>\r
+ </simplesect>\r
+ <simplesect>\r
+ <title>Joining to Multiple Tables</title>\r
+ <para>So far we have joined only two tables at a time. What if we need to join one table to two different tables?</para>\r
+ <para>Here's an example:</para>\r
+<programlisting> \r
+{\r
+ "select": { "aou":[ "id" ], "aout":[ "depth" ], "aoa":[ "street1" ] },\r
+ "from": {\r
+ "aou": {\r
+ "aout":{},\r
+ "aoa": {\r
+ "fkey":"holds_address"\r
+ }\r
+ }\r
+ }\r
+}\r
+</programlisting>\r
+ <para>The first join, to actor.org_unit_type, is simple. We could have specified join columns, but we don't have to, because json_query will construct that join on the basis of \r
+ what it finds in the IDL. Having no join attributes to specify, we leave that object empty.</para>\r
+ <para>For the second join, to actor.org_address, we have to specify at least the join column in the child table, as discussed earlier. We could also have specified the join \r
+ column from the parent table, but we don't have to, so we didn't.</para>\r
+ <para>Here is the resulting SQL:</para>\r
+<programlisting> \r
+SELECT\r
+ "aou".id AS "id",\r
+ "aout".depth AS "depth",\r
+ "aoa".street1 AS "street1"\r
+FROM\r
+ actor.org_unit AS "aou"\r
+ INNER JOIN actor.org_unit_type AS "aout"\r
+ ON ( "aout".id = "aou".ou_type )\r
+ INNER JOIN actor.org_address AS "aoa"\r
+ ON ( "aoa".id = "aou".holds_address ) ;\r
+</programlisting>\r
+ <para>Since there can be only one core table, the outermost object in the FROM clause can have only one entry, whose key is the class name of the core table. The next \r
+ level has one entry for every table that's joined to the core table.</para>\r
+ </simplesect>\r
+ <simplesect>\r
+ <title>Nested Joins</title>\r
+ <para>Let's look at that last query again. It joins three tables, and the core table is the one in the middle. Can we make one of the end tables the core table instead?</para>\r
+ <para>Yes, we can:</para>\r
+<programlisting> \r
+{\r
+ "select": { "aou":[ "id" ], "aout":[ "depth" ], "aoa":[ "street1" ] },\r
+ "from": {\r
+ "aoa": {\r
+ "aou": {\r
+ "field":"holds_address",\r
+ "join": {\r
+ "aout":{ "fkey":"ou_type" }\r
+ }\r
+ }\r
+ }\r
+ }\r
+}\r
+</programlisting>\r
+ <para>The <quote>join</quote> attribute introduces another level of join. In this case "aou" is the left table for the nested join, and the right table for the original join. \r
+ Here are the results:</para>\r
+<programlisting> \r
+SELECT\r
+ "aou".id AS "id",\r
+ "aout".depth AS "depth",\r
+ "aoa".street1 AS "street1"\r
+FROM\r
+ actor.org_address AS "aoa"\r
+ INNER JOIN actor.org_unit AS "aou"\r
+ ON ( "aou".holds_address = "aoa".id )\r
+ INNER JOIN actor.org_unit_type AS "aout"\r
+ ON ( "aout".id = "aou".ou_type ) ;\r
+</programlisting>\r
+ </simplesect>\r
+ <simplesect>\r
+ <title>Outer Joins</title>\r
+ <para>By default, json_query constructs an inner join. If you need an outer join, you can add the join type as an attribute of the join:</para>\r
+ <para>Yes, we can:</para>\r
+<programlisting> \r
+{\r
+ "select": { "aou":[ "id" ], "aoa":[ "street1" ] },\r
+ "from": {\r
+ "aoa": {\r
+ "aou": {\r
+ "field":"mailing_address",\r
+ "type":"left"\r
+ }\r
+ }\r
+ }\r
+}\r
+</programlisting>\r
+ <para>Here is the resulting SQL for this example:</para>\r
+<programlisting> \r
+SELECT\r
+ "aou".id AS "id",\r
+ "aoa".street1 AS "street1"\r
+FROM\r
+ actor.org_address AS "aoa"\r
+ LEFT JOIN actor.org_unit AS "aou"\r
+ ON ( "aou".mailing_address = "aoa".id ) ;\r
+</programlisting>\r
+ </simplesect>\r
+ <simplesect>\r
+ <title>Referring to Joined Tables in the WHERE Clause</title>\r
+ <para>In the WHERE clause of the generated SQL, every column name is qualified by a table alias, which is always the corresponding class name.</para>\r
+ <para>If a column belongs to the core table, this qualification happens by default. If it belongs to a joined table, the JSON must specify what class name \r
+ to use for an alias. For example:</para>\r
+<programlisting> \r
+{\r
+ "select": { "aou":[ "id" ], "aout":[ "name" ] },\r
+ "from": {\r
+ "aout":"aou"\r
+ },\r
+ "where": {\r
+ "+aou":{ "parent_ou":2 }\r
+ }\r
+}\r
+</programlisting>\r
+ <para>Note the peculiar operator <quote>+aou</quote> -- a plus sign followed by the relevant class name. This operator tells json_query to apply the specified class to the condition that \r
+ follows. The result:</para>\r
+<programlisting> \r
+SELECT\r
+ "aou".id AS "id",\r
+ "aout".name AS "name"\r
+FROM\r
+ actor.org_unit_type AS "aout"\r
+ INNER JOIN actor.org_unit AS "aou"\r
+ ON ( "aou".ou_type = "aout".id )\r
+WHERE\r
+ ( "aou".parent_ou = 2 );\r
+</programlisting>\r
+ <para>The plus-class operator may apply to multiple conditions:</para>\r
+<programlisting> \r
+{\r
+ "select": { "aou":[ "id" ], "aout":[ "name" ] },\r
+ "from": {\r
+ "aout":"aou"\r
+ },\r
+ "where": {\r
+ "+aou":{\r
+ "parent_ou":2,\r
+ "id":{ "<":42 }\r
+ }\r
+ }\r
+}\r
+ \r
+SELECT\r
+ "aou".id AS "id",\r
+ "aout".name AS "name"\r
+FROM\r
+ actor.org_unit_type AS "aout"\r
+ INNER JOIN actor.org_unit AS "aou"\r
+ ON ( "aou".ou_type = "aout".id )\r
+WHERE\r
+ (\r
+ "aou".parent_ou = 2\r
+ AND "aou".id < 42\r
+ );\r
+</programlisting>\r
+ <para>For these artificial examples, it would have been simpler to swap the tables, so that actor.org_unit is the core table. Then you wouldn't need to go through any \r
+ special gyrations to apply the right table alias. In a more realistic case, however, you might need to apply conditions to both tables. Just swapping the tables \r
+ wouldn't solve the problem.</para>\r
+ <para>You can also use a plus-class operator to compare columns from two different tables:</para>\r
+<programlisting> \r
+{\r
+ "select": { "aou":[ "id" ], "aout":[ "name" ] },\r
+ "from": {\r
+ "aout":"aou"\r
+ },\r
+ "where": {\r
+ "depth": { ">": { "+aou":"parent_ou" } }\r
+ }\r
+}\r
+ \r
+\r
+SELECT\r
+ "aou".id AS "id",\r
+ "aout".name AS "name"\r
+FROM\r
+ actor.org_unit_type AS "aout"\r
+ INNER JOIN actor.org_unit AS "aou"\r
+ ON ( "aou".ou_type = "aout".id )\r
+WHERE\r
+ (\r
+ "aout".depth > ( "aou".parent_ou )\r
+ );\r
+</programlisting>\r
+ <para>Please don't expect that query to make any sense. It doesn't. But it illustrates the syntax.</para>\r
+ </simplesect>\r
+ <simplesect>\r
+ <title>Join Filters</title>\r
+ <para>While the above approach certainly works, the special syntax needed is goofy and awkward. A somewhat cleaner solution is to include a condition in the JOIN clause:</para>\r
+<programlisting> \r
+{\r
+ "select": { "aou":[ "id" ], "aout":[ "name" ] },\r
+ "from": {\r
+ "aout": {\r
+ "aou": {\r
+ "filter": {\r
+ "parent_ou":2\r
+ }\r
+ }\r
+ }\r
+ }\r
+} \r
+\r
+SELECT\r
+ "aou".id AS "id", "aout".name AS "name"\r
+FROM\r
+ actor.org_unit_type AS "aout"\r
+ INNER JOIN actor.org_unit AS "aou"\r
+ ON ( "aou".ou_type = "aout".id\r
+ AND "aou".parent_ou = 2 ) ;\r
+</programlisting>\r
+ <para>By default, json_query uses AND to combine the <quote>filter</quote> condition with the original join condition. If you need OR, you can use the <quote>filter_op</quote> attribute to \r
+ say so:</para>\r
+<programlisting> \r
+{\r
+ "select": { "aou":[ "id" ], "aout":[ "name" ] },\r
+ "from": {\r
+ "aout": {\r
+ "aou": {\r
+ "filter": {\r
+ "parent_ou":2\r
+ },\r
+ "filter_op":"or"\r
+ }\r
+ }\r
+ }\r
+} \r
+\r
+SELECT\r
+ "aou".id AS "id",\r
+ "aout".name AS "name"\r
+FROM\r
+ actor.org_unit_type AS "aout"\r
+ INNER JOIN actor.org_unit AS "aou"\r
+ ON ( "aou".ou_type = "aout".id\r
+ OR "aou".parent_ou = 2 ) ;\r
+</programlisting>\r
+ <para>If the data tagged by <quote>filter_op</quote> is anything but <quote>or</quote> (in upper, lower, or mixed case), json_query uses AND instead of OR.</para>\r
+ <para>The condition tagged by <quote>filter</quote> may be much more complicated. In fact it accepts all the same syntax as the WHERE clause.</para>\r
+ <para>Remember, though, that it all gets combined with the the original join condition with an AND, or with an OR if you so specify. If \r
+ you're not careful, the result may be a confusing mixture of AND and OR at the same level.</para>\r
+ </simplesect>\r
+ <simplesect>\r
+ <title>Joining to a Subquery</title>\r
+ <para>In SQL you can put a subquery in a FROM clause, and select from it as if it were a table. A JSON query has no way to do that directly. The IDL, however, \r
+ can define a class as a subquery instead of as a table. When you SELECT from it, json_query inserts the corresponding subquery into the FROM clause. For example:</para>\r
+<programlisting> \r
+{\r
+ "select":{ "iatc":[ "id", "dest", "copy_status" ] },\r
+ "from": "iatc"\r
+}\r
+</programlisting>\r
+ <para>There's nothing special-looking about this JSON, but json_query expands it as follows:</para>\r
+<programlisting> \r
+SELECT\r
+ "iatc".id AS "id",\r
+ "iatc".dest AS "dest",\r
+ "iatc".copy_status AS "copy_status"\r
+FROM\r
+ (\r
+ SELECT t.*\r
+ FROM\r
+ action.transit_copy t\r
+ JOIN actor.org_unit AS s\r
+ ON (t.source = s.id)\r
+ JOIN actor.org_unit AS d\r
+ ON (t.dest = d.id)\r
+ WHERE\r
+ s.parent_ou <> d.parent_ou\r
+ ) AS "iatc" ;\r
+</programlisting>\r
+ <para>The <quote>iatc</quote> class is like a view, except that it's defined in the IDL instead of the database. In this case it provides a way to do a join that would otherwise be \r
+ impossible through a JSON query, because it joins the same table in two different ways (see the next subsection).</para>\r
+ </simplesect>\r
+ <simplesect>\r
+ <title>Things You Can't Do</title>\r
+ <para>In a JOIN, as with other SQL constructs, there are some things that you can't do with a JSON query.</para>\r
+ <para>In particular, you can't specify a table alias, because the table alias is always the class name. As a result:</para>\r
+ <itemizedlist>\r
+ <listitem>\r
+ <para>You can't join a table to itself. For example, you can't join actor.org_unit to itself in order to select the name of the parent for every org_unit.</para> \r
+ </listitem>\r
+ <listitem>\r
+ <para>You can't join to the same table in more than one way. For example, you can't join actor.org_unit to actor.org_address through four different foreign \r
+ keys, to get four kinds of addresses in a single query.</para> \r
+ </listitem>\r
+ </itemizedlist> \r
+ <para>The only workaround is to perform the join in a view, or in a subquery defined in the IDL as described in the previous subsection.</para>\r
+ <para>Some other things, while not impossible, require some ingenuity in the use of join filters.</para>\r
+ <para>For example: by default, json_query constructs a join condition using only a single pair of corresponding columns. As long as the database is designed accordingly, \r
+ a single pair of columns will normally suffice. If you ever need to join on more than one pair of columns, you can use join filters for the extras.</para> \r
+ <para>Likewise, join conditions are normally equalities. In raw SQL it is possible (though rarely useful) to base a join on an inequality, or to use a function call in a join \r
+ condition, or to omit any join condition in order to obtain a Cartesian product. If necessary, you can devise such unconventional joins by combining the normal join \r
+ conditions with join filters.</para>\r
+ <para>For example, here's how to get a Cartesian product:</para>\r
+<programlisting> \r
+{\r
+ "select": { "aou":[ "id" ], "aout":[ "name" ] },\r
+ "from": {\r
+ "aout": {\r
+ "aou": {\r
+ "filter": {\r
+ "ou_type":{ "<>": { "+aout":"id" } }\r
+ },\r
+ "filter_op":"or"\r
+ }\r
+ }\r
+ }\r
+}\r
+ \r
+\r
+SELECT\r
+ "aou".id AS "id",\r
+ "aout".name AS "name"\r
+FROM\r
+ actor.org_unit_type AS "aout"\r
+ INNER JOIN actor.org_unit AS "aou"\r
+ ON\r
+ (\r
+ "aou".ou_type = "aout".id\r
+ OR ("aou".ou_type <> ( "aout".id ))\r
+ ) ;\r
+</programlisting>\r
+ <para>Yes, it's ugly, but at least you're not likely to do it by accident.</para>\r
+ </simplesect>\r
+ <simplesect>\r
+ <title>Selecting from Functions</title>\r
+ <para>In SQL, you can put a function call in the FROM clause. The function may return multiple columns and multiple rows. Within the query, the function behaves like a table.</para>\r
+ <para>A JSON query can also select from a function:</para>\r
+<programlisting> \r
+{\r
+ "from": [ "actor.org_unit_ancestors", 5 ]\r
+}\r
+</programlisting>\r
+ <para>The data associated with <quote>from</quote> is an array instead of a string or an object. The first element in the array specifies the name of the function. Subsequent elements, \r
+ if any, supply the parameters of the function; they must be literal values or nulls.</para>\r
+ <para>Here is the resulting query:</para>\r
+<programlisting> \r
+SELECT *\r
+FROM\r
+ actor.org_unit_ancestors( '5' ) AS "actor.org_unit_ancestors" ;\r
+</programlisting>\r
+ <para>In a JSON query this format is very limited, largely because the IDL knows nothing about the available functions. You can't join the function to a table or to \r
+ another function. If you try to supply a SELECT list or a WHERE clause, json_query will ignore it. The generated query will always select every column, via a wild card asterisk, \r
+ from every row.</para>\r
+ </simplesect>\r
+ <simplesect>\r
+ <title>The ORDER BY Clause</title>\r
+ <para>In most cases you can encode an ORDER BY clause as either an array or an object. Let's take a simple example and try it both ways. First the array:</para>\r
+<programlisting> \r
+{\r
+ "select":{ "aou":[ "name" ] },\r
+ "from": "aou",\r
+ "order_by": [\r
+ { "class":"aou", "field":"name" }\r
+ ]\r
+}\r
+</programlisting>\r
+ <para>Now the object:</para>\r
+<programlisting> \r
+{\r
+ "select":{ "aou":[ "name" ] },\r
+ "from": "aou",\r
+ "order_by": {\r
+ "aou":{ "name":{} }\r
+ }\r
+}\r
+</programlisting>\r
+ <para>The results are identical from either version:</para>\r
+<programlisting> \r
+SELECT\r
+ "aou".name AS "name"\r
+FROM\r
+ actor.org_unit AS "aou"\r
+ORDER BY\r
+ "aou".name;\r
+</programlisting>\r
+ <para>The array format is more verbose, but as we shall see, it is also more flexible. It can do anything the object format can do, plus some things that the object \r
+ format can't do.</para>\r
+ </simplesect>\r
+ <simplesect>\r
+ <title>ORDER BY as an Array</title>\r
+ <para>In the array format, each element of the array is an object defining one of the sort fields. Each such object must include at least two tags:</para>\r
+ <itemizedlist>\r
+ <listitem>\r
+ <para>The <quote>class</quote> tag provides the name of the class, which must be either the core class or a joined class.</para> \r
+ </listitem>\r
+ <listitem>\r
+ <para>The <quote>field</quote> tag provides the field name, corresponding to one of the columns of the class.</para> \r
+ </listitem>\r
+ </itemizedlist> \r
+ <para>If you want to sort by multiple fields, just include a separate object for each field.</para>\r
+ <para>If you want to sort a field in descending order, add a <quote>direction</quote> tag:</para>\r
+<programlisting> \r
+{\r
+ "select":{ "aou":[ "name" ] },\r
+ "from": "aou",\r
+ "order_by": [\r
+ {\r
+ "class":"aou",\r
+ "field":"name",\r
+ "transform":"upper"\r
+ }\r
+ ]\r
+}\r
+ \r
+\r
+SELECT\r
+ "aou".name AS "name"\r
+FROM\r
+ actor.org_unit AS "aou"\r
+ORDER BY\r
+ upper("aou".name );\r
+</programlisting>\r
+ <para>If you need additional parameters for the function, you can use the <quote>params</quote> tag to pass them:</para>\r
+<programlisting> \r
+{\r
+ "select":{ "aou":[ "name" ] },\r
+ "from": "aou",\r
+ "order_by": [\r
+ {\r
+ "class":"aou",\r
+ "field":"name",\r
+ "transform":"substr",\r
+ "params":[ 1, 8 ]\r
+ }\r
+ ]\r
+}\r
+</programlisting>\r
+ <para>The additional parameters appear as elements in an array. They may be numbers, strings, or nulls.</para>\r
+<programlisting> \r
+SELECT\r
+ "aou".name AS "name"\r
+FROM\r
+ actor.org_unit AS "aou"\r
+ORDER BY\r
+ substr("aou".name,'1','8' );\r
+</programlisting>\r
+ <para>As we have seen elsewhere, all literal values are passed as quoted strings, even if they are numbers.</para>\r
+ <para>If the function returns multiple columns, you can use the <quote>result_field</quote> tag to indicate which one you want (not shown).</para>\r
+ </simplesect>\r
+\r
+ <simplesect>\r
+ <title>ORDER BY as an Object</title>\r
+ <para>When you encode the ORDER BY clause as an object, the keys of the object are class names. Each class must be either the core class or a joined class. The data for \r
+ each class can be either an array or another layer of object. Here's an example with one of each:</para>\r
+<programlisting> \r
+{\r
+ "select":{ "aout":"id", "aou":[ "name" ] },\r
+ "from": { "aou":"aout" },\r
+ "order_by": {\r
+ "aout":[ "id" ],\r
+ "aou":{ "name":{ "direction":"desc" } }\r
+ }\r
+}\r
+</programlisting>\r
+ <para>For the <quote>aout</quote> class, the associated array is simply a list of field names (in this case, just one). Naturally, each field must reside in the class with which \r
+ it is associated.</para>\r
+ <para>However, a list of field names provides no way to specify the direction of sorting, or a transforming function. You can add those details only if the class \r
+ name is paired with an object, as in the example for the "aou" class. The keys for such an object are field names, and the associated tags define other details.</para>\r
+ <para>In this example, we use the <quote>direction"</quote> tag to specify that the name field be sorted in descending order. This tag works the same way here as described earlier. \r
+ If the associated string starts with "D" or "d", the sort will be descending; otherwise it will be ascending.</para>\r
+ <para>Here is the resulting SQL:</para>\r
+<programlisting> \r
+SELECT\r
+ "aou".name AS "name"\r
+FROM\r
+ actor.org_unit AS "aou"\r
+ INNER JOIN actor.org_unit_type AS "aout"\r
+ ON ( "aout".id = "aou".ou_type )\r
+ORDER BY\r
+ "aout".id,\r
+ "aou".name DESC;\r
+</programlisting>\r
+<programlisting>\r
+{\r
+ "select":{ "aou":[ "name", "id" ] },\r
+ "from": "aou",\r
+ "order_by": {\r
+ "aou":{\r
+ "name":{ "transform":"substr", "params":[ 1, 8 ] }\r
+ }\r
+ }\r
+} \r
+\r
+SELECT\r
+ "aou".name AS "name",\r
+ "aou".id AS "id"\r
+FROM\r
+ actor.org_unit AS "aou"\r
+ORDER BY\r
+ substr("aou".name,'1','8' );\r
+</programlisting>\r
+ </simplesect>\r
+ <simplesect>\r
+ <title>Things You Can't Do</title>\r
+ <para>If you encode the ORDER BY clause as an object, you may encounter a couple of restrictions.</para>\r
+ <para>Because the key of such an object is the class name, all the fields from a given class must be grouped together. You can't sort by a column from one table, followed by \r
+ a column from another table, followed by a column from the first table. If you need such a sort, you must encode the ORDER BY clause in the array format, which suffers \r
+ from no such restrictions.</para>\r
+ <para>For similar reasons, with an ORDER BY clause encoded as an object, you can't reference the same column more than once. Although such a sort may seem perverse, \r
+ there are situations where it can be useful, provided that the column is passed to a transforming function.</para>\r
+ <para>For example, you might want a case-insensitive sort, except that for any given letter you want lower case to sort first. For example, you want <quote>diBona</quote> to sort \r
+ before <quote>Dibona</quote>. Here's a way to do that, coding the ORDER BY clause as an array:</para>\r
+<programlisting> \r
+{\r
+ "select":{ "au":[ "family_name", "id" ] },\r
+ "from": "au",\r
+ "order_by": [\r
+ { "class":"au", "field":"family_name", "transform":"upper" },\r
+ { "class":"au", "field":"family_name" }\r
+ ]\r
+}\r
+SELECT\r
+ "au".family_name AS "family_name",\r
+ "au".id AS "id"\r
+FROM\r
+ actor.usr AS "au"\r
+ORDER BY\r
+ upper("au".family_name ),\r
+ "au".family_name;\r
+</programlisting>\r
+ <para>Such a sort is not possible where the ORDER BY clause is coded as an object.</para>\r
+ </simplesect>\r
+ <simplesect>\r
+ <title>The GROUP BY Clause</title>\r
+ <para>A JSON query has no separate construct to define a GROUP BY clause. Instead, the necessary information is distributed across the SELECT clause. However, \r
+ the way it works is a bit backwards from what you might expect, so pay attention.</para>\r
+ <para>Here's an example:</para>\r
+<programlisting> \r
+{\r
+ "select": {\r
+ "aou": [\r
+ { "column":"parent_ou" },\r
+ { "column":"name", "transform":"max", "aggregate":true }\r
+ ]\r
+ },\r
+ "from": "aou"\r
+}\r
+</programlisting>\r
+ <para>The <quote>transform</quote> tag is there just to give us an excuse to do a GROUP BY. What's important to notice is the <quote>aggregate</quote> tag.</para>\r
+ <para>Here's the resulting SQL:</para>\r
+<programlisting> \r
+SELECT\r
+ "aou".parent_ou AS "parent_ou",\r
+ max("aou".name ) AS "name"\r
+FROM\r
+ actor.org_unit AS "aou"\r
+GROUP BY\r
+ 1;\r
+</programlisting>\r
+ <para>The GROUP BY clause references fields from the SELECT clause by numerical reference, instead of by repeating them. Notice that the field it references, \r
+ parent_ou, is the one that doesn't carry the <quote>aggregate</quote> tag in the JSON.</para>\r
+ <para>Let's state that more generally. The GROUP BY clause includes only the fields that do not carry the <quote>aggregate</quote> tag (or that carry it with a value of false).</para>\r
+ <para>However, that logic applies only when some field somewhere does carry the <quote>aggregate</quote> tag, with a value of true. If there is no <quote>aggregate</quote> tag, or \r
+ it appears only with a value of false, then there is no GROUP BY clause.</para>\r
+ <para>If you really want to include every field in the GROUP BY clause, don't use <quote>aggregate</quote>. Use the <quote>distinct</quote> tag, as described in the next section.</para>\r
+ </simplesect>\r
+ <simplesect>\r
+ <title>The DISTINCT Clause</title>\r
+ <para>JSON queries don't generate DISTINCT clauses. However, they can generate GROUP BY clauses that include every item from the SELECT clause. The effect is the same as \r
+ applying DISTINCT to the entire SELECT clause.</para>\r
+ <para>For example:</para>\r
+<programlisting> \r
+{\r
+ "select": {\r
+ "aou": [\r
+ "parent_ou",\r
+ "ou_type"\r
+ ]\r
+ },\r
+ "from":"aou",\r
+ "distinct":"true"\r
+}\r
+</programlisting>\r
+ <para>Note the <quote>distinct</quote> entry at the top level of the query object, with a value of <quote>true</quote>.</para>\r
+<programlisting> \r
+SELECT\r
+ "aou".parent_ou AS "parent_ou",\r
+ "aou".ou_type AS "ou_type"\r
+FROM\r
+ actor.org_unit AS "aou"\r
+GROUP BY\r
+ 1, 2;\r
+</programlisting>\r
+ <para>The generated GROUP BY clause references every column in the SELECT clause by number.</para>\r
+ </simplesect>\r
+ <simplesect>\r
+ <title>The HAVING Clause</title>\r
+ <para>For a HAVING clause, add a <quote>having</quote> entry at the top level of the query object. For the associated data, you can use all the same syntax \r
+ that you can use for a WHERE clause.</para>\r
+ <para>Here's a simple example:</para>\r
+<programlisting> \r
+{\r
+ "select": {\r
+ "aou": [\r
+ "parent_ou", {\r
+ "column":"id",\r
+ "transform":"count",\r
+ "alias":"id_count",\r
+ "aggregate":"true"\r
+ }\r
+ ]\r
+ },\r
+ "from":"aou",\r
+ "having": {\r
+ "id": {\r
+ ">" : {\r
+ "transform":"count",\r
+ "value":6\r
+ }\r
+ }\r
+ }\r
+}\r
+</programlisting>\r
+ <para>We use the <quote>aggregate</quote> tag in the SELECT clause to give us a GROUP BY to go with the HAVING. Results:</para>\r
+<programlisting> \r
+SELECT\r
+ "aou".parent_ou AS "parent_ou",\r
+ count("aou".id ) AS "id_count"\r
+FROM\r
+ actor.org_unit AS "aou"\r
+GROUP BY\r
+ 1\r
+HAVING\r
+ count("aou".id ) > 6 ;\r
+</programlisting>\r
+ <para>In raw SQL we could have referred to <quote>count( 1 )</quote>. But since JSON queries cannot encode arbitrary expressions, we applied the count function to a column that \r
+ cannot be null.</para>\r
+ </simplesect>\r
+ <simplesect>\r
+ <title>The LIMIT and OFFSET Clauses</title>\r
+ <para>To add an LIMIT or OFFSET clause, add an entry to the top level of a query object. For example:</para>\r
+<programlisting> \r
+{\r
+ "select": {\r
+ "aou": [ "id", "name" ]\r
+ },\r
+ "from":"aou",\r
+ "order_by": { "aou":[ "id" ] },\r
+ "offset": 7,\r
+ "limit": 42\r
+}\r
+</programlisting>\r
+ <para>The data associated with <quote>offset</quote> and <quote>limit</quote> may be either a number or a string, but if it's a string, it should have a number inside.</para>\r
+ <para>Result:</para>\r
+<programlisting> \r
+SELECT\r
+ "aou".id AS "id",\r
+ "aou".name AS "name"\r
+FROM\r
+ actor.org_unit AS "aou"\r
+ORDER BY\r
+ "aou".id\r
+LIMIT 42 \r
+OFFSET 7;\r
+</programlisting>\r
+ </simplesect>\r
+</chapter>\r
--- /dev/null
+<?xml version="1.0" encoding="utf-8"?>\r
+<chapter xml:id="supercat" xmlns="http://docbook.org/ns/docbook" version="5.0" xml:lang="EN"\r
+ xmlns:xi="http://www.w3.org/2001/XInclude" xmlns:xlink="http://www.w3.org/1999/xlink">\r
+ <info>\r
+ <title>SuperCat</title>\r
+ </info>\r
+ <section xml:id="usingsupercat">>\r
+ <title>Using SuperCat</title> \r
+ <indexterm><primary>SuperCat</primary></indexterm>\r
+ <para>SuperCat allows Evergreen record and information retrieval from a web browser using a based on a number of open web standards and formats. The following record types are \r
+ supported:</para>\r
+ <itemizedlist>\r
+ <listitem>isbn</listitem>\r
+ <listitem>metarecord</listitem>\r
+ <listitem>record</listitem>\r
+ </itemizedlist>\r
+ <simplesect>\r
+ <title>Return a list of ISBNs for related records</title>\r
+ <indexterm><primary>SuperCat</primary><secondary>ISBNs</secondary></indexterm>\r
+ <para>Similar to the OCLC xISBN service, Evergreen can return a list of related records based on its oISBN algorithm:</para>\r
+ <para><uri>http://<hostname>/opac/extras/osibn/<ISBN></uri></para>\r
+ <para>For example, <uri>http://dev.gapines.org/opac/extras/oisbn/0439136350</uri> returns:</para>\r
+<screen>\r
+<idlist metarecord="302670">\r
+<isbn record="250060">0790783525</isbn>\r
+<isbn record="20717">0736691316</isbn>\r
+<isbn record="250045">0790783517</isbn>\r
+<isbn record="199060">9500421151</isbn>\r
+<isbn record="250061">0790783495</isbn>\r
+<isbn record="154477">0807286028</isbn>\r
+<isbn record="227297">1594130027</isbn>\r
+<isbn record="26682">0786222743</isbn>\r
+<isbn record="17179">0807282316</isbn>\r
+<isbn record="34885">0807282316</isbn>\r
+<isbn record="118019">8478885196</isbn>\r
+<isbn record="1231">0738301477</isbn>\r
+</idlist>\r
+</screen>\r
+ </simplesect>\r
+ <simplesect>\r
+ <title>Return records</title>\r
+ <indexterm><primary>SuperCat</primary><secondary>records</secondary></indexterm>\r
+ <para>SuperCat can return records and metarecords in many different formats (see <xref linkend='supportedsupercatformats' /></para>\r
+ <para><uri>http://<hostname>/opac/extras/supercat/retrieve/<format>/<record-type>/<bib-ID></uri></para>\r
+ <para>For example, <uri>http://dev.gapines.org/opac/extras/supercat/retrieve/mods/record/555</uri> returns:</para>\r
+<screen>\r
+<mods:modsCollection version="3.0">\r
+ <mods:mods xsi:schemaLocation="http://www.loc.gov/mods/ http://www.loc.gov/standards/mods/mods.xsd">\r
+ <titleInfo>\r
+ <title>More Brer Rabbit stories /</title>\r
+ </titleInfo>\r
+ <typeOfResource>text</typeOfResource>\r
+ <originInfo>\r
+ <place>\r
+ <code authority="marc">xx</c0de>\r
+ </place>\r
+ <publisher>Award Publications</publisher>\r
+ <dateIssued>c1982, 1983</dateIssued>\r
+ <dateIssued encoding="marc" point="start">1983</dateIssued>\r
+ <dateIssued encoding="marc" point="end">1982</dateIssued>\r
+ <issuance>monographic</issuance>\r
+ </originInfo>\r
+ <language authority="iso639-2b">eng</language>\r
+ <physicalDescription>\r
+ <form authority="marcform">print</form>\r
+ <extent>unp. : col. ill.</extent>\r
+ </physicalDescription>\r
+ <note type="statement of responsibility">ill. by Rene Cloke.</note>\r
+ <subject authority="lcsh">\r
+ <topic>Animals</topic>\r
+ <topic>Fiction</topic>\r
+ </subject>\r
+ <subject authority="lcsh">\r
+ <topic>Fables</topic>\r
+ </subject>\r
+ <recordInfo>\r
+ <recordContentSource>(BRO)</recordContentSource>\r
+ <recordCreationDate encoding="marc">930903</recordCreationDate>\r
+ <recordChangeDate encoding="iso8601">19990703024637.0</recordChangeDate>\r
+ <recordIdentifier>PIN60000007 </recordIdentifier>\r
+ </recordInfo>\r
+ </mods:mods>\r
+</mods:modsCollection>\r
+</screen>\r
+ </simplesect>\r
+ <simplesect>\r
+ <title>Return a feed of recently edited or created records</title>\r
+ <indexterm><primary>SuperCat</primary><secondary>recent records</secondary></indexterm>\r
+ <para>SuperCat can return feeds of recently edited or created authority and bibliographic records:</para>\r
+ <para><uri>http://<hostname>/opac/extras/feed/freshmeat/<feed-type>/[authority|biblio]/[import|edit]/<limit>/<date></uri></para>\r
+ <para>The limit records imported or edited following the supplied date will be returned. If you do not supply a date, then the most recent limit records will be returned.</para>\r
+ <para>If you do not supply a limit, then up to 10 records will be returned.</para> \r
+ <para>Feed-type can be one of atom, html, htmlholdings, marcxml, mods, mods3, or rss2.</para> \r
+ <para><uri>For example, http://dev.gapines.org/opac/extras/feed/freshmeat/atom/biblio/import/10/2008-01-01</uri></para>\r
+ </simplesect>\r
+ <simplesect>\r
+ <title>Browse records</title>\r
+ <para>SuperCat can browse records in HTML and XML formats:</para>\r
+ <para><uri>http://<hostname>/opac/extras/supercat/browse/<format>/call_number/<org_unit>/<call_number></uri></para>\r
+ <para>For example, <uri>http://dev.gapines.org/opac/extras/browse/xml/call_number/-/GV</uri> returns:</para>\r
+<screen>\r
+<hold:volumes xmlns:hold='http://open-ils.org/spec/holdings/v1'>\r
+ <hold:volume id="tag:open-ils.org,2008:asset-call_number/130607" lib="FRRLS-FA" label="GUTCHEON BETH">\r
+ <act:owning_lib id="tag:open-ils.org,2008:actor-org_unit/111" name="Fayette County Public Library"/>\r
+ <record xsi:schemaLocation="http://www.loc.gov/MARC21/slim http://www.loc.gov/ \r
+ standards/marcxml/schema/MARC21slim.xsd"\r
+ id="tag:open-ils.org,2008:biblio-record_entry/21669/FRRLS-FA">\r
+ <leader>09319pam a2200961 a 4500</leader>\r
+ <controlfield tag="001"/>\r
+ <controlfield tag="005">20000302124754.0</controlfield>\r
+ <controlfield tag="008">990817s2000 nyu 000 1 eng </controlfield>\r
+ <datafield tag="010" ind1=" " ind2=" ">\r
+ <subfield code="a"> 99045936</subfield>\r
+ </datafield>\r
+ ..\r
+ </record>\r
+ <record>\r
+ ..\r
+ </record>\r
+ </hold:volume>\r
+</hold:volumes> \r
+</screen>\r
+ </simplesect>\r
+ <simplesect xml:id="supportedsupercatformats">\r
+ <title>Supported formats</title>\r
+ <indexterm><primary>SuperCat</primary><secondary>formats</secondary></indexterm>\r
+ <para>SuperCat maintains a list of supported formats for records and metarecords:</para>\r
+ <para><uri>http://<hostname>/opac/extras/supercat/formats/<record-type></uri></para>\r
+ <para>For example, <uri>http://dev.gapines.org/opac/extras/supercat/formats/record</uri> returns:</para>\r
+<screen>\r
+<formats>\r
+ <format>\r
+ <name>opac</name>\r
+ <type>text/html</type>\r
+ </format>\r
+ <format>\r
+ <name>htmlholdings</name>\r
+ <type>text/html</type>\r
+ </format>\r
+...\r
+</screen>\r
+ </simplesect>\r
+ </section>\r
+ <section xml:id="addingsupercatformats">\r
+ <title>Adding new SuperCat Formats</title>\r
+ <indexterm><primary>SuperCat</primary><secondary>formats</secondary><tertiary>adding</tertiary></indexterm>\r
+ <para>Adding SuperCat formats requires experience editing XSL files and familiarity with XML and Perl.</para> \r
+ <para>SuperCat web services are based on the OpenSRF service, <systemitem class="service">>open-ils.supercat</systemitem>.</para> \r
+ <para>Developers are able to add new formats by adding the <emphasis>xsl</emphasis> stylesheet for the format. By default, the location of the stylesheets is <filename class="directory">/openils/var/xsl/</filename>. You must also add the feed to the perl \r
+ modules <filename>openils/lib/perl5/OpenILS/WWW/SuperCat/feed.pm</filename> and <filename>openils/lib/perl5/OpenILS/WWW/SuperCat.pm</filename>. An Evergreen restart is \r
+ required for the feed to be activated.</para>\r
+ <tip><para>Use an existing xsl stylesheet and Perl module entry as a template for your new format.</para></tip>\r
+ </section>\r
+ <section xml:id="editingsupercatformats">\r
+ <title>Customizing SuperCat Formats</title>\r
+ <indexterm><primary>SuperCat</primary><secondary>formats</secondary><tertiary>customizing</tertiary></indexterm> \r
+ <para>Editing SuperCat formats requires experience editing XSL files and familiarity with XML..</para> \r
+ <para>It is possible to customize existing supercat formats using XSL stylesheets. You are able to change the content to be displayed and the design of the pages.</para> \r
+ <para>In order to change the display of a specific format, edit the corresponding XSL file(s) for the particular format. The default location for the XSL stylesheets is \r
+ <filename class="directory">/openils/var/xsl/</filename>. </para>\r
+ </section>\r
+</chapter>\r
+\r
<title>Development</title>\r
</info>\r
<xi:include href="development/development_intro.xml"/>\r
+ <xi:include href="development/directoriesandFiles.xml"/> \r
+ <xi:include href="development/customizingstaffclient.xml"/>\r
+ <xi:include href="development/customize_opac.xml"/>\r
+ <xi:include href="development/OpenSRF_intro.xml"/>\r
+ <xi:include href="development/datamodelsandaccess.xml"/>\r
+ <xi:include href="development/introduction_to_sql.xml"/>\r
+ <xi:include href="development/json.xml"/> \r
+ <xi:include href="development/supercat.xml"/>\r
+\r
</part>\r
<part xml:id="Appendices">\r
<info>\r
projects / working / Evergreen.git / commitdiff