From 8bf1f95aee67cd88e1ac0d6dca5380952ad404a9 Mon Sep 17 00:00:00 2001 From: Galen Charlton Date: Thu, 27 Feb 2014 16:53:26 -0800 Subject: [PATCH] change log for 2.3.0-beta Signed-off-by: Galen Charlton --- ChangeLog | 1604 +++++++++++++++++++++++-------------------------------------- 1 file changed, 597 insertions(+), 1007 deletions(-) diff --git a/ChangeLog b/ChangeLog index f4eaa05..5332cac 100644 --- a/ChangeLog +++ b/ChangeLog @@ -1,1324 +1,914 @@ ChangeLog ========= -OpenSRF 2.1.2 +OpenSRF 2.3.2 ------------- -commit b0dd15b964f5856059d0fd40981f8985ef8ea56f -Author: Dan Scott -Date: Sun Nov 18 07:41:46 2012 -0500 +commit d6688eade4a5947efd5280768fc549942a22e451 +Author: Galen Charlton +Date: Thu Feb 27 16:52:15 2014 -0800 - Bump version numbers for 2.1.2 + bump up version numbers of OpenSRF 2.3.0-beta - Signed-off-by: Dan Scott + Signed-off-by: Galen Charlton 2 2 src/perl/lib/OpenSRF.pm +1 1 src/python/setup.py 1 1 version.m4 -commit 07ab6dd958a905cb439f9cded9ae2828b7888a2f -Author: Dan Scott -Date: Fri Nov 16 10:36:28 2012 -0500 +commit e4453fbb43032f1154e1aa6dfdaf6183682026d9 +Author: Galen Charlton +Date: Thu Feb 27 16:46:46 2014 -0800 - Whitespace in OpenSRF/System.pm - - Spaces not tabs. + release notes for OpenSRF 2.3.0 beta - Signed-off-by: Dan Scott + Signed-off-by: Galen Charlton -21 21 src/perl/lib/OpenSRF/System.pm +90 0 doc/RELEASE_NOTES.txt + create mode 100644 doc/RELEASE_NOTES.txt -commit 5efbb97cbeb87d37b4f64bcef8babeab5e5c54b1 -Author: Jason Stephenson -Date: Thu Nov 15 12:20:11 2012 -0500 +commit 3692bb33cccdf3106df96883033b20ab9170c5a5 +Author: Bill Erickson +Date: Mon Feb 24 09:09:46 2014 -0500 - Move setting of OpenSRF::Application::shared_conf to load_bootstrap_config. + LP#1284137: Avoid WARN logging on router shutdown - shared_conf was not available and broke clients that were not in a - service, such as the Evergreen CStoreEditor. + Log received signals as DEBUG (misc signal) or INFO (during shutdown) + in the router, since these are not error conditions, but normal + operations. - Signed-off-by: Jason Stephenson - Signed-off-by: Dan Scott + A warning may ultimately be logged at a higher level when the signal is + unexpected. + + Signed-off-by: Bill Erickson + Signed-off-by: Galen Charlton -8 9 src/perl/lib/OpenSRF/System.pm +1 1 src/router/osrf_router.c +2 2 src/router/osrf_router_main.c -commit 2cb5313d0d5bf3e2b912ebbcc653cedcac22336d -Author: Dan Scott -Date: Tue Oct 30 23:39:25 2012 -0400 +commit 02a93da43f4fcebf7b00e5893ecd3e855dc2dbba +Author: Galen Charlton +Date: Fri Mar 15 10:22:17 2013 -0400 - Bump release numbers, create 2.1.1 change log - - Signed-off-by: Dan Scott + LP#1155446: add Debian/Ubuntu deps required for Python tests to pass - Conflicts: - ChangeLog + Signed-off-by: Galen Charlton + Signed-off-by: Ben Shum -1243 1 ChangeLog -2 2 src/perl/lib/OpenSRF.pm -1 1 version.m4 +5 0 src/extras/Makefile.install -commit 42e369b2efcbbc31447c43b5fbaa3999ad88cf30 +commit 2fae35660282044e8462cde5b8152ff3d24f77e3 Author: Bill Erickson -Date: Mon May 21 16:42:12 2012 -0400 +Date: Fri Oct 12 16:57:15 2012 -0400 - ChopChop is dead, long live ChopChop. + LP#1066131: srfsh.py should not require opensrf.settings - It was never fully baked, nor was it secure, nor does it compile cleanly - with -Wall, nor does it provide any useful examples for working with - opensrf, etc. Time to clear out. So long, chopchop. + Update srfsh.py to connect to the opensrf network without automatically + reading the opensrf.setting server data. This allows srfsh.py to run a + non-private network. Signed-off-by: Bill Erickson - Signed-off-by: Dan Scott - -0 6 Makefile.am -0 21 configure.ac -0 13 examples/opensrf_core.xml.example -0 4 src/Makefile.am -0 23 src/jserver/Makefile.am -0 1082 src/jserver/osrf_chat.c -0 45 src/jserver/osrf_chat.h -0 92 src/jserver/osrf_chat_main.c - delete mode 100644 src/jserver/Makefile.am - delete mode 100644 src/jserver/osrf_chat.c - delete mode 100644 src/jserver/osrf_chat.h - delete mode 100644 src/jserver/osrf_chat_main.c - -OpenSRF 2.1.1 -------------- - -commit e93c7c932b144aac2633fe5d058f298a29909f02 -Author: Dan Scott -Date: Tue Oct 30 23:34:04 2012 -0400 - - Release notes for 2.1.1: input log redaction - - Signed-off-by: Dan Scott + Signed-off-by: Galen Charlton -12 3 doc/RELEASE_NOTES.txt +1 1 src/python/srfsh.py -commit fd367d0a978cd85d2726a1ea2bd0412abf57fb12 -Author: Dan Scott -Date: Tue Oct 30 23:27:34 2012 -0400 +commit 989ac2737759a0fc6a80ec08b1a68b35eab2ff60 +Author: Galen Charlton +Date: Thu Apr 18 16:14:00 2013 -0400 - Purge the wildly out-of-date Roadmap document + LP#1170484: log failure to load library for C app as an error - Signed-off-by: Dan Scott - -0 51 doc/Roadmap.txt - delete mode 100644 doc/Roadmap.txt + Signed-off-by: Galen Charlton + Signed-off-by: Ben Shum -commit 97a520bce382c8806ad1772f98e1e9169a2297b9 -Author: Dan Scott -Date: Tue Nov 6 12:32:48 2012 -0500 +1 1 src/libopensrf/osrf_application.c - Bump libopensrf version-info revision and age - - The log redaction functionality changed the source files (thus 'age' - gets bumped) and is backwards-compatible (thus 'revision' gets bumped) - but maintains the same interface (thus 'current' stays the same). - - Signed-off-by: Dan Scott +commit 9a01063b3c78a33707a414a36d397e061ca62b1d +Merge: d90a415 2cc896b +Author: Galen Charlton +Date: Fri Jan 31 16:40:58 2014 -0500 -1 1 src/libopensrf/Makefile.am + Merge branch 'rel_2_3' -commit b81a8c2ada734fe8c47e758c681e2bb952f29c39 -Author: Dan Wells -Date: Wed Oct 31 10:21:14 2012 -0400 +commit 2cc896b7d103fbade96054881c6c57046eef01e5 +Author: Galen Charlton +Date: Fri Jan 31 16:20:26 2014 -0500 - Protect against empty/invalid log_protect sections - - Connectivity would fail if the log_protect section of opensrf_core.xml - was empty, or contained only comments, as it does in the default - example. Add a simple guard against this potential problem. This fix - will also protect against cases where the 'log_protect' section - contains text rather than separate configuration elements. + fix typo and remove repeated word - Credit to Dan Scott for finding the bug and working out how to fix it. - - Signed-off-by: Dan Wells - Signed-off-by: Dan Scott + Signed-off-by: Galen Charlton -6 4 src/perl/lib/OpenSRF/Application.pm +3 3 bin/opensrf-perl.pl.in -commit a2a286ef7fbf70e695ab23289fe9f131a9037922 +commit a98d4b5195a0f5e77d14d4d247dde47f71b573c2 Author: Bill Erickson -Date: Wed Oct 17 15:57:14 2012 -0400 - - Consolidate duplicate osrfMethodVerifyContext handler - - Log redaction was not occuring in some cases because the code was using - the OSRF_METHOD_VERIFY_CONTEXT macro instead of the similarly named - function which does the same thing. This change points the macro at the - function so that all code uses the same underlying code. - - Note this change turns on CALL param logging unconditionally, whereas - users of the macro would previously have been able to avoid CALL logging - via the OSRF_LOG_PARAMS variable. In practice, little code uses the - macro and all code enables OSRF_LOG_PARAMS. If we need to add this - control back, it can be added directly to osrfMethodVerifyContext(). - For now, it's one less env variable we need to define. +Date: Tue Jan 7 14:06:06 2014 -0500 + + osrf_control router-de/re-register and reload commands + + Added support for the following new command line options: + + --router-de-register + --router-de-register-all + Sends a SIGUSR1 signal to the selected service(s), which causes each + service's listener process to send an "unregister" command to all + registered routers. The --all variant sends the signal to all + running listeners. The non-(--all) variant requires a --service. + + --router-re-register + --router-re-register-all + Sends a SIGUSR2 signal to the selected service(s), which causes each + service's listener process to send a "register" command to all + configured routers routers. The --all variant sends the signal to + all running listeners. The non-(--all) variant requires a --service. + + --reload + --reload-all + Sends a SIGHUP signal to the selected service(s). SIGHUP causes + each Listener process to reload its opensrf_core.xml config file + and gracefully re-launch drone processes. The -all variant sends + the signal to all services. The non-(-all) variant requires a + --service. Signed-off-by: Bill Erickson - Signed-off-by: Dan Scott + Signed-off-by: Galen Charlton -2 36 include/opensrf/osrf_application.h +46 3 bin/opensrf-perl.pl.in -commit 157ad5d6553b29585afd4b2180db91e2d13b5961 -Author: Dan Wells -Date: Fri Oct 12 09:45:35 2012 -0400 +commit d90a415f0c98a62560d9f824da4b18b6ee9eba84 +Author: Galen Charlton +Date: Fri Jan 31 16:25:16 2014 -0500 - Log redaction for sensitive input values, Perl side - - This commit attempts to do the same as the C log redaction fix, - but now at the Perl level. The Perl configuration code was a - little more crufty than the C side, so an additional feature was - added to Config.pm to support the new 'shared' section. At some - point we should consider a ground-up rewrite of Config.pm, as the - code seems to suffer some from its INI file roots. + x - Signed-off-by: Dan Wells - Signed-off-by: Dan Scott + Signed-off-by: Galen Charlton -3 3 examples/opensrf_core.xml.example -21 1 src/perl/lib/OpenSRF/Application.pm -9 0 src/perl/lib/OpenSRF/System.pm -48 12 src/perl/lib/OpenSRF/Utils/Config.pm +2 2 bin/opensrf-perl.pl.in -commit fe9617eda262cc3a3890d068e938ef0ffb99d07a -Author: Dan Wells -Date: Wed Oct 10 17:28:07 2012 -0400 +commit 0c0b02e0e8252d5a3e63dea03f8b8d8519526d5a +Author: Galen Charlton +Date: Fri Jan 31 16:20:26 2014 -0500 - Log redaction for sensitive input values, C side - - Some service/methods deal with sensitive information (passwords, - financial, etc.). All input values (e.g. gateway method params) - are currently logged in the activity log regardless of your log - level. This commit will allow you to redact the params of any - method which matches a configurable set of left-anchored string - values. - - This commit adds the initial config, and covers redaction of method - parameters sent through the gateway, the translator, and more general - OpenSRF C applications. + typo fix - Signed-off-by: Dan Wells - Signed-off-by: Dan Scott - -12 0 examples/opensrf_core.xml.example -1 0 include/opensrf/osrf_application.h -2 0 include/opensrf/osrf_system.h -25 10 src/gateway/osrf_http_translator.c -22 7 src/gateway/osrf_json_gateway.c -23 4 src/libopensrf/osrf_application.c -6 0 src/libopensrf/osrf_system.c + Signed-off-by: Galen Charlton -commit cd24bb1c94c95027310f63909b692e4dbb05507f -Author: Dan Scott -Date: Mon Oct 22 11:50:21 2012 -0400 +1 1 bin/opensrf-perl.pl.in - Use apt-get instead of aptitude on Debian / Ubuntu - - Reports from the field state that aptitude is not installed by default on - Ubuntu any longer; thus go with the sure thing. +commit 5b3f8046bf05191424425cbe6bad5342ccb9ebbf +Author: Bill Erickson +Date: Tue Jan 7 14:06:06 2014 -0500 + + osrf_control router-de/re-register and reload commands + + Added support for the following new command line options: + + --router-de-register + --router-de-register-all + Sends a SIGUSR1 signal to the selected service(s), which causes each + service's listener process to send an "unregister" command to all + registered routers. The --all variant sends the signal to all + running listeners. The non-(--all) variant requires a --service. + + --router-re-register + --router-re-register-all + Sends a SIGUSR2 signal to the selected service(s), which causes each + service's listener process to send a "register" command to all + configured routers routers. The --all variant sends the signal to + all running listeners. The non-(--all) variant requires a --service. + + --reload + --reload-all + Sends a SIGHUP signal to the selected service(s). SIGHUP causes + each Listener process to reload its opensrf_core.xml config file + and gracefully re-launch drone processes. The -all variant sends + the signal to all services. The non-(-all) variant requires a + --service. - Signed-off-by: Dan Scott + Signed-off-by: Bill Erickson + Signed-off-by: Galen Charlton -1 1 README -1 1 src/extras/Makefile.install +46 3 bin/opensrf-perl.pl.in -commit 20a115a421c56b56968accc27b63da7656a70ee8 -Author: Dan Scott -Date: Mon Oct 22 11:53:24 2012 -0400 +commit 14287dc536142b14bc67806b1417b9a4850fa8cc +Author: Bill Erickson +Date: Thu Nov 21 16:37:38 2013 -0500 - Remove the reference to Evergreen in the README + recover osrf_control router start - Stephen Wills reported problems installing Evergreen because of missing - dependencies, and reading the OpenSRF README shows that it refers to the - Evergreen prerequisite installer in a confusing way that might lead to - users using only the Evergreen Makefile.install, rather than using the - OpenSRF Makefile.install. + With the addition of service verification, which ensures that a service + is meant to run on the requested host, we lost the ability to start the + router by itself, since the router is never explicitly configured to run + anywhere. This patch assumes the router is always a valid service. - In addition, Fedora 16 is going to be out of support soon, and Fedora 17 - and 18 are well-tested targets now. - - Signed-off-by: Dan Scott + Signed-off-by: Bill Erickson + Signed-off-by: Dan Wells -2 2 README +1 0 bin/opensrf-perl.pl.in -commit 83dd58abfca26418f8bdaecb73dbd2c177cc0093 -Author: Jason Stephenson -Date: Tue Nov 6 09:16:18 2012 -0500 +commit b59aee460f2b84f94ecabb758051cf00209574e6 +Author: Bill Erickson +Date: Thu Nov 21 16:37:38 2013 -0500 - Fix installation on Ubuntu Precise. + recover osrf_control router start - Remove libreadline5-dev and replace with libreadline-dev for Ubuntu - and Debian. This installs libreadline6, but everything still works - since there's nothing version specific in our use of libreadline. + With the addition of service verification, which ensures that a service + is meant to run on the requested host, we lost the ability to start the + router by itself, since the router is never explicitly configured to run + anywhere. This patch assumes the router is always a valid service. - Signed-off-by: Jason Stephenson - Signed-off-by: Dan Scott - -2 3 src/extras/Makefile.install + Signed-off-by: Bill Erickson + Signed-off-by: Dan Wells -OpenSRF 2.1.0 -------------- +1 0 bin/opensrf-perl.pl.in -commit 79694c390e0350a199be409ceadb19945ddf2b5c -Author: Dan Scott -Date: Wed May 30 23:12:32 2012 -0400 +commit c5ca743810acded29b0dcf9866ea513d5fcffdb9 +Author: Bill Erickson +Date: Thu Dec 5 12:27:44 2013 -0500 - No, Ubuntu Hardy is not well-tested for 2.1.0 + OpenSRF client disconnect robustification (Perl) - One more reference to Ubuntu Hardy that we can remove from the - README/INSTALL. + * Improve detection of client connection state with jabber server - Signed-off-by: Dan Scott - -0 1 README - -commit 885a4d87cd06f31071ff9b2f728fd7aa1c83d7be -Author: Dan Scott -Date: Wed May 30 23:11:14 2012 -0400 - - Generate ChangeLog for 2.1.0 release + Be more aggressive in testing and responding to failed connectivity. + It can happen in various ways, depending on what's happening to the + jabber server (death, delay, graceful shutdown). With these changes + we attempt to cover all the bases and detect as early as possible that + a connection has died, specifically when trying to create a new + connection after one has failed. - Signed-off-by: Dan Scott - -821 59 ChangeLog - -commit 3eb7c54a734369a7bffc4a3c30bef2e65573be52 -Author: Dan Scott -Date: Sun May 27 22:52:23 2012 -0400 - - 2.1.0 release notes + * Do a better job of cleaning up dead connections - A summary of some of the highlights of the new release, for - functionality that has been added and deleted. Does not address bug - fixes such as the much more robust Perl child handling; perhaps - we should focus on those as well? + Transport::PeerHandle now has a reset() action which disconnects (when + possible) and clears the global connection reference. - Signed-off-by: Dan Scott - -89 0 doc/RELEASE_NOTES.txt - create mode 100644 doc/RELEASE_NOTES.txt - -commit cdd0816c624ff0843ee5ab201346521ae91f474a -Author: Dan Scott -Date: Wed May 30 22:33:17 2012 -0400 - - Bump version numbers to help with library compatibility - - By bumping the OpenSRF version numbers for Perl libs, Python libraries, - and the libtool versioning for the libopensrf library, projects that - build on OpenSRF will be able to flag the minimum required version and - bail out early if an older version of OpenSRF has been installed. - - Signed-off-by: Dan Scott + Signed-off-by: Bill Erickson + Signed-off-by: Galen Charlton -2 2 src/libopensrf/Makefile.am -1 1 src/perl/lib/OpenSRF.pm -1 1 src/python/setup.py -1 1 version.m4 +8 2 src/perl/lib/OpenSRF/System.pm +6 0 src/perl/lib/OpenSRF/Transport/SlimJabber/Client.pm +6 0 src/perl/lib/OpenSRF/Transport/SlimJabber/PeerConnection.pm +1 0 src/perl/lib/OpenSRF/Transport/SlimJabber/XMPPReader.pm -commit ee2c79929aa63ff8d6749da7d10c15ab967d9fd5 -Author: Dan Scott -Date: Tue May 22 16:45:33 2012 -0400 +commit b0a41d3973c944259c193fb4da1dce7f0c690f70 +Author: Bill Erickson +Date: Thu Dec 5 12:27:44 2013 -0500 - Add "tar" to base prereqs to appease autotools + OpenSRF client disconnect robustification (Perl) - On a minimal install of Fedora 17, I was running into the perplexing - error message whilst running 'autoreconf -i': + * Improve detection of client connection state with jabber server - libtoolize: can not copy `/usr/share/libtool/config/ltmain.sh' to `./' + Be more aggressive in testing and responding to failed connectivity. + It can happen in various ways, depending on what's happening to the + jabber server (death, delay, graceful shutdown). With these changes + we attempt to cover all the bases and detect as early as possible that + a connection has died, specifically when trying to create a new + connection after one has failed. - This was confusing because a manual copy would, in fact, copy the file - just fine. + * Do a better job of cleaning up dead connections - http://lists.gnu.org/archive/html/libtool/2009-07/msg00030.html finally - lead to enlightenment: autotools uses "tar" to copy files, not "cp". - Thus, to avoid similar head-scratching install problems in the future, - add "tar" to the base prerequisites for building OpenSRF (both to the - DEBs list and the RPMs list). + Transport::PeerHandle now has a reset() action which disconnects (when + possible) and clears the global connection reference. - Signed-off-by: Dan Scott + Signed-off-by: Bill Erickson + Signed-off-by: Galen Charlton -3 1 src/extras/Makefile.install +8 2 src/perl/lib/OpenSRF/System.pm +6 0 src/perl/lib/OpenSRF/Transport/SlimJabber/Client.pm +6 0 src/perl/lib/OpenSRF/Transport/SlimJabber/PeerConnection.pm +1 0 src/perl/lib/OpenSRF/Transport/SlimJabber/XMPPReader.pm -commit 788a396a2b380c5950c09368f85e8d4a79ec2ca8 -Author: Dan Scott -Date: Tue May 22 02:26:04 2012 -0400 +commit af8cab4b4b77f668e3c5ac85d191acf94537ef32 +Author: Mike Rylander +Date: Thu May 16 10:17:15 2013 -0400 - Update OS support in README / Makefile.install + Protect subrequests from post-complete messages - Kick Debian Lenny, Ubuntu Hardy, and RHEL 5 / CentOS 5 to the curb. + Subrequests (AKA method_lookup()-based API calls) sometimes append + final-lvalue responses to the response list even after respond_complete() + has been called from within the implementation method. This commit + discards all post-respond_complete values, protecting against extra, + spurious values being returned to the ultimate client. - Also change the name of the README to the more generic & instructive - "Installing OpenSRF" rather than "README for OpenSRF #.#.#". - - Signed-off-by: Dan Scott - - Conflicts: - - README - - Signed-off-by: Dan Scott + Signed-off-by: Mike Rylander + Signed-off-by: Galen Charlton -3 9 README -3 243 src/extras/Makefile.install +16 2 src/perl/lib/OpenSRF/AppSession.pm -commit 3dd57f268b8b5d39f85139e4c4e5246bd9e2cda5 -Author: Thomas Berezansky -Date: Wed May 23 10:13:41 2012 -0400 +commit 8d9b6b60cec68910ab48e4477e5de2d451013dc5 +Author: Galen Charlton +Date: Mon Dec 16 08:08:51 2013 -0800 - Nagios Example Plugin for monitoring services + LP#1180849: test case - ignoring subrequest responses after respond_complete() - Does not check that all drones in a brick are fully up, just that the - entire brick will respond to each service, even if only one drone happens - to be running the listener properly (and said listener is responding). + This patch adds a regression test for verifying that subrequests + ignore additional responses after respond_complete() is called. - Signed-off-by: Thomas Berezansky - Signed-off-by: Dan Scott + Signed-off-by: Galen Charlton -219 0 examples/nagios/check_osrf_services - create mode 100755 examples/nagios/check_osrf_services +9 1 src/perl/t/02-AppSession.t -commit e7431d6fa114c35d3dc5b430fbb4bdae99edaa88 -Author: Bill Erickson -Date: Mon Apr 2 14:55:56 2012 -0400 +commit a0d5b0536231cc06552fb9d7533a713c314ef9d3 +Author: Mike Rylander +Date: Thu May 16 10:17:15 2013 -0400 - Detect and repair multipart/mixed message delivery errors + Protect subrequests from post-complete messages - For unknown reasons, the Content-Type header will occasionally be - included in the XHR.responseText for multipart/mixed messages. When - this happens, strip the header and newlines from the message body and - re-parse. + Subrequests (AKA method_lookup()-based API calls) sometimes append + final-lvalue responses to the response list even after respond_complete() + has been called from within the implementation method. This commit + discards all post-respond_complete values, protecting against extra, + spurious values being returned to the ultimate client. - Signed-off-by: Bill Erickson - Signed-off-by: Dan Scott + Signed-off-by: Mike Rylander + Signed-off-by: Galen Charlton -33 1 src/javascript/opensrf.js +16 2 src/perl/lib/OpenSRF/AppSession.pm -commit 832f166b38acc9c68c9c1fcea1432d2ca881316b +commit 59b4dd74e8695bc9bd7ad6c379b0f313db1cbb61 Author: Galen Charlton -Date: Mon May 21 17:29:05 2012 -0400 +Date: Mon Dec 16 08:08:51 2013 -0800 - fix up index/position type for calls of various osrfList* functions - - osrfListSet, osrfListRemove, osrfListGetIndex, and osrfListExtract - all expect unsigned ints for the index/position parameter. + LP#1180849: test case - ignoring subrequest responses after respond_complete() - src/jserver is ignored by this commit as its removal is pending. + This patch adds a regression test for verifying that subrequests + ignore additional responses after respond_complete() is called. Signed-off-by: Galen Charlton - Signed-off-by: Dan Scott -1 1 src/gateway/osrf_http_translator.c -2 2 src/libopensrf/osrf_json_tools.c -1 1 src/libopensrf/osrf_list.c -1 1 src/libopensrf/string_array.c -1 1 src/router/osrf_router.c +9 1 src/perl/t/02-AppSession.t -commit bdc4cfe354051e4132d6ffa2da3e0942acb3f780 -Author: Dan Scott -Date: Sat May 5 01:58:22 2012 -0400 +commit 4ec8e134da80422f288e3e84024a8aa137b22259 +Author: Jeff Godin +Date: Tue Dec 3 13:52:39 2013 -0500 - Remove comparisons that can never evaluate to true - - Using clang as the compiler results in 4 warnings like the following: + Add some additional boolean-related JSON tests - osrf_list.c:106:23: warning: comparison of unsigned expression < 0 is - always false [-Wtautological-compare] - if(!list || position < 0) return NULL; - ~~~~~~~~ ^ ~ + Add some additional boolean-related JSON tests, enable use strict / + use warnings for JSON tests. - (Explanation: "position" is an unsigned int; thus the comparison to < 0 - can never evaluate to true). - - Signed-off-by: Dan Scott + Signed-off-by: Jeff Godin Signed-off-by: Galen Charlton -4 4 src/libopensrf/osrf_list.c +12 5 src/perl/t/09-Utils-JSON.t -commit 3aabf1932f93bc6e1e6693001734a1f1bcdbdce0 -Author: Dan Scott -Date: Sat May 5 01:32:25 2012 -0400 +commit 9a367d3621706693445b945b283d54281d6dfa0a +Author: Mike Rylander +Date: Tue Dec 3 09:57:39 2013 -0500 - LP954059: Silence uninitialized var warning - - Compiling osrf_utf8.c generates the following warning: + LP#1257264: Use the built-in JSON-y test for bools - osrf_utf8.c:510:29: warning: utf8_char may be used uninitialized in this - function [-Wuninitialized] + This removes a dependency on internal details of JSON::XS's + implementation of Boolean types which changed with the release + of JSON::XS 3.0. - So... initialize utf8_char when we declare it, and make the compiler - happy. - - Signed-off-by: Dan Scott + Signed-off-by: Mike Rylander Signed-off-by: Galen Charlton + Signed-off-by: Jeff Godin -1 1 src/libopensrf/osrf_utf8.c +3 4 src/perl/lib/OpenSRF/Utils/JSON.pm -commit 4fe1fdc7d4561e04212444786fd61d95eae69eb0 +commit db87a311765e1c9bd538504827349a797a9b1bc4 Author: Galen Charlton -Date: Mon May 21 17:02:34 2012 -0400 +Date: Tue Dec 3 09:43:54 2013 -0800 - LP# 953299 - defend against null and zero-length cache keys - - Ignore undefined and zero-length (after key normalization) cache - keys. + LP#1257264: make test cases for JSON::XS Boolean-ness more generic Signed-off-by: Galen Charlton - Signed-off-by: Dan Scott + Signed-off-by: Jeff Godin -9 2 src/perl/lib/OpenSRF/Utils/Cache.pm +5 5 src/perl/t/09-Utils-JSON.t -commit 3e97b0f69be04957eefa149d06e05111f3ad4291 -Author: Dan Scott -Date: Mon May 21 12:18:41 2012 -0400 +commit b93e0ca9c88e76d8a6a3c2451cbb49756c6ecf68 +Author: Jeff Godin +Date: Tue Dec 3 13:52:39 2013 -0500 - LP# 953299 - Prevent get/set of invalid cache keys + Add some additional boolean-related JSON tests - Clients of OpenSRF::Utils::Cache occasionally request cache keys that - contain invalid characters (a particular case is ISBNs that contain - spaces), so strip those out of incoming get/set requests to avoid ugly - memcached errors. + Add some additional boolean-related JSON tests, enable use strict / + use warnings for JSON tests. - Signed-off-by: Dan Scott + Signed-off-by: Jeff Godin Signed-off-by: Galen Charlton -25 0 src/perl/lib/OpenSRF/Utils/Cache.pm -3 1 src/perl/t/09-Utils-Cache.t +12 5 src/perl/t/09-Utils-JSON.t -commit c6cf4d9b6530f5b8b6bdf95c26db9986656fcce8 -Author: Dan Scott -Date: Mon May 21 12:41:25 2012 -0400 +commit a5be2f1654ea2211abef57f6065d8f6c5ee7f58a +Author: Mike Rylander +Date: Tue Dec 3 09:57:39 2013 -0500 - .gitignore - ignore more built stuff in fewer lines + LP#1257264: Use the built-in JSON-y test for bools - Use the power of glob to catch more built files that we want to ignore, - while simplifying the .gitignore file significantly. + This removes a dependency on internal details of JSON::XS's + implementation of Boolean types which changed with the release + of JSON::XS 3.0. - Signed-off-by: Dan Scott - -23 108 .gitignore - -commit 2c0fb3a9f6aeea9c93846c573ced8d235ec03c7c -Author: Dan Scott -Date: Mon May 21 11:48:26 2012 -0400 - - Revert inadvertently introduced debugging code - - In commit 91b8790c, several debugging lines were accidentally committed - to the OpenSRF caching utility. Revert that change. - - Signed-off-by: Dan Scott + Signed-off-by: Mike Rylander + Signed-off-by: Galen Charlton + Signed-off-by: Jeff Godin -0 3 src/perl/lib/OpenSRF/Utils/Cache.pm +3 4 src/perl/lib/OpenSRF/Utils/JSON.pm -commit f6d38086a42cd6a60d7ed45461d99076cf4d9e5e -Author: Dan Scott -Date: Tue May 1 22:11:00 2012 -0400 +commit ff472c0e3c1dd47b5a584ebaf6a64d0534c5438b +Author: Galen Charlton +Date: Tue Dec 3 09:43:54 2013 -0800 - Minor cleanups to Java dependency build process - - While configure.ac still hardcodes the expected location of the Java - dependencies (JSON and Memcached classes), we can clean up some of the - lower-level challenges with the Java build. + LP#1257264: make test cases for JSON::XS Boolean-ness more generic - * StAX and WSTX were rolled into the core Java environment long ago, - so there's no need to download them and even less need to verify - that the downloaded version exist. - * The Memcached class is now up to 2.0.1, per deps.sh, but the deps.inc - file was pointing at 1.5.1. - * Speaking of the deps.inc file, nothing seems to use it; delete it. - * The Makefile was dying because the variables for the Memcached and - JSON classes weren't defined. Define those at the top of the Makefile. - - Signed-off-by: Dan Scott + Signed-off-by: Galen Charlton + Signed-off-by: Jeff Godin -4 3 src/java/Makefile.am -0 5 src/java/deps.inc - delete mode 100644 src/java/deps.inc +5 5 src/perl/t/09-Utils-JSON.t -commit d9487c42a1e3432be9fa3da9b2fad2b42d24a645 -Author: Dan Scott -Date: Tue May 1 22:08:23 2012 -0400 +commit 9028b0238106923a64534b88eb6382bf59783ea5 +Author: Galen Charlton +Date: Thu Sep 12 12:37:57 2013 -0700 - Clean up root directory: INSTALL, .gitignore, autoreconf - - * Update the README to avoid running autoreconf with the "-f" flag. - * Make the INSTALL file a symbolic link to the README to avoid - creating a generic INSTALL file each time autoreconf is run. - * Add a .gitignore file that ignores all of the built files. + LP#1224647: remove two invalid tests - Signed-off-by: Dan Scott - -181 0 .gitignore -1 0 INSTALL -1 1 README - create mode 100644 .gitignore - create mode 120000 INSTALL - -commit c4c2d62cdf47ee7e5d02fac44cce5515a77c362c -Author: Dan Scott -Date: Tue May 1 21:31:38 2012 -0400 - - Change Java deps.sh to include . in CLASSPATH + The patch for bug 1204123 changes the client_recv() + to not set the transport_client error flag, as most + of the circumstances where it would have been set were + for cases where the call to select() was interrupted + by a signal, which is not an error per se. - Following the directions given by running deps.sh was a little bit - frustrating; we need to work in some documentation on how to use the - Java bits of OpenSRF - and perhaps look at a more standard option like - maven for dependencies. + This patch removes two tests that were therefore + made invalid, restoring the ability of the test suite + to pass. - Signed-off-by: Dan Scott + Signed-off-by: Galen Charlton + Signed-off-by: Bill Erickson -1 1 src/java/deps.sh +0 4 tests/check_transport_client.c -commit d7d8923e98a91c070668a529957d7ee7033529a7 +commit e203c6edd4e02437a56d98f24a7b8c52b142faee Author: Bill Erickson -Date: Fri Mar 16 09:56:13 2012 -0400 +Date: Thu Sep 5 10:32:09 2013 -0400 - Java Gateway interface improved exception handling + osrf_control --start/stop/restart-services - Handle any exceptions that should not reasonably occur in normal - operation under the covers. Bubble the rest up. Update test code with - examples. + Provide commands for controling non-router services. This is useful in + a multi-server environment where the router runs on a different server. Signed-off-by: Bill Erickson - Signed-off-by: Dan Scott + Signed-off-by: Thomas Berezansky -57 51 src/java/org/opensrf/net/http/GatewayRequest.java -16 8 src/java/org/opensrf/net/http/HttpConnection.java -41 12 src/java/org/opensrf/net/http/HttpRequest.java -5 2 src/java/org/opensrf/net/http/HttpRequestHandler.java -50 30 src/java/org/opensrf/test/TestGateway.java +44 7 bin/opensrf-perl.pl.in -commit ad253eb0d67098b69c71141061563b1802f33f97 +commit 0ef66eea902b50efa360815114c11d40d14560d2 Author: Bill Erickson -Date: Thu Mar 15 17:26:16 2012 -0400 +Date: Tue Sep 3 14:56:27 2013 -0400 - Java gateway interface test class + LP#1204123 repair makefile.am warning + + Avoid trailing space after backslash Signed-off-by: Bill Erickson - Signed-off-by: Dan Scott + Signed-off-by: Jason Stephenson -58 0 src/java/org/opensrf/test/TestGateway.java - create mode 100644 src/java/org/opensrf/test/TestGateway.java +1 1 src/Makefile.am -commit 5b5e28f16be77d9b23a98579d10173103dd907ed +commit ca94866879192a2ea962ce12d15573aa83115cd8 Author: Bill Erickson -Date: Mon Feb 27 18:05:07 2012 -0500 +Date: Fri Aug 30 14:17:02 2013 -0400 - Java HTTP gateway interface + LP#1204123 osrf_control warn and exit on bad host - Supports sync and async requests. Async requests support onResponse, - onComplete, and onError handlers. + Any time a start action is issued (start, start_all, restart, + restart_all), osrf_control will first verify that the selected hostname + (or localhost) hosts the requested service(s). If not, the user is + warned and the script exits before any action is taken. - Supports a max-threads value to limit the number of activately - communicating threads over any connection. When max-threads is reached, - requests are queued and delivered as soon as there is room. + Note: we do not make a similar verification for stop and signal actions, + since those may be reasonably be used even when a service is not + configured to run on the selected host. - Note that since this is talking to the OpenSRF gateway and not the - translater, responses are simply collected and passed one at a time to - onResponse. They are not streamed. The goal of supporting onResponse - is to provide the same client API for both the gateway and translator. + Thanks to Jason Stephenson for the suggestion. Signed-off-by: Bill Erickson - Signed-off-by: Dan Scott - -129 0 src/java/org/opensrf/net/http/GatewayRequest.java -97 0 src/java/org/opensrf/net/http/HttpConnection.java -66 0 src/java/org/opensrf/net/http/HttpRequest.java -25 0 src/java/org/opensrf/net/http/HttpRequestHandler.java - create mode 100644 src/java/org/opensrf/net/http/GatewayRequest.java - create mode 100644 src/java/org/opensrf/net/http/HttpConnection.java - create mode 100644 src/java/org/opensrf/net/http/HttpRequest.java - create mode 100644 src/java/org/opensrf/net/http/HttpRequestHandler.java - -commit 85830ce301654e2dfefcd186a2bd63bd3785a967 -Author: Dan Scott -Date: Mon Apr 23 14:17:35 2012 -0400 - - Remove GNU default INSTALL file - - GNU autotools generate an INSTALL file that is generic and which - conflicts with the instructions given in the README. Installation is - already hard enough without conflicting instructions, so delete the - INSTALL file. - - Signed-off-by: Dan Scott - -0 237 INSTALL - delete mode 100644 INSTALL - -commit bc36b120e78f9fa995856a144b9054e2a0ab1f3e -Author: Dan Scott -Date: Mon Apr 23 14:13:37 2012 -0400 - - autotools - do not explicitly include m4 directory - - Some versions of autotools complain bitterly if an m4 include directory - is specified that does not exist. - - Signed-off-by: Dan Scott - -0 2 Makefile.am -0 1 configure.ac - -commit 2bdd580e2bcc6660b073b6853dc1544d5c68a6fd -Author: Dan Scott -Date: Mon Apr 23 13:45:14 2012 -0400 - - Name the bootstrapping steps in buildbot - - Signed-off-by: Dan Scott + Signed-off-by: Jason Stephenson -10 6 examples/buildbot.cfg +30 7 bin/opensrf-perl.pl.in -commit f21b0a5aa647e7303cc0d6d3fb70cadf0e8fc812 -Author: Dan Scott -Date: Mon Apr 23 13:16:25 2012 -0400 +commit f7c7fd436ffdd85abef2f8c76c8e6f59755777ab +Author: Bill Erickson +Date: Fri Aug 30 13:09:27 2013 -0400 - Fix buildbot configuration + LP#1204123 osrf_control --diagnostic improvements - Need to define our tests before they're called; also, can't blindly - invoke a step out of the proper scope. Duh. + Assessing the state of each PID for each service for error conditions. - Signed-off-by: Dan Scott + Minor cleanup. - Conflicts: - - examples/buildbot.cfg - -55 18 examples/buildbot.cfg - -commit 4d58c3d24b65f96e0648e37a3f4da82051a58917 -Author: Dan Scott -Date: Mon Apr 23 12:52:57 2012 -0400 - - Switch to autoreconf instead of autogen.sh - - Update the buildbot config accordingly to avoid erroneous errors of - erroneosity. - - Signed-off-by: Dan Scott + Signed-off-by: Bill Erickson + Signed-off-by: Jason Stephenson -1 1 README -0 43 autogen.sh -12 4 examples/buildbot.cfg - delete mode 100755 autogen.sh +28 41 bin/opensrf-perl.pl.in -commit 5849a119bd363b152b41ed7a39c787f009412572 -Author: Dan Scott -Date: Mon Apr 2 16:48:15 2012 -0400 +commit e6cdba1fe3b3af91d91b13ea0ad4a4d8ceaa4b21 +Author: Bill Erickson +Date: Fri Aug 30 11:27:49 2013 -0400 - Bump version numbers for 2.1.0-RC1 release + LP#1204123 osrf_control --diagnostic - Also update the ChangeLog with relevant entries. + Command prints information about running services, including + descrepencies between running vs configured and PS listing vs PID files. - Signed-off-by: Dan Scott + Signed-off-by: Bill Erickson + Signed-off-by: Jason Stephenson -21 0 ChangeLog -1 1 src/perl/lib/OpenSRF.pm -1 1 version.m4 +96 4 bin/opensrf-perl.pl.in -commit 0ed34101e67dc04292f906945dd5752c73985412 -Author: Dan Scott -Date: Fri Mar 23 13:05:31 2012 -0400 +commit d1d2894e69ca2741b54b936737ac442b1774cd95 +Author: Bill Erickson +Date: Tue Aug 27 15:56:46 2013 -0400 - Add trailing period, "localhost" to ejabberd.cfg example + LP1204123 osrf_ctl.sh improvement - Thanks to Yamil Suarez for noticing a discrepancy between the older wiki - instructions and the README. + Removing no-longer-supported --verbose flag from osrf_ctl.sh stub - Signed-off-by: Dan Scott + Signed-off-by: Bill Erickson + Signed-off-by: Jason Stephenson -1 1 README +1 2 bin/osrf_ctl.sh.in -commit cf4d4798c5949abaf6bc9a4c3a62bf9a6533977b +commit 9c07e2ad85a3a569914070b59c3b185d1cc0f1cb Author: Bill Erickson -Date: Wed Mar 14 11:38:18 2012 -0400 +Date: Tue Aug 27 15:52:48 2013 -0400 - Prevent undef warnings on perl socket read nbytes test + LP1204123 osrf_control doc updates Signed-off-by: Bill Erickson - Signed-off-by: Dan Scott - -1 1 src/perl/lib/OpenSRF/Transport/SlimJabber/XMPPReader.pm - -commit fff96812ff55cc50374d5a16e7c803fac2c2f2a5 -Author: Dan Scott -Date: Sun Mar 11 11:22:14 2012 -0400 - - Prepare 2.1.0-alpha1 for release - - Signed-off-by: Dan Scott + Signed-off-by: Jason Stephenson -555 0 ChangeLog 2 2 README -2 2 src/perl/lib/OpenSRF.pm -1 1 version.m4 +1 1 examples/opensrf.xml.example -commit f470b55b86fd59f31dd142d8cb55fe812265892e +commit 7fed81ab4a24236ef2db4bd0f1d6bd40c65e0b80 Author: Bill Erickson -Date: Mon Feb 20 14:40:30 2012 -0500 +Date: Tue Aug 27 11:47:52 2013 -0400 - Detect remote disconnect in Perl XMPP reader + C SIGHUP handling and config reloading - When the jabber server severs the connection, it leaves the Perl libs - with a socket that perpetually appears ready for reading, but always - returns 0 bytes. This produces a loop in the client code. This change - detects this situation, logs an error, and throws an exception. + Sending the HUP signal to a C Listener process now results in the + following: - Signed-off-by: Bill Erickson - Signed-off-by: Dan Scott - -13 2 src/perl/lib/OpenSRF/Transport/SlimJabber/XMPPReader.pm - -commit 974d3a0dd4ddd134033b16bcd9e2fde34302ffd5 -Author: Dan Wells -Date: Tue Mar 6 15:08:33 2012 -0500 - - Protect gateway from format-string crashes in data - - As a common security measure, printf-style formatting codes are - not allowed to be directly interpreted from a writable segment. - The gateway code currently has the following function call: - - osrfLogActivity( OSRF_LOG_MARK, act->buf ); + * Reload the opensrf_core config + * update the log file level + * kill idle child processes + * child processes that are active when the signal is received are + tracked and killed once they become idle. + * New children are spawned per the min child settings - This is a variadic function which expects the 'act->buf' position - to contain a format string and any trailing arguments to be the - values passed to the formatter. Since act->buf is the value of - what we passed in, some data inadvertantly contains format strings, - and since it is a writable segment, the program crashes. Here is - an example of a crash-causing call: + The primary use case for these changes is temporarily changing the log + level for a given service for debug purposes. It may also + be used, for example, to gracefully recover excess RAM consumed by a + child process. - http://localhost/osrf-gateway-v1?service=test&method=test¶m=%22%251n%22 + opensrf.xml (opensrf.settings config) is not reloaded, so min/max child + settings will not be affected. - The param is interpreted as "%1n" and abruptly fails. - - The simple solution is to include a formatter so that our param gets - demoted to being mere data, i.e.: - - osrfLogActivity( OSRF_LOG_MARK, "%s", act->buf ); - - Signed-off-by: Dan Wells - Signed-off-by: Dan Scott + Signed-off-by: Bill Erickson + Signed-off-by: Jason Stephenson -1 1 src/gateway/osrf_json_gateway.c +2 0 include/opensrf/osrfConfig.h +6 0 src/libopensrf/osrfConfig.c +134 19 src/libopensrf/osrf_prefork.c -commit fd92a859e2ba3eeeaafe4904cd04973eb8cd572b -Author: Thomas Berezansky -Date: Wed Mar 7 16:42:21 2012 -0500 +commit 76ceb9d1e4d662c7fa8c41434df940647bf32fef +Author: Bill Erickson +Date: Fri Aug 16 10:07:10 2013 -0400 - Stop warning about missing parentheses... + LP1204123 more osrf_ctl/control script cleanup - ...by adding them. - - Signed-off-by: Thomas Berezansky + Signed-off-by: Bill Erickson + Signed-off-by: Jason Stephenson -1 1 src/perl/lib/OpenSRF/Application/Validator.pm +1 1 bin/osrf_ctl.sh.in -commit 8cfa0ae50935176a1574e8f233c8d4d88442c0ad +commit 9f8d384361e88d786d70a350f6f154e93a592d7b Author: Bill Erickson -Date: Thu Feb 16 17:31:29 2012 -0500 +Date: Fri Aug 16 10:04:55 2013 -0400 - Refresh child file handles on status read - - If a child process dies while the parent is attempting to read status - information from the child, the pipe connecting the two will be cleaned - up. The parent mustn't attempt to read from the dead pipe or it will - result in a bad file descriptor error, culminating in a high-cpu - infinite loop on the parent (Listener) process. + LP1204123 more osrf_ctl/control script cleanup Signed-off-by: Bill Erickson - Signed-off-by: Thomas Berezansky Signed-off-by: Jason Stephenson -4 3 src/perl/lib/OpenSRF/Server.pm +5 3 bin/opensrf-perl.pl.in +1 1 bin/osrf_ctl.sh.in -commit 37606bf64f2829258cc612e730782e081e2ac6bb +commit d849a24e417898d1a4dc2590af51f95f56b508ff Author: Bill Erickson -Date: Fri Jan 13 10:57:59 2012 -0500 +Date: Mon Aug 12 11:39:28 2013 -0400 - Python libs for OpenSRF ingress tracking - - osrf.ses.Session.ingress(ingress) + LP1204123 osrf_control improved sample usage - This also set the "srfsh" ingress value for srfsh.py. + Use a more realistic default usage message. --config and --pid-dir have + sane defaults, so people will not normally need to specify those. Signed-off-by: Bill Erickson - Signed-off-by: Mike Rylander + Signed-off-by: Jason Stephenson -11 1 src/python/osrf/ses.py -2 0 src/python/osrf/stack.py -1 0 src/python/srfsh.py +1 1 bin/opensrf-perl.pl.in -commit 26c31a9d5728adbd884a1369034caff41471130b +commit 7e274fb696485b43bba31315138fad866221b97b Author: Bill Erickson -Date: Fri Jan 13 09:35:18 2012 -0500 +Date: Mon Aug 12 11:33:11 2013 -0400 - Set OpenSRF ingress value for srfsh/gateways + LP1204123 install opensrf-perl.pl as osrf_control - Sets the "srfsh", "gateway-v1", and "translator-v1" ingress values - accordingly. + Symlink osrf_control to opensrf-perl.pl to act as the replacement for + osrf_ctl.sh. I chose "osrf_control" since it matches more closely to + "osrf_config" -- also it's probably less confusing than calling it + osrf_ctl, which would likely get confused w/ osrf_ctl.sh - For the translater, it's necessary to stamp the unpacked messages with - the updated ingress, then re-serialize before sending the messages along. + TODO: documentation Signed-off-by: Bill Erickson - Signed-off-by: Mike Rylander + Signed-off-by: Jason Stephenson -18 21 src/gateway/osrf_http_translator.c -1 0 src/gateway/osrf_json_gateway.c -1 0 src/srfsh/srfsh.c +3 0 src/Makefile.am -commit 7ec92808fea116ead923e475ad5242a54b688798 +commit 79522df516441ea34fa615bd1e83e8bc4ee93d66 Author: Bill Erickson -Date: Fri Jan 13 09:15:05 2012 -0500 +Date: Mon Aug 12 11:31:56 2013 -0400 - C libs for OpenSRF ingress tracking + LP1204123 opensrf-perl.pl improve logging + + Log messages by default. Replace --verbose flag with --quiet flag. - osrfAppSessionSetIngress(); + Also treat all messages the same, instead of separating them into info + and debug. They're all pretty useful. Signed-off-by: Bill Erickson - Signed-off-by: Mike Rylander + Signed-off-by: Jason Stephenson -5 0 include/opensrf/osrf_app_session.h -5 0 include/opensrf/osrf_message.h -26 0 src/libopensrf/osrf_app_session.c -32 0 src/libopensrf/osrf_message.c -4 0 src/libopensrf/osrf_stack.c +35 21 bin/opensrf-perl.pl.in -commit b12de37f9480c68b1c8b033da18c3cc0d0ebc8f0 +commit 5f21d07c7d5c61c3cb6c7ecd891fe79e68a06aae Author: Bill Erickson -Date: Thu Jan 12 16:36:16 2012 -0500 +Date: Fri Aug 9 10:40:29 2013 -0400 - Perl libs for OpenSRF ingress tracking - - Ingress is a free-form text value which represents the entry point for - the client into the opensrf network. The value is passed within opensrf - messages, similar to "locale". Clients should specify the ingress - before any opensrf communication occurs. - - OpenSRF::AppSession->ingress($ingress); - - Stock values include the following: + LP1204123 opensrf-perl.pl expansion continued - opensrf (default) - srfsh - translator-v1 - gateway-v1 + * improved cleanup of stale pid files during startup + * added --force-clean-process option to kill orphan (no pidfile) + processes during startup + * other minor cleanup Signed-off-by: Bill Erickson - Signed-off-by: Mike Rylander - -9 0 src/perl/lib/OpenSRF/AppSession.pm -20 1 src/perl/lib/OpenSRF/DomainObject/oilsMessage.pm -2 0 src/perl/lib/OpenSRF/Transport.pm + Signed-off-by: Jason Stephenson -commit 67d4d5de8656128f32b20f4c04dbf81a1028e929 -Author: Dan Scott -Date: Wed Feb 29 10:48:38 2012 -0500 +78 38 bin/opensrf-perl.pl.in +3 7 bin/osrf_ctl.sh.in - Copy opensrf_core.xml.example to a clean opensrf_core.xml file - - Ben Shum with the assist again. - - Also, add ~/.srfsh.xml to the definition list of interesting OpenSRF - configuration files. +commit 69b139d87447e9ddc546b5a772e9fa1725e23e6c +Author: Bill Erickson +Date: Mon Aug 5 16:58:26 2013 -0400 + + LP1204123 opensrf-perl.pl expansion / replace osrf_ctl + + opensrf-perl.pl can now stop/start/signal/etc. all opensrf services, + regardless of language. It is in effect a replacement for osrf_ctl.sh + with considerably broader powers. + + To ease transition, osrf_ctl.sh has been taught to pass + start/stop/restart_all commands through to opensrf-perl.pl. + + The layout of commands has changed some (e.g. --start-all instead of + --action start_all) and a host of new commands have been added. + + Below are the full set of commands. See --help for full descriptions: + + --config + --pid-dir + --settings-startup-pause + --localhost + --service + --verbose + --no-daemon + --help + --start-all + --start + --restart-all + --restart + --stop-all + --stop + --graceful-shutdown-all + --graceful-shutdown + --fast-shutdown-all + --fast-shutdown + --immediate-shutdown-all + --immediate-shutdown + --kill-with-fire + --signal-all + --signal + --signal-timeout - Signed-off-by: Dan Scott + Signed-off-by: Bill Erickson + Signed-off-by: Jason Stephenson -6 3 README +390 49 bin/opensrf-perl.pl.in +10 339 bin/osrf_ctl.sh.in -commit 68025993b01cc95f66a0ca723841e1abb406709f -Author: Dan Scott -Date: Wed Feb 29 10:42:58 2012 -0500 +commit ca1a08ff5864a2a4ef8e11070330eb0f5eb3c518 +Author: Bill Erickson +Date: Thu Aug 1 10:39:21 2013 -0400 - Include explicit instructions to copy the opensrf config files + LP1204123 Remove osrf_ctl debug echos - Also noted by the sharp-eyed Ben Shum, we didn't tell people to actually - copy opensrf.xml.example / opensrf_core.xml.example, which could lead to - failure. - - Signed-off-by: Dan Scott + Signed-off-by: Bill Erickson + Signed-off-by: Jason Stephenson -35 17 README +0 4 bin/osrf_ctl.sh.in -commit 75f27021d4b45eebbf52a89a35f96e775f92a207 -Author: Dan Scott -Date: Wed Feb 29 10:27:36 2012 -0500 +commit 06368dfa151f04c6cbb0b843b7e9dc795a8193c5 +Author: Bill Erickson +Date: Wed Jul 31 17:43:12 2013 -0400 - Update max_user_sessions suggestion to 10000 to match wiki + LP1204123 Improved per-service control (C/Perl) - Ben Shum noticed that the README still specified 1000 for - max_user_sessions in the ejabberd.cfg file, whereas current larger - OpenSRF systems with many services may blow past that limit. Update to - match the current wiki specification of 10000. + C opensrf no longer mantains a single master process. Instead, like + Perl, each Listener process writes its own PID file and can be managed + individually. Related to this, much code was cleaned up in + osrf_system.c. - Signed-off-by: Dan Scott - -1 1 README - -commit 61fdb582dc918a1003351ff25842a23aaff24f71 -Author: Dan Scott -Date: Wed Jan 4 15:21:36 2012 -0500 - - Remove reference to Fedora specific version in prereq installer + osrf_ctl.sh now has the ability to stop/start individual services for + Perl and C, using the new -s option. Examples: - Fedora changes every 6 months, so it's probably silly to have a make - target of "fedora14" when that is no longer supported by the Fedora - project itself. As Fedora has packaged all of the OpenSRF dependencies - (thanks, Ben Webb!), in theory it should be supported by the latest - releases of Fedora in the future... as long as we don't introduce any - dependencies on deprecated versions of packages. + osrf_ctl.sh -l -a restart_c -s opensrf.math + osrf_ctl.sh -l -a restart_perl -s opensrf.settings - Signed-off-by: Dan Scott + Signed-off-by: Bill Erickson + Signed-off-by: Jason Stephenson -2 2 src/extras/Makefile.install +73 8 bin/osrf_ctl.sh.in +5 0 include/opensrf/osrf_system.h +57 13 src/libopensrf/opensrf.c +1 0 src/libopensrf/osrf_prefork.c +205 389 src/libopensrf/osrf_system.c -commit ba6d4c7cc2988dafb6ca8de2d1f2f93194dcf3b3 -Author: Dan Scott -Date: Wed Jan 4 15:17:48 2012 -0500 +commit ab83a04551cd333870d0ee6679e5aa6f4342cfb3 +Author: Bill Erickson +Date: Wed Jul 31 09:34:19 2013 -0400 - Update README to match Evergreen's format - - Reference the various accounts in use consistently. - - Correct reference to "fedora" target in Makefile.install. - - Put the developer preamble up front. + LP1204123 osrf_ctl.sh supports sending signals - Provide more Fedora examples. + Using the following new options to osrf_ctl.sh, the script can now send + signals to any/all OpenSRF Listener processes: - Do not include leading "#" and "$" in bash examples, to make it easier - for people to copy and paste. + -a signal / signal_all + -k [value passed to 'kill -s'] + -s - Include source highlighting instructions and titles for example - commands. + With an action of 'signal' and service provided by -s, the specified + signal is sent only to the listener process of the service. - Signed-off-by: Dan Scott - -152 98 README - -commit cc7a12f74a9ce4555d5abd8989dfab43290fb41c -Author: Bill Erickson -Date: Fri Feb 24 16:28:27 2012 -0500 - - Java dependencies update - - As of Java 6, XML Stax parsing is natively supported. This change - removes the external Stax dependencies and updates how the JSON libs are - fetched (in deps.sh), which now come from github. + With an action of 'signal_all', the specified signal is sent to all + listener prcesses. Signed-off-by: Bill Erickson - Signed-off-by: Dan Scott - -17 20 src/java/deps.sh -1 4 src/java/org/opensrf/net/xmpp/XMPPReader.java -1 4 src/java/org/opensrf/util/XMLFlattener.java - -commit 07b2eff0a7dfc492afd8e67788eae159c5ab96b7 -Author: Dan Scott -Date: Tue Feb 21 09:55:57 2012 -0500 - - We're not in Subversion anymore, Dorothy - - We're in git now, make the README reflect that accordingly. Thanks to - Warren Layton for the tip! - - Signed-off-by: Dan Scott + Signed-off-by: Jason Stephenson -2 2 README +73 2 bin/osrf_ctl.sh.in -commit aeeb4acdc8695a640021dbc6902ab3279652583d +commit cb6bdf156a69a60fa0a55006b6f638853da3b9f0 Author: Bill Erickson -Date: Tue Feb 14 09:10:58 2012 -0500 +Date: Wed Jul 24 16:28:15 2013 -0400 - Perl parent/child write improvements + LP1204123 Don't treat interrupts as errors - * Updated variable names for clarity - * Added more inline comments - * Added additional error logging - * For severe read errors, allow the child to gracefully skip the request + Interrupted calls to select() are generally not errors, but more likely + signals received by the process. Avoid setting the transport_client + error flag to true in these cases. Otherwise, after one signal is + received, no other outbound communication can occur on the client + connection. Signed-off-by: Bill Erickson Signed-off-by: Jason Stephenson - Signed-off-by: Mike Rylander -25 19 src/perl/lib/OpenSRF/Server.pm +1 3 src/libopensrf/transport_client.c +0 1 src/libopensrf/transport_session.c -commit 08ee4f993fe773e37233b139961cbcdae2fe93b8 +commit ab845cdebe7588a7fd4fb67317b7deab557fa210 Author: Bill Erickson -Date: Mon Feb 13 16:53:59 2012 -0500 +Date: Tue Jul 23 12:06:48 2013 -0400 - Perl pipe reading overhaul : data size header + LP1204123 SIGUSR2 causes router re-register (Perl/C) - The lockfile mechanism for preventing premature end of reads on child - processes suffers from one serious flaw: if the data to write exceeds - the pipe buffer size, the parent will block on syswrite and the service - will lock up. It's also not as effecient (for the normal case) as the - code was without the lockfile, becasue the writes and reads are - serialized. + When a listener process receives the USR2 signal, the 'register' command + is sent to all known routers. This allows for the addition of new + routers to running opensrf services. - This commit replaces the lockfile mechanism with a protocol header in - the data. The first X (currently 12) bytes of data written to the child - process will contain the full length of the data to be written (minus - the header size). The child now reads the data in parallel with the parent as - data is available. If the child reads all available data (in the pipe) - but not all of the expected data, the child will go back into a select() - wait pending more data from the parent. The process continues until all - data is read. + If a register command is sent to a router for which the process is + already successfully register, the router will safely discard it, so + multiple SIGUSR2 calls are safe. - This same mechanism is already used to commicate status info from child - processes to the parent. + Minor int => bool clarification repair to existing C shutdown + (prefork_clear) code. Signed-off-by: Bill Erickson Signed-off-by: Jason Stephenson - Signed-off-by: Mike Rylander - -49 73 src/perl/lib/OpenSRF/Server.pm -1 2 src/perl/lib/OpenSRF/System.pm -commit 04558f38c1c1d314acb978a37193dacb4a6eba31 -Author: Lebbeous Fogle-Weekley -Date: Fri Jan 27 09:05:07 2012 -0500 +20 5 src/libopensrf/osrf_prefork.c +1 0 src/perl/lib/OpenSRF/Server.pm - JSON_v0 has been superseded, and it has insidious bugs anyway. - - For example, you cannot round-trip this through JSON2js() and js2JSON() - in IE8: http://paste.lisp.org/display/127338 - - This will make Evergreen's build fail until the change specified in - LP #922609 is applied. - - Signed-off-by: Lebbeous Fogle-Weekley - Signed-off-by: Bill Erickson - -1 1 src/Makefile.am -0 135 src/javascript/JSON_v0.js - delete mode 100644 src/javascript/JSON_v0.js - -commit 89f41c82798dbac56716fdd5beeb6c0de3e4ce2e +commit 84d949cc09de9e2689b76541caeae524ab3b1f1e Author: Bill Erickson -Date: Mon Nov 7 17:30:44 2011 -0500 +Date: Tue Jul 23 11:31:34 2013 -0400 - Sync parent/child write/read with lock file + LP1204123 opensrf.settings reload file option - Wrap parent writes to child socket and initial child reads of the - socket in file lock (via flock()) to prevent rare race condition - where child process reads to the end of the data before the parent - has written all bytes. + opensrf.settings API calls - This create a new lock file on the system, which resides in the same - directory as the pid files. The lock file is created and destroyed by - the opensrf perl mods at service start up and shutdown. + opensrf.settings.host_config.get + opensrf.settings.default_config.get - See also https://bugs.launchpad.net/opensrf/+bug/883155 + now sport an additional "reload" boolean parameter. When set, the settings + file (opensrf.xml) will be reloaded from the file system to pick up any + changes which have occurred since the last load/reload. Signed-off-by: Bill Erickson - Signed-off-by: Dan Scott - -1 1 bin/opensrf-perl.pl.in -60 8 src/perl/lib/OpenSRF/Server.pm -3 2 src/perl/lib/OpenSRF/System.pm - -commit 3d089bdfd067676f99807f7cb2014e122c4dbf72 -Author: Dan Scott -Date: Mon Dec 12 13:09:13 2011 -0500 - - Update Perl OpenSRF build and tests for O:A:Validator - - Kickstarted by wanting to bring OpenSRF::Application::Validator into the - MANIFEST and testing coverage, I noticed a few other files that needed - to be added to the MANIFEST as well. This gets us closer to acceptable - according to "./Build distcheck" - - Signed-off-by: Dan Scott + Signed-off-by: Jason Stephenson -1 0 src/perl/Build.PL -22 0 src/perl/MANIFEST -6 1 src/perl/t/01-Application.t +8 5 src/perl/lib/OpenSRF/Application/Settings.pm +13 24 src/perl/lib/OpenSRF/Utils/SettingsClient.pm +23 26 src/perl/lib/OpenSRF/Utils/SettingsParser.pm -commit 5e1fbcc1c8ae2f969dbeac93fe1da80c008ca42b -Author: Thomas Berezansky -Date: Fri Dec 9 15:19:12 2011 -0500 +commit 0e535cb775b78699f7ff15d9b73479f0aeaa4246 +Author: Bill Erickson +Date: Mon Jul 22 16:44:26 2013 -0400 - OpenSRF Validator Service - - Add a new Validator service, and EmailAddress validators. - - The service runs a chain of one or more validators, each one being fed the - normalized output of the previous one. - - The return from each validator should be a hash of valid (0 or 1), the new - normalized output (the untouched input if invalid or nothing needed to be - changed), and if invalid an error string. Optionally, a validator can also - include an "additionals" hash of extra information to be included in the - final response. + LP1204123 SIGTERM graceful shutdown (C) - The complete list of validators included is: + When a TERM signal is received, the listener parent process will wait + for all child processes to complete any active tasks before killing the + child processes and shutting down. - OpenSRF::Application::Validator::Base - The base validator. Always returns valid. - OpenSRF::Application::Validator::Invalid - Always returns invalid for testing purposes. - OpenSRF::Application::Validator::EmailAddress::Regex - Does a very basic regular expression check on email addresses. - OpenSRF::Application::Validator::EmailAddress::DNS - Uses Net::DNS to look up the domain on an email address + Receiving a SIGINT or SIGQUIT results in a non-graceful shutdown. - Signed-off-by: Thomas Berezansky - Signed-off-by: Dan Scott - -40 0 examples/opensrf.xml.example -3 0 src/extras/Makefile.install -50 0 src/perl/lib/OpenSRF/Application/Validator.pm -12 0 src/perl/lib/OpenSRF/Application/Validator/Base.pm -96 0 src/perl/lib/OpenSRF/Application/Validator/EmailAddress/DNS.pm -23 0 src/perl/lib/OpenSRF/Application/Validator/EmailAddress/Regex.pm -16 0 src/perl/lib/OpenSRF/Application/Validator/Invalid.pm - create mode 100644 src/perl/lib/OpenSRF/Application/Validator.pm - create mode 100644 src/perl/lib/OpenSRF/Application/Validator/Base.pm - create mode 100644 src/perl/lib/OpenSRF/Application/Validator/EmailAddress/DNS.pm - create mode 100644 src/perl/lib/OpenSRF/Application/Validator/EmailAddress/Regex.pm - create mode 100644 src/perl/lib/OpenSRF/Application/Validator/Invalid.pm - -commit b6b64deb16ed3f005d64f99365b0220a82d72431 -Author: Jason Stephenson -Date: Wed Dec 7 15:08:42 2011 -0500 - - Support installation for Ubuntu 12.04 Precise Pangolin. - - Update the README to include Precise Pangolin as a supported release. - - Update src/extras/Makefile.install to install two new debs required on - Ubuntu 12.04 alpha1. + In all cases, the system first un-registers with all routers. + Signed-off-by: Bill Erickson Signed-off-by: Jason Stephenson - Signed-off-by: Chris Sharp - -1 0 README -15 1 src/extras/Makefile.install - -commit 3a4ae77be13349fae180fdc81bcc23e5a93032b4 -Author: Dan Scott -Date: Fri Nov 4 09:42:50 2011 -0400 - - Add explicit chown command to README - - Warren Layton noticed that there was no explicit command to change the - ownership of the files in the / directory to be owned by - "opensrf", although there was a statement that the files needed to be - owned by "opensrf". To reduce the chance of failure, add the explicit - command. - - TODO (for a willing volunteer): teach the installer to change the - ownership at the time the files are installed! - - Signed-off-by: Dan Scott -1 0 README +75 20 src/libopensrf/osrf_prefork.c -commit a7be31f137ccf6e2f4522c9a4c690a23b5636db8 +commit fc2dc403640ed245ff2ba8c3f44beea727d0e7cc Author: Bill Erickson -Date: Fri Oct 28 11:33:24 2011 -0400 +Date: Tue Jul 23 10:40:09 2013 -0400 - Add SIGPIPE retry handling to child data sysread + LP1204123 SIGUSR1 causes router unregister (Perl/C) - Similar to the SIGPIPE retry logic wrapped around the parent process' - syswrite call (for sending data to a child process), protect the child's - sysread call (as it reads data from the parent). In pre-2.0, the - sysread step was handled by Net::Server, but now we need to protect it - ourselves. + $ kill -s USR1 + + The listener process will unregister all router connections. This + allows services to remove themselves from consideration for new + inbound requests while allowing their drone processes to complete + any currently running tasks. Signed-off-by: Bill Erickson - Signed-off-by: Galen Charlton + Signed-off-by: Jason Stephenson -10 0 src/perl/lib/OpenSRF/Server.pm +50 13 src/libopensrf/osrf_prefork.c +4 1 src/perl/lib/OpenSRF/Server.pm -commit 40a753b10a3db58baa9179d75a76df1266589ea1 -Author: Dan Scott -Date: Wed Oct 19 11:39:41 2011 -0400 +commit 8cc60399d97466c43a87df7e48e80dbc3b88c2d6 +Author: Bill Erickson +Date: Fri Jun 21 16:43:01 2013 -0400 - LP878284: stop_osrf action should stop, not start, Perl - - As reported by Vicent Mas to the Evergreen - developer's mailing list: + LP1204123 SIGTERM causes graceful shutdown (Perl) - """ - It seems I've found a small bug in the - /openils/bin/osrf_ctl.sh script. The line for the osrf_stop action is: + When sent the SIGTERM signal, listener processes will wait for all child + processes to complete their currently running tasks before killing all + child processes and exiting. - "stop_osrf") stop_python; stop_c; start_perl;; + SIGQUIT and SIGINT can still be used for semi-graceful shutdowns, where + the listener de-registers and cleans up child processes, but does not + wait for child processes to finish their tasks. - but should be: + To kill with fire, SIGKILL is still your friend. - "stop_osrf") stop_python; stop_c; stop_perl;; - """ - - Signed-off-by: Dan Scott + Signed-off-by: Bill Erickson + Signed-off-by: Jason Stephenson -1 1 bin/osrf_ctl.sh.in +20 4 src/perl/lib/OpenSRF/Server.pm -commit 07b5a335656be741353ab70c1d4717dacdbebc1b +commit c69aced833683b5256554330aa652f3dd7556d10 Author: Bill Erickson -Date: Tue Oct 18 09:17:10 2011 -0400 +Date: Wed Jun 26 14:51:52 2013 -0400 - Warn when sending very large messages + LP1204123 Set SIGCHLD handler to DEFAULT for children (perl) - Depending on configuration, messages of a certain size sent through a - Jabber server will cause the jabber server to disconnect the client. - This change allows admins to configure a message size warning threshold. - When a message meets or exceeds the size threshold, a warning is issued - to the logs with the message size (in bytes) and the message recipient. - It does not prevent the message from being delivered. It's purely - informational. - - Use 1 800 000 as the default threhold. + Remove the special SIGCHLD handler applied to the parent/listener + process from spawned child processes (drones) by setting child handler to + DEFAULT. Otherwise, if an OpenSRF drone process spawns a sub-process, + the drone will execute the SIGCHLD handler meant for the Listener when + the sub-process dies, wherein hijinks ensue. Signed-off-by: Bill Erickson - Signed-off-by: Galen Charlton - -3 0 examples/opensrf_core.xml.example -11 1 src/perl/lib/OpenSRF/Transport/SlimJabber/Client.pm - -commit ffb8bb73a2f505cfda9d8b97fab1f82aa3fb7097 -Author: Dan Scott -Date: Sat Sep 10 12:22:08 2011 -0400 - - Fix README typo: 'mod_offlinex' - - mod_offline is not that x-treme. - - Signed-off-by: Dan Scott - -1 1 README - -commit 0725d1ddced0f16c351a5953f5fd3c14714cda1a -Author: Dan Scott -Date: Thu Aug 25 11:41:19 2011 -0400 - - Don't define a variable inside a conditional block - - ./configure --disable-core --enable-javascript was dying with an error - due to CHECK_TESTS not being defined; move it outside the conditional - block so that we can ensure that it is defined as either yes or no. - - Signed-off-by: Dan Scott - -8 9 configure.ac - -commit b24e90f1a1b1f2309ca3cdf0728cdd54f7822597 -Author: Dan Scott -Date: Mon Aug 22 09:49:45 2011 -0400 + Signed-off-by: Jason Stephenson - Make distro targets in README match Makefile.install - - Swap ubuntu-karmic for ubuntu-lucid and pull fedora14 as it is really - close to end of life. - - Signed-off-by: Dan Scott +1 1 src/perl/lib/OpenSRF/Server.pm -2 2 README -- 2.11.0