summary |
shortlog | log |
commit |
commitdiff |
tree
first ⋅ prev ⋅ next
Bill Erickson [Tue, 12 Apr 2016 15:49:04 +0000 (11:49 -0400)]
Log more info on batch completion
Log the last record processed for non-browse reingest. Make logging
more consistent across browse and non-browse reingest stages.
Signed-off-by: Bill Erickson <berickxx@gmail.com>
Bill Erickson [Tue, 12 Apr 2016 14:43:17 +0000 (10:43 -0400)]
Logging consolidation and syslog support
Log all messages to syslog. See $syslog_* variables to modify behavior.
Add support for --log-stdout option, which causes all log messages to be
cloned to STDOUT.
Signed-off-by: Bill Erickson <berickxx@gmail.com>
Bill Erickson [Tue, 12 Apr 2016 14:18:36 +0000 (10:18 -0400)]
Min/Max ID clarification; --sort-id-desc
Replace start-id/end-id arguments with min-id/max-id to avoid any
confusion when using descending sort options.
Add support for --sort-id-desc to process records in reverse order of
record ID. This works better across multiple batches when you don't
know what both the min/max ID values are goign to be ahead of time.
Signed-off-by: Bill Erickson <berickxx@gmail.com>
Bill Erickson [Wed, 2 Mar 2016 15:57:56 +0000 (10:57 -0500)]
pingest --newest-first option
When set, --newest-first causes the script to processes records in
descending order of edit_date, so that more recently edited/created
records are ingested first.
Signed-off-by: Bill Erickson <berickxx@gmail.com>
Bill Erickson [Tue, 16 Feb 2016 17:59:55 +0000 (12:59 -0500)]
pingest supports max/min ID, duration, more ops
From the new help text:
--batch-size
Number of records to process per batch
--max-child
Max number of worker processes
--skip-browse
--skip-attrs
--skip-search
--skip-facets
Skip the selected reingest component
--start-id
Start processing at this record ID.
--end-id
Stop processing when this record ID is reached
--max-duration
Stop processing after this many total seconds have passed.
--help
Show this help text.
Signed-off-by: Bill Erickson <berickxx@gmail.com>
Bill Erickson [Tue, 16 Feb 2016 18:09:18 +0000 (13:09 -0500)]
Cloning pingest from git.mvlcstaff.org/jason/evergreen_utilities.git
Signed-off-by: Bill Erickson <berickxx@gmail.com>
Thomas Berezansky [Thu, 22 Mar 2012 17:06:59 +0000 (13:06 -0400)]
Add quick README to the master branch
Otherwise the default clone branch will be empty.
Signed-off-by: Thomas Berezansky <tsbere@mvlc.org>