*** This version of Confluence is for testing only and contains a copy of content from June 29th 2026. No changes will be preserved. ***
DEPRECATED THIS PROCEDURE IS NO LONGER UPDATED
ScopeOverview:This procedure provides instructions for running automated import, export, cleanup, reporting and various related jobs using scripts found in LS Tools web site and other tools created by the Library Systems Office and LTS Batch Processing staff. For issues surrounding the scheduling of large batch loads, please consult Bulk Loading: Scheduling (#46d). Unless otherwise noted, the items listed below are descriptions of scripts and tools found on LS Tools. Jobs may include scripts known as "mod jobs" run from a staff member's individual LS Tools folder, as opposed to scripts that are accessible directly from the LS Tools web interface. Non-LS Tools jobs are most commonly referred to as "manual" jobs. For information on staff batch job assignments, please refer to the Bulk Job List.
ScopeLoading: List of Jobs (#46c).
Unit: Batch Processing and Metadata Management
Date last updated: 07/23/2018
Date of next review: July 2019
Note: Access to LS Tools is limited to trained technical services staff, divided into several levels, depending upon the degree of complexity and the impact of the tools accessible at each level.
- Staff with basic-level access can execute pre-written scripts created by the Library Systems Office to load vendor supplied MARC records and create basic modification jobs. Staff member must have a basic understanding of and familiarity with Windows-related software. Loads often require manual examination of discards, requiring the understanding of procedures and workflows associated with the materials being loaded.
...
*NOTE: Some scripts are placed at this level while they are in the development stage and then later moved to a lower level so that the staff assigned to run it may access it once it is ready for production. Some scripts that are one-time, large scale projects are placed here for use by expert-level users, and removed once the project is complete.
...
Contact: Gary Branch
Unit: Database Management Services
Date last updated: 07/22/2016
Date of next review: July 2017
...
For EDI invoices download and ftp placement in the Voyager\EDI\Incoming folder for the Acquisitions client to load. After selecting the button for either Law or LTS, the specific job selected will connect to the vendor’s ftp server’s invoice folder and allow automatic placement of the file into the production server. For all of the below, In LS Tools under "EDI Import (LTS or LAW)/[vendor name]."
a. AuxAmateurs_firm (remote)
b. AuxAmateurs_standing (remote)
c. Casalini (remote)
d. EBSCO
e. Harrassowitz_firm (remote)
f. Harrassowitz_law (remote)
g. Harrassowitz_serials (remote)
h. Harrassowitz_standing (remote)
i. Hein_Law (remote) *EXCEPTION: This job is done manually by Tech Services at Law.
j. Nardecchia (remote)
k. Yankee (remote)
...
Section for tools used in the editing and manipulation of .mrc record files.
a. ENACT BulkImport: Takes a MARC file that has been queued for bulk import and loads the file into Voyager.
b. Modify .MRC: Add, modify, or delete fields in a set of MARC records. Certain common additions are available under the “Special” button.
c. Remove Carriage Returns: Can check for or remove carriage returns in MARC, harvest jobs, modification jobs, or non-MARC output lists.
d. View .LIST: Can examine a list of bib or holdings ids.
e. View MARC Records: Can reorder MARC fields in a file based on rules from cataloging, display a MARC file in the browser, or convert a MARC file to MARCxml.
...
Loads electronic record sets supplied by vendor. Modifies the records per script prior to the load.
a. Alexander Street Press: Vendor record load, run from a .pl script in the input/vendor records/Alex Street folder. Files are downloaded from the vendor website.
b. Bach: Vendor record load, run from a "mod job" in staff member's LS Tools folder.
c. BNA e-Journals: Vendor record load, run from a "mod job" in staff member's LS Tools folder. Both new and updated records are downloaded from the vendor's website.
d. Books 24x7: Vendor record load, run from a "mod job" in staff member's LS Tools folder. Records are retrieved via FTP.
e. Columbia International Affairs Online (CIAO) e-Books: Harvests Voyager records for Columbia International Affairs Online and compares to file of OCLC records for the resource. Discards those without a valid link to the CIAO resource and duplicates in Voyager to prepare a file for loading.
f. Coutts MyiLibrary: Loads ebook records from Coutts MyiLibrary. Records are retrieved via FTP.
g. DRAM e-Resources (OCLC): Run as needed, this script is a manual load of records for music e-books. Records for this job are downloaded from the OCLC website.
h. Early Arabic Books: Vendor load of ebooks. Files are received via email.
i. ElgarOnline: Vendor load of law e-book records for ElgarOnline records. Files are received via email.
j. e-Thesis: This manual load takes metadata from both the grad school offices and from e-Commons to create .mrc records for the electronic versions of theses written by Cornell students. Records are delivered via email from the grad school office.
k. HeinOnline Legal Classics Library AND HeinOnline World Trials Collection: Takes a file of Hein package records, compares those in Voyager, retaining all links to other Hein packages while adding appropriate 899 codes.
l. Henry Stewart Talks: Vendor record load, run from a .pl script in the Batch_Processing/Scripts folder. Email is sent to staff to announce availability of new records, then files are downloaded from the vendor website.
m. Kanopy: Vendor record load, run from a .pl script in the Batch_Processing/Scripts folder.
n. Lexis Record Load: Prepares a file of Lexis e-resources. Job is similar to the HeinOnline Load.
o. Lexis-Nexis Law Overdrive: Vendor load of e-book records. Records are sent via email to Law Technical Services, where this script is run and records are loaded.
p. Memso e-Books: Vendor load of e-book records. Records are received quarterly via email.
q. Morgan Claypool: Vendor load of e-book records. Files are downloaded from the vendor website.
r. Naxos: Vendor load of e-book records. Files are downloaded from the vendor website.
s. NBER: Vendor load of e-book records. Records are imported from the NBER website. Run as needed.
t. NPAC-LC Jakarta: Vendor load of e-book records, uses .pl script to extract LCCNs from Jakarta and batch search Voyager for missing records, edit and load into Voyager.
u. OverDrive E-Books and Audio Books: Vendor load of both e-book and audiobook records, uses .pl script in the Batch_Processing/Scripts folder. Email is sent to staff to announce availability of new records, then files are downloaded from the vendor website.
v. PLI Discover Plus: Vendor load of e-book records for PLI Discover Plus Brev and PLI Discover Plus Treatise records. Files are received via email.
w. RAND E-books: Vendor load of e-book records for RAND.
x. Westlaw Record Load: Prepares a file of Westlaw e-resources. Job is similar to the HeinOnline load.
...
Acquisitions bulk import of vendor supplied MARC records with or without purchase order creation via embedded order data. Selection of this button in LS Tools gives a choice of various pre-scripted jobs connecting to a folder on the vendor’s website or locally on a server.
a. AuxAmateurs Approvals: Choose from AuxAmateurs FineArts Print, AuxAmateurs ILR Print, or AuxAmateurs Olin Print. Loads print records from vendor AuxAmateurs that were retrieved via FTP.
b. Berenguer Approvals: Similar to Yankee as it connects to our libfiles server, but it this case loads a vendor supplied preliminary record. Files are received via email attachment.
c. Casalini Basic Approval, Casalini Enhanced Approval, and Casalini Full Approval: Loads print records from Casalini, which were retrieved via FTP.
d. Chinese Approvals: This is a two-step task. We receive a file of MARC records for Chinese blanket order books that is loaded as "On Order" while the books are in shipment. We then reload the file on arrival, creating a second holdings record. We batch delete the original holdings and do editing as appropriate for Oversize, accompanying material, and multi-volume works.
e. Coutts PDP: Loads patron directed print records from Coutts, which were retrieved via FTP.
f. Coutts Approval: Extracts ISBNs from Yankee file, searches OCLC, prepares file of best available Bibs, bulk imports them into Voyager.
g. Eastview Print: Loads acquisitions-level print records from vendor.
h. Euyloo: In this manual job, the staff member takes a tab-delimited file of data from the vendor, converts it into MARC records using MarcEdit, and loads it into Voyager.
i. Harrassowitz Full Print: Selecting this will load full level Harrassowitz MARC records and create purchase orders. Connects directly to the Harrassowitz vendor’s server and pulls in the file (via FTP) for processing.
j. Harrassowitz Regular Print: Same as Harrassowitz Full Print, but does not have full level records.
k. Icelandic Firm Orders: The selector for the Icelandic collection sends a .mrc file of acquisition level records to the cts-bulk email list; these records are then edited by .pl script and loaded into Voyager.
l. Leila: Vendor-supplied records are edited using an LS Tools modification job kept in a staff member's LS Tools folder.
m. Lexicon: Vendor print load of preliminary records. Files are received via email.
n. LC Cairo Arabic Records: Vendor load of print records. Files are delivered via email; statistics and local fields are added and records are loaded into Voyager.
o. MIPP Print: Loads preliminary-level vendor records.
p. PanMun: In this manual job, the staff member takes a tab-delimited file of data from the vendor, converts it into MARC records using MarcEdit, and loads it into Voyager.
q. QingYin Firm Order: Loads preliminary print ordering records from the vendor.
r. Serbica: Vendor print load of preliminary records. Files are received via email.
s. SofiaBooks Print: Loads preliminary print ordering records from the vendor. Records are received via email attachment.
t. Vietnamese Print: In this manual job, the staff member takes a tab-delimited file of data from the vendor, converts it into MARC records using MarcEdit, and loads it into Voyager.
...
Take a set of bib or holdings ids, either generated by a harvest or other means, and extract the associated MARC records.
...
Develop queries for the extraction of bibs or holdings based on specific field criteria.
...
Specialized loads that require additional expertise on the part of the staff member to run.
a. Authorities Names Load: Loads vendor-supplied name authority records into Voyager.
b. Authorities Subjects Load: Loads vendor-supplied subject authority records into Voyager.
c. Marcadia BatchMatch: Takes less-than-full bibs and sends to OCLC for automated matching with full records, retrieves, bulk imports them into Voyager. First part of process: In LS Tools under "Reports and Extracts/OCLC Batch Records Export." Creates extract that is to be sent to OCLC. Then the BatchMatch script loads the resulting records.
d. Marcive: Loads Federal Government e-resource records. Creates files of new and existing Marcive records for loading.
e. PDC Building Plans at the Annex: Each month Cornell's Planning, Design and Construction Department sends us an email with a spreadsheet of barcodes for design plans that are headed to the Library Annex. We use a script to add item records to suppressed records for each barcode with a roll number in the enumeration. Those plans are then sent to the Annex for processing.
f. Skillsoft Books 24x7 Videos: Load of vendor records for e-videos. Records for this job are retrieved via the vendor's FTP site.
...
Location for more advanced reports and record loads with higher stakes if mistakes happen.
a. Add Donor Names to 902$a: Checks fund code in Bib against fund code table, adds name of donor(s) to 902 field.
b. Add Donor Plate Links: Monthly extract of approved invoice line items paid for on endowed gift funds. The fund is linked to a specific donor book plate which has been placed on the web. A link to that plate is placed in the holdings as an 856 field.
c. Bib Flip 490 to 830 Records: Adds 830 series field to records with only a 490 field for search ability in Voyager.
d. Enhance Level 3 Encoding Records: Finds lvl3 records in Voyager, uses ISBN to search OCLC for a better record, merges found record with Voyager record to improve encoding level.
e. Fast Headings Update Via OCLC Spreadsheet: This script looks for Voyager records that lack fast subject headings and creates a .mrc file for export to OCLC.
f. Icelandic BIB Cleanup: This job adds a 710 "Fiske Icelandic Collection" to Icelandic records, and is run from a "modify" job in the LS Tools folder of a staff member.
g. Initial Article and Title Tag Cleanup: Looks at previous month’s cataloging for errors in indicators relating to articles in title fields.
h. Load Serial Solutions e-Book Metadata: Prepares Serial Solutions e-book file for bulk import.
i. Load Serial Solutions e-Book-Delete Metadata: Prepares Serial Solutions e-book deletion file for bulk import and batch deletion. Produces histogram of 899 counts.
j. Load Serial Solutions Journal Metadata: Prepares Serial Solutions e-journal file for bulk import and a separate file for batch deletion.
k. Remove Unnecessary 948 Fields from Voyager: This is a monthly maintenance job to limit the number of 948 fields in individual BIB records.
l. Replace Tech Services Statistics: This script allows an operator to manually gather the 948 data for a desired date range.
m. Update Backlog Books: Finds “in process” records for desired storage code in Voyager, searches OCLC by ISBN for a better record, merges with Voyager record, and queues for loading into Voyager.
...
Allows selection of bulk import profile to load into production or test version of Voyager.
...
Used after loading authority record files to perform global headings jobs 11 and 12, which index bib and authority headings in global headings queue for authority maintenance.
...
Reports and processes that the Batch Processing or LTS management has access to, often a space to test new processes.
a. Columbia POOF Vendor Matrices: Displays web version of vendor rule table for POOF loads.
b. Configure LS Tools Vendors: Sets up local directory, .pl script, etc. associated with each vendor job.
c. Configure Vendor Connection: Used to set up FTP usernames and passwords for access to vendor data files.
d. Convert DOAB MARC for Blacklight: Takes .mrc file of open access books and prepares for import into integrative layer of Blacklight.
e. Cornell POOF Vendor Matrices: Displays web version of vendor rule table for POOF and ITSO loads.
f. ITSO Records from Local Data: Is used as a back-up for a CRON job. Takes WorldCat records taken from manual download, and loads them into Voyager.
g. ITSO Records from OCLC: Takes WorldCat records chosen by selectors for ordering, loads bibs and creates POs in Voyager. ITSO is the script that does this work.
h. ITSO/POOF Analysis: Looks at the number of instances of a rule's use.
i. Kirtas Record Creation: Goes to internet archive, grabs data, looks at print records and creates record for e-resource version of the title.
j. POOF Records from Columbia: Prepares POOF order file for Columbia.
k. POOF Records from Cornell: Loads Cornell's POOF records.
l. Replace Columbia POOF Vendor Matrix: Infrequent manual job - Updates tables for automated ordering for vendors.
m. Replace Cornell POOF Vendor or Fund Matrices: Infrequent manual job - Updates tables for automated ordering for vendors.
n. Replace Harvard POOF Vendor Matrix: Infrequent manual job - Updates tables for automated ordering for vendors.
o. TS Stats Upload to Database: The monthly CRON pushes a package of monthly 948 data to “Disc”; in this job, a staff member pushes that same package to the “DB” (database) used by supervisors for creating individual and department statistics reports.
...
Section of scripts that allow for the manual running of a CRON job. Sub-sections include OCLC, General, and Privileged Cron Jobs.
a. 948 Cleanup Maintenance: Each day we receive a report of incorrectly coded 948 statistics fields. We manually correct those, so that they can then be added to our next statistics extract.
b. Automated Monthly Export of No-Ex Records: This script extracts proprietary records for matching in OCLC, without adding the vendor's .mrc record to WorldCat. If it finds a matching record, it adds the OCLC number to the Voyager record.
c. Cleanup 008 Single Date + Cleanup 008 Multi Date: Compares date in 260 or 264 to contents of 008/07-14, corrects content to make fields "congruent."
...
e. Daily Technical Services Statistics: This CRON job runs daily, and gathers the 948 data from the previous day's records.
f. E-Book Enhancement Scripts: Three scripts for harvesting e-resource bib records lacking call numbers. One searches OCLC using the 035, the second searches OCLC by 776, and the third searches by ISBN. If an OCLC record with call number is found, it is merged with Voyager record and queued for loading into Voyager.
g. EDI Export from Voyager Server: Sends EDI orders to vendors.
h. Eliminate Redundant OCLC 035 Fields: Reviews previous week’s records in Voyager, finds malformed or duplicate OCLC 035 fields, removes duplicates and malformed instances, reports out BIBs that require further examination for manual cleanup.
i. Generate MetaData for RAPID: Generates tab delimited list of serials or monographs for export to RAPID for Inter-Library Loan use.
j. Monthly Technical Services Statistics: Gathers statistical information for technical services statistics database.
k. Notify Selectors of Rejected Records: Pickup of records for materials not ordered for delivery to selector who placed the order.
l. OCLC 035 Cleanup: Each Friday a cron job looks at updated Voyager records of the previous week to identify those records with multiple OCLC numbers. Most are duplicate numbers that can be eliminated during the automated portion of the cleanup. The rest are reported out for manual correction, as appropriate.
m. OCLC Daily Export w/Batch Delete: Prepares for export that day’s cataloging and deletions for export to OCLC.
n. OCLC Institutional Export: Weekly export to OCLC of Law Library and pre-1900 imprint updates and newly cataloged material.
o. OCLC Manuscript-Archive Export: Weekly export to OCLC of updated manuscript and archive bibs.
p. OCLC Master Number Maintenance: Checks master number against OCLC and adds fast headings to Voyager record, if found on master but missing in Voyager.
q. OCLC MFHD Export (Local Holdings Records): Choose "3 day old non-serial mfhd export to OCLC" to send local holdings data to OCLC for non-serial records. Choose "3 day old serial mfhd export to OCLC" to send local holdings data to OCLC for serial records.
r. OCLC System Number Mapping: Better known as OCLC mapping job, as it adds the OCLC number to new cataloging from OCLC cross reference reports.
s. OCLC WorldShare: Takes WorldShare records from OCLC ftp, searches Voyager for matching records, merges where incoming record is fuller than existing.
t. Record Extraction to Coutts OASIS: Extracts added or changed bib records to Coutts for loading into Oasis.
u. Records Missing OCLC Numbers: Runs harvest of cataloged records that are lacking OCLC numbers. Resubmits monograph records to OCLC; prints out list of serials lacking OCLC numbers, to be reviewed by staff.
v. Report of Bibs with Duplicate ISBNs: Reports out bibs with duplicate ISBNs based on the previous days 005 date.
...
...
Basic Access Group (See Access Description)
Advanced Access Group (See Access Description)
Batch Processing/Heads/Expert-Level Access Group (See Access Description)
Administrative Group (See Access Description)
...
Anchor basic basic
Basic Access Group
| basic | |
| basic |
I. “General Update” Section of LS ToolsAnchor GU GU
A. Move LTS or Law EDI File to Voyager Folder (CGI08 for LTS, CGI09 for Law*): For EDI invoices download and ftp placement in the Voyager\EDI\Incoming folder for the Acquisitions client to load. After selecting the button for either Law or LTS, the specific job selected will connect to the vendor’s ftp server’s invoice folder and allow automatic placement of the file into the production server. For all of the below, In LS Tools under "EDI Import (LTS or LAW)/[vendor name]."
a. AuxAmateurs_firm (remote)
b. AuxAmateurs_standing (remote)
c. Casalini (remote)
d. EBSCO
e. Harrassowitz_firm (remote)
f. Harrassowitz_law (remote)*
g. Harrassowitz_serials (remote)
h. Harrassowitz_standing (remote)
i. Hein_Law (remote)*
j. Nardecchia (remote)
k. Yankee (remote)
B. Check for/Remove Carriage Returns (CGI19): Can check for or remove carriage returns in MARC, harvest jobs, modification jobs, or non-MARC output lists.
C. MARC Modification Job (CGI12): Add, modify, or delete fields in a set of MARC records. Certain common additions are available under the “Special” button.
D. ENACT Bulk Import (CGI15): Takes a MARC file that has been queued for bulk import and loads the file into Voyager.
E. Prepare a Vendor e-Book, e-Journal, e-Audio, or e-Video Load: Loads electronic record sets supplied by vendor. Modifies the records per script prior to the load.Anchor VEBK VEBK
a. Alexander Street Press (Not in LS Tools): Vendor record load, run from a .pl script in the input/vendor records/Alex Street folder. Collections include e-books, e-videos, e-audio. Files are downloaded from the vendor website, loaded quarterly.
b. BNABloom e-Journals (New (VEL16) and Revised (VEL17)): Formerly BNA e-Journals. Vendor record load, run by Law Tech Services staff. Both new and updated records are downloaded from the vendor's website.
c. DDA Proquest Ebooks (VEL09): Loads ebook records from vendor. Records are retrieved via FTP.
d. Early Arabic Books (Not in LS Tools): Vendor load of ebooks. Files are received via email.
e. ElgarOnline (VEL10): Vendor load of law e-book records for ElgarOnline records. Files are received via email.
f. e-Thesis (Not in LS Tools): This manual load takes metadata from both the grad school offices and from e-Commons to create .mrc records for the electronic versions of theses written by Cornell students. Records are delivered via email from the grad school office.
g. HeinOnline Legal Classics Library AND HeinOnline World Trials Collection (VEL04): Takes a file of Hein package records, compares those in Voyager, retaining all links to other Hein packages while adding appropriate 899 codes.
h. Henry Stewart Talks (Not in LS Tools): Vendor record load, run from a .pl script in the Batch_Processing/Scripts folder. Email is sent to staff to announce availability of new records, then files are downloaded from the vendor website.
i. Lexis Record Load (VEL08): Prepares a file of Lexis e-resources. Job is similar to the HeinOnline Load.
j. Lexis-Nexis Law Overdrive (VEL05): Vendor load of e-book records. Records are sent via email to Law Technical Services, where this script is run and records are loaded.
k. LLMC Digital (VEL07): Vendor load of e-book records. Law Technical Services Librarian accesses vendor website and harvests spreadsheet.
l. Memso e-Books (Not in LS Tools): Vendor load of e-book records. Records are received quarterly via email.
m. National Film Board of Canada (Not in LS Tools): Vendor load of streaming video records, run manually by Batch Processing staff.
n. Naxos (Not in LS Tools): Vendor load of e-book records. Files are downloaded from the vendor website.
o. NBER (Not in LS Tools): Vendor load of e-book records. Records are imported from the NBER website. Run as needed.
p. NPAC-LC Jakarta (Not in LS Tools): Vendor load of e-book records, uses .pl script to extract LCCNs from Jakarta and batch search Voyager for missing records, edit and load into Voyager.
q OverDrive E-Books and Audio Books (Not in LS Tools): Vendor load of both e-book and audiobook records, uses .pl script in the Batch_Processing/Scripts folder. Email is sent to staff to announce availability of new records, then files are downloaded from the vendor website.
r. PLI Discover Plus (Brev VEL14 and Treatise VEL15 ): Files are received via email.
s. RAND E-books (Not in LS Tools): Vendor load of e-book records for RAND.
t. Skillsoft E-books (BPro VEL12, ITPro VEL11, ENGPro VEL13): Vendor record load for Skillsoft Ebooks, formerly BKS24x7. Records are retrieved via FTP.
u. Skillsoft E-videos (ITProdesktop MSC07, LDC MSC08): Vendor record load for Skillsoft Evideos, formerly BKS24x7. Records are retrieved via FTP.
F. Prepare a Vendor Print Load: Acquisitions bulk import of vendor supplied MARC records with or without purchase order creation via embedded order data. Selection of this button in LS Tools gives a choice of various pre-scripted jobs connecting to a folder on the vendor’s website or locally on a server.Anchor VPRT VPRT
a. Andinos (VPR22): Vendor Print load for Andinos/Booksur. Staff member retrieves and places .mrc files in vendor record folder.
b. AuxAmateurs Approvals: Choose from AuxAmateurs FineArts Print (VPR10), AuxAmateurs ILR Print (VPR11), AuxAmateurs Olin Print (VPR12) or AuxAmateurs Olin2 Print (VPR19). Loads print records from vendor AuxAmateurs that were retrieved via FTP.
c. Bach (VPR33): Vendor print record load.
d.Berenguer Approvals(VPR01): Similar to Yankee as it connects to our libfiles server, but it this case loads a vendor supplied preliminary record. Files are received via email attachment.
e. Cambeiro Print (VPR23): Vendor print load. Staff member retrieves and places .mrc files in vendor record folder.
f. Casalini Approvals (VPR02): Loads print records from Casalini, which are retrieved via FTP.
g. Chinese Approvals (Not in LS Tools): This is a two-step task. We receive a file of MARC records for Chinese blanket order books that is loaded as "On Order" while the books are in shipment. We then reload the file on arrival, creating a second holdings record. We batch delete the original holdings and do editing as appropriate for Oversize, accompanying material, and multi-volume works.
h. Coutts PDP (VPR08): Loads patron directed print records from Coutts, which were retrieved via FTP.
i. Coutts Approval (VPR09): Extracts ISBNs from Yankee file, searches OCLC, prepares file of best available Bibs, bulk imports them into Voyager.
j Derex Print (VPR03):
k. Eastview Print (VPR13): Loads acquisitions-level print records from vendor.
l.Euyloo(Not in LS Tools): In this manual job, the staff member takes a tab-delimited file of data from the vendor, converts it into MARC records using MarcEdit, and loads it into Voyager.
m. Harrasowitz Music Records (VPR05)
n. Harrassowitz Full Print (VPR06): Selecting this will load full level Harrassowitz MARC records and create purchase orders. Connects directly to the Harrassowitz vendor’s server and pulls in the file (via FTP) for processing.
o. Harrassowitz Regular Print (VPR04): Same as Harrassowitz Full Print, but does not have full level records.
p.Icelandic Firm Orders(Not in LS Tools): The selector for the Icelandic collection sends a .mrc file of acquisition level records to the cts-bulk email list; these records are then edited by .pl script and loaded into Voyager.
q. Itturiaga Print (VPR21):
r.Korean Print(Not in LS Tools): Vendor-supplied records processed manually by inputting staff member.
s. Kozmenko Print (VPR13)
t. Leila (VPR18): Vendor-supplied records are edited using an LS Tools modification job kept in a staff member's LS Tools folder.
u. LC Cairo Arabic Records (VPR31): Vendor load of print records. Files are delivered via email; statistics and local fields are added and records are loaded into Voyager.
v. Lexicon (VPR16): Vendor print load of preliminary records. Files are received via email.
w. MIPP Print (VPR14): Loads preliminary-level vendor records.
x. PanMun (Not in LS Tools): In this manual job, the staff member takes a tab-delimited file of data from the vendor, converts it into MARC records using MarcEdit, and loads it into Voyager.
y. Retta Print (VPR07):
z. Serbica (VPR17): Vendor print load of preliminary records. Files are received via email.
aa. SofiaBooks Print (VPR15): Loads preliminary print ordering records from the vendor. Records are received via email attachment.
bb. Vietnamese Print(Not in LS Tools): In this manual job, the staff member takes a tab-delimited file of data from the vendor, converts it into MARC records using MarcEdit, and loads it into Voyager.
II. “Voyager Harvest” Section of LS ToolsAnchor #VHarvest #VHarvest
A. MARC File Examination (CGI17): Can reorder MARC fields in a file based on rules from cataloging, display a MARC file in the browser, or convert a MARC file to MARCxml.
B. View Harvest Output (CGI18): Can examine a list of bib or holdings ids.
C. Develop a Harvest query to identify Voyager Records (CGI11): Develop queries for the extraction of bibs or holdings based on specific field criteria.
D. Extract MARC Records from Voyager (CGI13): Extracts MARC Records from Voyager using a list of BIB or MFHD id's.
III. “CLAMMS subsystem” (CGI28): Also known as the Automated Media Shelflisting System, this tool allows catalogers to determine the proper accession number for videos and microforms. Access is given only to those individual catalogers who need to use it.Anchor CLAMMS CLAMMS
Anchor ADV ADV
Advanced Access Group
| ADV | |
| ADV |
I. “Voyager Update Utilities” Section of LS ToolsAnchor VUU VUU
A. Prepare a Miscellaneous MARC Input File: Specialized loads that require additional expertise on the part of the staff member to run.
a. Authorities Names Load (MSC04): Loads vendor-supplied name authority records into Voyager. (Global Headings job(s) are also part of this process)
b. Authorities Subjects Load (MSC03): Loads vendor-supplied subject authority records into Voyager. (Global Headings job(s) are also part of this process)
c. BatchMatch*: Takes less-than-full bibs and sends to OCLC for automated matching with full records, retrieves, bulk imports them into Voyager. First part of process: In LS Tools under "Reports and Extracts/OCLC Batch Records Export." Creates extract that is to be sent to OCLC. Then the BatchMatch script loads the resulting records. *OCLC Batchmatch Monthly by Date Ranges is a cron (CRN38); OCLC Batchmatch by Bib Id Range can be run at any time via Privileged Procedure (PR338).
d. Coutts Shelf Ready (MSC05): Processes and loads records for shelf-ready books, reports out records requiring manual intervention for oversize, multivol, added pieces, etc.
e. General OCLC MARC Selection from a List of ISBNS (MSC09): Takes ISBNs, runs them against the OCLC rule matrix and finds the best available matching record.
f. Marcive (MSC02): Loads Federal Government e-resource records. Creates files of new and existing Marcive records for loading.
g. PDC Building Plans at the Annex (Not in LS Tools): Each month Cornell's Planning, Design and Construction Department sends us an email with a spreadsheet of barcodes for design plans that are headed to the Library Annex. We use a script to add item records to suppressed records for each barcode with a roll number in the enumeration. Those plans are then sent to the Annex for processing.
B. Execute a Privileged Production Procedure: Location for more advanced reports and record loads with higher stakes if mistakes happen.Anchor PRIVPROD PRIVPROD
a. Add Donor Names to 902$a (PR326): Checks fund code in Bib against fund code table, adds name of donor(s) to 902 field.
b. Add Donor Plate Links (Not in LS Tools): Monthly extract of approved invoice line items paid for on endowed gift funds. The fund is linked to a specific donor book plate which has been placed on the web. A link to that plate is placed in the holdings as an 856 field. This job is run from a .pl script.
c. Bib Flip 490 to 830 Records (PR321): Adds 830 series field to records with only a 490 field for search ability in Voyager.
d. Create 856 link for Table of Contents Images (PR707): Scans serial volume tables of contents, uploads .pdf's tagged with the bib id. Adds an 856 for the URL and reloads.
e. Enhance Level 3 Encoding Records(PR324): Finds lvl3 records in Voyager, uses ISBN to search OCLC for a better record, merges found record with Voyager record to improve encoding level.
f. Enhance Voyager Records with Vernacular Characters (PR333): Using the OCLC Search API and the process described in the specification Component: OCLC Record Retrieval we will identify the best possible match to our Voyager record and copy vernacular name and title fields to our record and load back into voyager.
g.Fast Headings Update Via OCLC Spreadsheet (PR714): This script looks for Voyager records that lack fast subject headings and creates a .mrc file for export to OCLC.
h. General Bib field Cleanup (CRN27): This script searches for invalid fields, subfields, and indicators in BIB records, and reports out the errors for manual cleanup.
j. Initial Article and Title Tag Cleanup (PR327): Looks at previous month’s cataloging for errors in indicators relating to articles in title fields.
k. Order Record Enhancements from OCLC (CRN35): Running every Sunday, this job looks at preliminary level records with a date of the past thirty days, searches for a better record in OCLC, and merges it with the Voyager record if found.
l. Serial Solutions e-Books Combined (PR354): Prepares Serial Solutions e-book files for bulk import, prepares Serial Solutions e-book deletion file for bulk import and batch deletion.
m. Serial Solutions Journals Combined (PR344): Prepares Serial Solutions e-journal file for bulk import and a separate file for batch deletion.
n. Update Backlog Books (PR322): Finds “in process” records for desired storage code in Voyager, searches OCLC by ISBN for a better record, merges with Voyager record, and queues for loading into Voyager.
C. Queue BulkImport (CGI34): Allows selection of bulk import profile to load records into production or test version of Voyager.
D. Run a Voyager Utility Program (VOY01, VOY02, VOY03): Used after loading authority record files to perform global headings jobs 11 and 12, which index bib and authority headings in global headings queue for authority maintenance.
II. “Voyager Reporting Utilities” Section of LS ToolsAnchor VRep VRep
Location for scripts that create reports or extract records as part of routine database maintenance processes.
A. Execute a Report or Extract
a. MFHD Copy Number Clean-up (PR101): Compare the copy number in the MFHD 852 subfield t with the copy number in the item record. Output contains some routine fixes which are extracted for manual correction.
b. General Withdrawals Report (PR102): Generates a count by library of items withdrawn over a given time period, typically the prior month.
c. ISBN Extract (PR108): Extracts ISBNs from a vendor MARC record file. Staff member can then batch search OCLC using these ISBNs.
d. Report Duplicate Government Documents (PR110): Extracts a list of government document Voyager bibs where the OCLC number and 245 subfield a match.
B. Match BibIDs from external list: Bibs matching standard numbers (PR106)
C. Other reports/extracts, which are NOT in LS Tools
a. OCLC LHR Exception Reports (Not in LS Tools): Receive email announcements of our local holdings exports. The Exception Reports can be examined on the OCLC Produce Services website. Any holding that was not added will have our Institutional Record Number, i.e., our holdings ID along with a reason why it was not loaded. Staff member must correct each holding in Voyager as appropriate so that the holding will be resent to OCLC. (This is not an LS Tools script.)
b. OCLC Unresolved Records (Not in LS Tools): Receive email announcements of our daily cataloging export results. Any coded as unresolved appear in a MARC file which is downloaded from OCLC Product Services website. Our holdings for these records were not added to WorldCat due to MARC errors. Staff member must manually examine each record in Connexion to correct error(s) and add our holdings to OCLC.
Anchor EXPERT EXPERT
Batch Processing/Heads/Expert-Level Access Group
| EXPERT | |
| EXPERT |
I. “ALL LS Tools Production” Section of LS ToolsAnchor AllProd AllProd
A. Business Analysis Web Displays
a. ITSO/POOF Analysis (CGI21): Analyzes the orders placed through POOF; looks at the number of instances of a rule's use.
b. LS Tools Usage Analysis (CGI22)
c. BulkImport Statistics (CGI23)
d. Batch Business Analysis (CGI29)
B. Cron Jobs for Interaction with OCLC
a. OCLC System Number Mapping (CRN05): Better known as OCLC mapping job, as it adds the OCLC number to new cataloging from OCLC cross reference reports.
b. Automated Monthly Export of No-Ex Records (CRN07): This script extracts proprietary records for matching in OCLC, without adding the vendor's .mrc record to WorldCat. If it finds a matching record, it adds the OCLC number to the Voyager record;
c. Records with no OCLC numbers (CRN08): Runs harvest of cataloged records from the previous month that are lacking OCLC numbers. Resubmits monograph records to OCLC; prints out list of serials lacking OCLC numbers, to be reviewed by staff.
d. WorldShare Local Holdings Export to OCLC (CRN10): Choose "3 day old non-serial mfhd export to OCLC" to send local holdings data to OCLC for non-serial records. Choose "3 day old serial mfhd export to OCLC" to send local holdings data to OCLC for serial records.
e. OCLC Manuscript-Archive Export (CRN12): Weekly export to OCLC of updated manuscript and archive bibs.
f. OCLC Daily Delete (CRN15): Daily export to OCLC of holdings to be deleted.
g. OCLC Daily Export (CRN16): Daily export to OCLC.
h. OCLC BatchMatch for Date Ranges (CRN38): Searches for less-than-full bibs within a given date range and sends to OCLC for automated matching with full records, retrieves, bulk imports them into Voyager. First part of process: In LS Tools under "Reports and Extracts/OCLC Batch Records Export." Creates extract that is to be sent to OCLC. Then the BatchMatch script loads the resulting records. Note: OCLC Batchmatch by Bib Id Range can be run at any time via Privileged Procedure (PR338).
i. OCLC WorldShare Update (PR335): Takes WorldShare records from OCLC ftp, searches Voyager for matching records, merges where incoming record is fuller than existing.
C. Execute a Google Utility : Various scripts for Google and HathiTrust Voyager extracts. Some are run by LTS staff; some by the project manager for the Google Project and who also serves as liaison to HathiTrust.Anchor GGL GGL
a. Build MARC XML File for GOOGLE Based on List of Barcodes (GGL05): Used when Google metadata is missing or needs replacing.
b. Build MARC XML File for HATHI Based on List of Barcodes (GGL06): Used when HathiTrust metadata is missing or otherwise not useable.
c. Build MARC XML for HATHI - via IA - Kirtas digitized (GGL16): Used when HathiTrust KIRTAS metadata is missing or otherwise not useable.
d. Hathi Metadata Shipment (GGL07): Takes barcode to get BIB, transforms BIBS to MARC XML for shipment to Hathi. Used to correct previously sent data.
e. HathiTrust MultiVol Monograph Extract (GGL09): Reports status of currently held/withdrawn/lost multi-vol items to Hathi (at item level).
f. HathiTrust Serials Title Extract (GGL10): Reports currently held serials to Hathi (at title level).
g. HathiTrust Single Monograph Extract (GGL08): Reports status of currently held/withdrawn/lost single mono items to Hathi (at item level).
h. Update oai.library.cornell.edu/ (GGL24)
D. Cron Jobs with General AccessAnchor CRNGEN CRNGEN
a. Report of bibs with duplicate ISBNs (CRN03): Reports out bibs with duplicate ISBNs based on the previous days 005 date.
b. Bib call number maintenance (CRN18)
c. Serial Metadata List for RAPID (CRN28): Generates tab delimited list of serials or monographs for export to RAPID for Inter-Library Loan use/
d. Monograph Metadata List for RAPID (CRN29): Generates tab delimited list of serials or monographs for export to RAPID for Inter-Library Loan use.
e. Wrapper for CRN29 Rapid Monographs (CRN99): Generates tab delimited list of serials or monographs for export to RAPID for Inter-Library Loan use.
f. Annex boundwith items (PR103): Identifies bound-withs so that they can be flipped and empty item records can be created.
g. Combined report on Bib/Mfhd suppression (PR104): Looks in Voyager for bibs without at least one active (unsuppressed) MFHD.
h. Histogram of 899 with NoEx needing 035 OCoLC (PR105): Identifies all codes used in 899 $a for cleanup or comparison against the database of “approved” codes in use.
i. Eliminate redundant OCLC 035 fields (PR323):Reviews previous week’s records in Voyager, finds malformed or duplicate OCLC 035 fields, removes duplicates and malformed instances, reports out BIBs that require further examination for manual cleanup.
j. Daily Annex Flip and Location Report (CRN21): Identifies bound-with's so they can be flipped, and empty item records are created. This cron job is part of Voyager Utilities and runs every weekday.
E. Cron Jobs with Privileged AccessAnchor CRNPRIV CRNPRIV
a. EDI Export from Voyager Server (CRN13): Sends EDI orders to vendors.
b. Notify selectors of rejected records (CRN19): Pickup of records for materials not ordered for delivery to selector who placed the order.
c. Send e-mails to book requesters and purge 981, 980$s (CRN20): Harvest requestors’ email from day’s purchase order line items and sends emails to patron informing them that the title has been ordered for them. Removes requestor information from bib found in 981 subfield n.
...
d. General MFHD cleanup (CRN24)
e. Cleanup 008 Single Date (CRN25) and Cleanup 008 Multi Date (CRN26): Compares date in 260 or 264 to contents of 008/07-14, corrects content to make fields "congruent.”
f. General BIB field cleanup (CRN27): This script searches for invalid fields, subfields, and indicators in BIB records, and reports out the errors for manual cleanup.
g. Daily Tech Services Statistics (CRN33): This CRON job runs daily, and gathers the 948 data from the previous day's records. Each day we receive a report of incorrectly coded 948 statistics fields. We manually correct those, so that they can then be added to our next statistics extract.
h. Monthly Tech Services Statistics (CRN34): Gathers statistical information for technical services statistics database.
i. Order record enhancements from OCLC (CRN35): Running every Sunday, this job looks at preliminary level records with a date of the past thirty days, searches for a better record in OCLC, and merges it with the Voyager record if found.
j. E-Book updates from OCLC (CRN37): Three scripts* for harvesting e-resource bib records lacking call numbers. One searches OCLC using the 035, the second searches OCLC by 776, and the third searches by ISBN. If an OCLC record with call number is found, it is merged with Voyager record and queued for loading into Voyager. * Ebook Updates from OCLC Based on 035, Ebook Updates from OCLC Based on 776, Ebook Updates from OCLC Based on ISBN.
k. Order records from Coutts via Oasis with fund and location separate (CRN41): Takes WorldCat records chosen by selectors for ordering, loads bibs and creates POs in Voyager.
l. Vendor order pickup--Harrassowitz (CRN42)
m. POOF records for Cornell fund and location together (CRN43): Loads Cornell's POOF records.
n. POOF records from Columbia (CRN44): Prepares POOF order file for Columbia.
o. Bib flip 490 to 830 records (PR321): Adds 830 series field to records with only a 490 field for search ability in Voyager.
p. OCLC Master Number Maintenance Project (PR336): Checks master number against OCLC and adds fast headings to Voyager record, if found on master but missing in Voyager.
q. Daily Record Extraction to OASIS (PR705): Extracts added or changed bib records to Coutts for loading into Oasis.
r. Create 856 link for Table of Contents Images (PR707): Scans serial volume tables of contents, uploads .pdf's tagged with the bib id.
F. Execute an Administrative Production ProcedureAnchor ADPROD ADPROD
a. Rank OCLC records returned for an isbn, issn, etc. (CGI01)
b. TS STATS Summary (CGI24): The monthly CRON pushes a package of monthly 948 data to “Disc”; in this job, a staff member pushes that same package to the “DB” (database) used by supervisors for creating individual and department statistics reports.
c. Configure VENDOR CONNECTION (CGI31): Used to set up FTP usernames and passwords for access to vendor data files.
d. Configure LS TOOLS VENDORS (CGI36): Sets up local directory, .pl script, etc. associated with each vendor job.
e. Row/Column Display of POOF2 Vendor Matrices (CGI41): Displays web version of vendor rule table for POOF and ITSO loads.
f. Cornell ITSO Vendor Matrix Replacement (PR713): Updates tables for automated ordering for vendors.
g. Fast Headings updates via spreadsheet from OCLC (PR714): This script looks for Voyager records that lack fast subject headings and creates a .mrc file for export to OCLC.
h. Cornell POOF Vendor Matrix Replacement (PR716): Infrequent manual job - Updates tables for automated ordering for vendors.
i. Replace Harvard POOF Vendor Matrix (Not in LS Tools): Infrequent manual job - Updates tables for automated ordering for vendors.
...
Anchor ADMIN ADMIN
Administrative Access Group
| ADMIN | |
| ADMIN |
I. “Administer LS Tools Itself” Section of LS ToolsAnchor AdLSTools AdLSTools
A. Report/Clean Up Voyager MARC Record Errors
...
Directly export a MARC file of Voyager records to OCLC for processing.
...
Various scripts for Google and HathiTrust Voyager extracts. Some are run by LTS staff; some by the project manager for the Google Project and who also serves as liaison to HathiTrust.
a. Build MARC XML File for GOOGLE Based on List of Barcodes: Used when Google metadata is missing or needs replacing.
b. Build MARC XML File for HATHI Based on List of Barcodes: Used when HathiTrust metadata is missing or otherwise not useable.
c.Google Metadata Shipment: Takes barcode to get BIB, transforms BIBS to MARC XML for shipment to Google. Used to correct previously sent data.
d. Hathi Metadata Shipment: Takes barcode to get BIB, transforms BIBS to MARC XML for shipment to Hathi. Used to correct previously sent data.
e. HathiTrust MultiVol Monograph Extract: Reports status of currently held/withdrawn/lost multi-vol items to Hathi (at item level).
f. HathiTrust Serials Title Extract: Reports currently held serials to Hathi (at title level).
g. HathiTrust Single Monograph Extract: Reports status of currently held/withdrawn/lost single mono items to Hathi (at item level).
h. Test a File of Item Barcodes for Valid Checksum: Makes sure barcodes are valid.
...
Location for scripts that create reports or extract records as part of routine database maintenance processes.
a. General Withdrawals Report: Generates a count by library of items withdrawn over a given time period, typically the prior month.
b. Inconsistent Item Location Report: Creates a serial and non-serial list of MFHDs and items where their location codes do not match.
c. ISBN Extract: Extracts ISBNs from a MARC record file. Staff member can then batch search OCLC using these ISBNs.
d. Lost/Missing Reports: Generates a list of titles for books withdrawn due to the item being lost or missing.
e. Mfhd Copy Number Clean-Up: Compare the copy number in the MFHD 852 subfield t with the copy number in the item record. Output contains some routine fixes which are extracted for manual correction.
f. OCLC LHR Exception Reports: Receive email announcements of our local holdings exports. The Exception Reports can be examined on the OCLC Produce Services website. Any holding that was not added will have our Institutional Record Number, i.e., our holdings ID along with a reason why it was not loaded. Staff member must correct each holding in Voyager as appropriate so that the holding will be resent to OCLC. (This is not an LS Tools script.)
g. OCLC Unresolved Records: Receive email announcements of our daily cataloging export results. Any coded as unresolved appear in a MARC file which is downloaded from OCLC Product Services website. Our holdings for these records were not added to WorldCat due to MARC errors. Staff member must manually examine each record in Connexion to correct error(s) and add our holdings to OCLC. (This is not an LS Tools script.)
h. Report Duplicate Government Documents: Extracts a list of government document Voyager bibs where the OCLC number and 245 subfield a match.
i. Suppressed Mfhds with Unsuppressed Bibs: Looks in Voyager for bibs without at least one active (unsuppressed) MFHD.
...
This section of LS Tools contains a set of tools that are currently in development. Each of them contains a menu of fields/issues that can be chosen to be either reported upon or corrected. The "Clean Up" tools offer the options of checking the length of the 006 and correcting its length if necessary, checking that the pub date matches the 008 date and correcting it, dropping leading zeros or extra text from 035 OCoLC fields, removing 995 Hivolm fields, and/or correcting malformed 948 fields. The "Report" tools offer the options of reporting out bibs that contain 880 fields but no corresponding 066 fields, reporting out malformed 948 fields, and/or reporting invalid subfields within a bib record.
a. Clean Up a Local MARC File CleanUP Voyager data (CLN01): This job can take a .mrc file from a staff member’s LS Tools/Marc folder can take a set of Voyager BIB records and clean up whatever issues are chosen from the set menu of common problems.
b. Clean Up Voyager BIB Data CleanUP a local MARC file (CLN02): This job can take a set of Voyager BIB records .mrc file from a staff member’s LS Tools/Marc folder and clean up whatever issues are chosen from the set menu of common problems.
c. Report on a Local MARC FileVoyager data (CLN11): Creates a report of any and all of three specific errors found in a .mrc file from a staff member’s LS Tools/Marc folder.d. Report Voyager BIB Dataset of Voyager BIB records.
d. Report on a local MARC file (CLN12): Creates a report of any and all of three specific errors found in a set of Voyager BIB records.
...
a .mrc file from a staff member’s LS Tools/Marc folder.
B. Configure LS Tools Tables & Directories
The jobs in this section are used by administrators to edit existing LS Tools jobs’ defaults, add new users, edit permission levels for various jobs, etc.
a. Configure CRON Job Defaults
b. Configure e-Book Aggregators
c. Configure JOB CONTROL (“Families”)
d. Configure LS Tools JOBS
e. Configure LS Tools USERS
f. Configure Vendor Fund Mapping
g. LS Tools Usage Analysis
...
etc.
a. Configure LS TOOLS USERS (CGI26)
b. Configure Ebook Aggregators (CGI32)
c. Configure Vendor Fund Mapping (CGI34)
d. Configure LS TOOLS JOBS (CGI35)
e. Configure Cron Job Defaults (CGI37)
f. Configure JOB CONTROL (Families) (CGI38)
...
...
This tool allows catalogers to determine the proper accession number for videos and microforms. Access is given only to those individual catalogers who need to use it.