*** This version of Confluence is for testing only and contains a copy of content from June 29th 2026. No changes will be preserved. ***
DEPRECATED THIS PROCEDURE IS NO LONGER UPDATED
Scope: This procedure provides instructions for running automated import, export, cleanup, reporting and various related jobs using scripts found in LS Tools web site and other tools created by the Library Systems Office and LTS Batch Processing staff. For issues surrounding the scheduling of large batch loads, please consult Bulk Loading: Scheduling (#46d). Unless otherwise noted, the items listed below are descriptions of scripts and tools found on LS Tools. Jobs may include scripts known as "mod jobs" run from a staff member's individual LS Tools folder, as opposed to scripts that are accessible directly from the LS Tools web interface. Non-LS Tools jobs are most commonly referred to as "manual" jobs. For information on staff batch job assignments, please refer to the Bulk Loading: List of Jobs (#46c).
Contact: Gary Branch Jenn Colt
Unit: Batch Processing and Metadata Management
Date last updated: 07/23/2018
Date of next review: July 2019
Note: Access to LS Tools is limited to trained technical services staff, divided into several levels, depending upon the degree of complexity and the impact of the tools accessible at each level.
- Staff with basic-level access can execute pre-written scripts created by the Library Systems Office to load vendor supplied MARC records and create basic modification jobs. Staff member must have a basic understanding of and familiarity with Windows-related software. Loads often require manual examination of discards, requiring the understanding of procedures and workflows associated with the materials being loaded.
...
...
Basic Access Group (See Access Description)
Advanced Access Group (See Access Description)
Batch Processing/Heads/Expert-Level Access Group (See Access Description)
Administrative Group (See Access Description)
...
Anchor basic basic
Basic Access Group
| basic | |
| basic |
Anchor GU GU
...
*NOTE: Some scripts are placed at this level while they are in the development stage and then later moved to a lower level so that the staff assigned to run it may access it once it is ready for production. Some scripts that are one-time, large scale projects are placed here for use by expert-level users, and removed once the project is complete.
...
Basic Access Group
General Update
Voyager Harvest
CLAMMS
Advanced Access Group
Voyager Update Utilities
Voyager Reporting Utilities
Batch Processing/Heads/Expert-Level Access Group
ALL LS Tools Production
Administer LS Tools Itself
...
I. “General Update” Section of LS Tools
...
D. ENACT Bulk Import (CGI15): Takes a MARC file that has been queued for bulk import and loads the file into Voyager.
E. Prepare a Vendor a Vendor e-Book, e-Journal, e-Audio, or e-Book Video Load: Loads electronic record sets supplied by vendor. Modifies the records per script prior to the load.Anchor VEBK VEBK
a. Alexander Street Press (Not in LS Tools): Vendor record load, run from a .pl script in the input/vendor records/Alex Street folder. Collections include e-books, e-videos, e-audio. Files are downloaded from the vendor website, loaded quarterly.
b.Bach (Not in LS Tools): Vendor record load, run from a "mod job" in staff member's LS Tools folder.c. BNA e-Journals (Not in LS Tools): BNABloom e-Journals (New (VEL16) and Revised (VEL17)): Formerly BNA e-Journals. Vendor record load, run from a "mod job" in staff member's LS Tools folderby Law Tech Services staff. Both new and updated records are downloaded from the vendor's website.
d. Books 24x7 (Not in LS Tools): Vendor record load, run from a "mod job" in staff member's LS Tools folder. Records are retrieved via FTP.
e. c. DDA Proquest Ebooks (VEL09): Loads ebook records from vendor. Records are retrieved via FTP.
fd. Early Arabic Books (Not in LS Tools): Vendor load of ebooks. Files are received via email.
ge. ElgarOnline (VEL10): Vendor load of law e-book records for ElgarOnline records. Files are received via email.
hf. e-Thesis (Not in LS Tools): This manual load takes metadata from both the grad school offices and from e-Commons to create .mrc records for the electronic versions of theses written by Cornell students. Records are delivered via email from the grad school office.
ig. HeinOnline Legal Classics Library AND HeinOnline World Trials Collection (VEL04): Takes a file of Hein package records, compares those in Voyager, retaining all links to other Hein packages while adding appropriate 899 codes.
jh. Henry Stewart Talks (Not in LS Tools): Vendor record load, run from a .pl script in the Batch_Processing/Scripts folder. Email is sent to staff to announce availability of new records, then files are downloaded from the vendor website.
ki. Lexis Record Load (VEL08): Prepares a file of Lexis e-resources. Job is similar to the HeinOnline Load.
lj. Lexis-Nexis Law Overdrive (VEL05): Vendor load of e-book records. Records are sent via email to Law Technical Services, where this script is run and records are loaded.
mk. LLMC Digital (VEL07): Vendor load of e-book records. Law Technical Services Librarian accesses vendor website and harvests spreadsheet.
nl. Memso e-Books (Not in LS Tools): Vendor load of e-book records. Records are received quarterly via email.
om. National Film Board of Canada (Not in LS Tools): Vendor load of streaming video records, run manually by Batch Processing staff.
pn. Naxos (Not in LS Tools): Vendor load of e-book records. Files are downloaded from the vendor website.
qo. NBER (Not in LS Tools): Vendor load of e-book records. Records are imported from the NBER website. Run as needed.
rp. NPAC-LC Jakarta (Not in LS Tools): Vendor load of e-book records, uses .pl script to extract LCCNs from Jakarta and batch search Voyager for missing records, edit and load into Voyager.s.
q OverDrive E-Books and Audio Books (Not in LS Tools): Vendor load of both e-book and audiobook records, uses .pl script in the Batch_Processing/Scripts folder. Email is sent to staff to announce availability of new records, then files are downloaded from the vendor website.
tr. PLI Discover Plus (VEL14 and VEL15)Brev VEL14 and Treatise VEL15 ): Files are received via email.
su. RAND E-books (Not in LS Tools): Vendor load of e-book records for RAND.
v. Westlaw Record Load (VEL07): Prepares a file of Westlaw e-resources. Job is similar to the HeinOnline load.
t. Skillsoft E-books (BPro VEL12, ITPro VEL11, ENGPro VEL13): Vendor record load for Skillsoft Ebooks, formerly BKS24x7. Records are retrieved via FTP.
u. Skillsoft E-videos (ITProdesktop MSC07, LDC MSC08): Vendor record load for Skillsoft Evideos, formerly BKS24x7. Records are retrieved via FTP.
F. Prepare a Vendor Print Load: Acquisitions bulk import of vendor supplied MARC records with or without purchase order creation via embedded order data. Selection of this button in LS Tools gives a choice of various pre-scripted jobs connecting to a folder on the vendor’s website or locally on a server.Anchor VPRT VPRT
a. Andinos (VPR22): Vendor Print load for Andinos/Booksur. Staff member retrieves and places .mrc files in vendor record folder.
b. AuxAmateurs Approvals AuxAmateurs Approvals (Not in LS Tools): Choose from AuxAmateurs FineArts Print (VPR10), AuxAmateurs ILR Print (VPR11), or AuxAmateurs Olin Print (VPR12) or AuxAmateurs Olin2 Print (VPR19). Loads print records from vendor AuxAmateurs that were retrieved via FTP.were retrieved via FTP.
c. Bach (VPR33): Vendor print record load.
db.Berenguer Approvals(VPR01): Similar to Yankee as it connects to our libfiles server, but it this case loads a vendor supplied preliminary record. Files are received via email attachment.
e. Cambeiro Print (VPR23): Vendor print load. Staff member retrieves and places .mrc files in vendor record folder.
fc. Casalini Approvals (VPR02): Loads print records from Casalini, which were are retrieved via FTP.
dg. Chinese Approvals (Not in LS Tools): This is a two-step task. We receive a file of MARC records for Chinese blanket order books that is loaded as "On Order" while the books are in shipment. We then reload the file on arrival, creating a second holdings record. We batch delete the original holdings and do editing as appropriate for Oversize, accompanying material, and multi-volume works.
eh. Coutts PDP (VPR08): Loads patron directed print records from Coutts, which were retrieved via FTP.
fi. Coutts Approval (VPR09): Extracts ISBNs from Yankee file, searches OCLC, prepares file of best available Bibs, bulk imports them into Voyager.
g. j Derex Print (VPR03):
hk. Eastview Print (VPR13): Loads acquisitions-level print records from vendor.
il.Euyloo(Not in LS Tools): In this manual job, the staff member takes a tab-delimited file of data from the vendor, converts it into MARC records using MarcEdit, and loads it into Voyager.
m. Harrasowitz Music Records (VPR05)
nj. Harrassowitz Full Print (VPR06): Selecting this will load full level Harrassowitz MARC records and create purchase orders. Connects directly to the Harrassowitz vendor’s server and pulls in the file (via FTP) for processing.
ko. Harrassowitz Regular Print (VPR04): Same as Harrassowitz Full Print, but does not have full level records.
lp.Icelandic Firm Orders(Not in LS Tools): The selector for the Icelandic collection sends a .mrc file of acquisition level records to the cts-bulk email list; these records are then edited by .pl script and loaded into Voyager.
q. Itturiaga Print (VPR21):
rm.Korean Print(Not in LS Tools): Vendor-supplied records processed manually by inputting staff member.
ns. Kozmenko Print (VPR13)
ot. Leila (VPR18): Vendor-supplied records are edited using an LS Tools modification job kept in a staff member's LS Tools folder.
pu. LC Cairo Arabic Records (VPR31): Vendor load of print records. Files are delivered via email; statistics and local fields are added and records are loaded into Voyager.
qv. Lexicon (VPR16): Vendor print load of preliminary records. Files are received via email.
rw. MIPP Print (VPR14): Loads preliminary-level vendor records.
sx. PanMun (Not in LS Tools): In this manual job, the staff member takes a tab-delimited file of data from the vendor, converts it into MARC records using MarcEdit, and loads it into Voyager.
y. Retta Print (VPR07):
zt. Serbica (VPR17): Vendor print load of preliminary records. Files are received via email.
uaa. SofiaBooks Print (VPR15): Loads preliminary print ordering records from the vendor. Records are received via email attachment.
vbb. Vietnamese Print(Not in LS Tools): In this manual job, the staff member takes a tab-delimited file of data from the vendor, converts it into MARC records using MarcEdit, and loads it into Voyager.
II. “Voyager Harvest” Section of LS ToolsAnchor #VHarvest #VHarvest
...
D. Extract MARC Records from Voyager (CGI13): Extracts MARC Records from Voyager using a list of BIB or MFHD id's.
Anchor
III. “CLAMSS “CLAMMS subsystem” (CGI28): Also known as the Automated Media Shelflisting System, this tool allows catalogers to determine the proper accession number for videos and microforms. Access is given only to those individual catalogers who need to use it. CLAMMS CLAMMS
Anchor ADV ADV
Advanced Access Group
| ADV | |
| ADV |
I. “Voyager Update Utilities” Section of LS ToolsAnchor VUU VUU
...
c. BatchMatch*: Takes less-than-full bibs and sends to OCLC for automated matching with full records, retrieves, bulk imports them into Voyager. First part of process: In LS Tools under "Reports and Extracts/OCLC Batch Records Export." Creates extract that is to be sent to OCLC. Then the BatchMatch script loads the resulting records. *OCLC Batchmatch Monthly by Date Ranges is a cron (CRN38); OCLC Batchmatch by Bib Id Range can be run at any time via Privileged Procedure (PR338).
d. Coutts Shelf Ready (MSC05): Processes and loads records for shelf-ready books, reports out records requiring manual intervention for oversize, multivol, added pieces, etc.
e. General OCLC MARC Selection from a List of ISBNS (MSC09): Takes ISBNs, runs them against the OCLC rule matrix and finds the best available matching record.
f. Marcive (MSC02): Loads Federal Government e-resource records. Creates files of new and existing Marcive records for loading.
g. PDC Building Plans at the Annex (Not in LS Tools): Each month Cornell's Planning, Design and Construction Department sends us an email with a spreadsheet of barcodes for design plans that are headed to the Library Annex. We use a script to add item records to suppressed records for each barcode with a roll number in the enumeration. Those plans are then sent to the Annex for processing.
h. Skillsoft Books 24x7 Videos (MSC08): Load of vendor records for e-videos. Records for this job are retrieved via the vendor's FTP site.
to the Annex for processing.
B. Execute a Privileged Production Procedure: Location for more advanced reports and record loads with higher stakes if mistakes happen.Anchor PRIVPROD PRIVPROD
a. Add Donor Names to 902$a (PR326): Checks fund code in Bib against fund code table, adds name of donor(s) to 902 field.
b. Add Donor Plate Links (Not in LS Tools): Monthly extract of approved invoice line items paid for on endowed gift funds. The fund is linked to a specific donor book plate which has been placed on the web. A link to that plate is placed in the holdings as an 856 field. This job is run from a .pl script.
c. Bib Flip 490 to 830 Records (PR321): Adds 830 series field to records with only a 490 field for search ability in Voyager.
d. Create 856 link for Table of Contents Images (PR707): Scans serial volume tables of contents, uploads .pdf's tagged with the bib id. Adds an 856 for the URL and reloads.
e. Enhance Level 3 Encoding Records(PR324): Finds lvl3 records in Voyager, uses ISBN to search OCLC for a better record, merges found record with Voyager record to improve encoding level.
f. Enhance Voyager Records with Vernacular Characters (PR333): Using the OCLC Search API and the process described in the specification Component: OCLC Record Retrieval we will identify the best possible match to our Voyager record and copy vernacular name and title fields to our record and load back into voyager.
g.Fast Headings Update Via OCLC Spreadsheet (PR714): This script looks for Voyager records that lack fast subject headings and creates a .mrc file for export to OCLC.
h. General Bib field Cleanup (CRN27): This script searches for invalid fields, subfields, and indicators in BIB records, and reports out the errors for manual cleanup.
i. Icelandic BIB Cleanup (Not in LS Tools): This job adds a 710 "Fiske Icelandic Collection" to Icelandic records, and is run from a "modify" job in the LS Tools folder of a staff member.
j. Initial Article and Title Tag Cleanup (PR327): Looks at previous month’s cataloging for errors in indicators relating to articles in title fields.
k. Order Record Enhancements from OCLC (CRN35): Running every Sunday, this job looks at preliminary level records with a date of the past thirty days, searches for a better record in OCLC, and merges it with the Voyager record if found.
l. Serial Solutions e-Books Combined (PR354): Prepares Serial Solutions e-book files for bulk import, prepares Serial Solutions e-book deletion file for bulk import and batch deletion.
m. Serial Solutions Journals Combined (PR344): Prepares Serial Solutions e-journal file for bulk import and a separate file for batch deletion.
n. Update Backlog Books (PR322): Finds “in process” records for desired storage code in Voyager, searches OCLC by ISBN for a better record, merges with Voyager record, and queues for loading into Voyager.
...
D. Run a Voyager Utility Program (VOY01, VOY02, VOY03): Used after loading authority record files to perform global headings jobs 11 and 12, which index bib and authority headings in global headings queue for authority maintenance.
II. “Voyager Reporting Utilities” Section of LS ToolsAnchor VRep VRep
...
a. MFHD Copy Number Clean-up (PR101): Compare the copy number in the MFHD 852 subfield t with the copy number in the item record. Output contains some routine fixes which are extracted for manual correction.
b. General Withdrawals Report (PR102): Generates a count by library of items withdrawn over a given time period, typically the prior month.
...
b. OCLC Unresolved Records (Not in LS Tools): Receive email announcements of our daily cataloging export results. Any coded as unresolved appear in a MARC file which is downloaded from OCLC Product Services website. Our holdings for these records were not added to WorldCat due to MARC errors. Staff member must manually examine each record in Connexion to correct error(s) and add our holdings to OCLC.
Anchor EXPERT
| EXPERT |
...
EXPERT
Batch Processing/Heads/Expert-Level Access
| EXPERT |
...
Group
I. “ALL LS Tools Production” Section of LS ToolsAnchor AllProd AllProd
...
B. Cron Jobs for Interaction with OCLC
a. OCLC System Number Mapping (CRN05): Better known as OCLC mapping job, as it adds the OCLC number to new cataloging from OCLC cross reference reports.
b. Automated Monthly Export of NoEx No-Ex Records(CRN07): This script extracts proprietary records for matching in OCLC, without adding the vendor's .mrc record to WorldCat. If it finds a matching record, it adds the OCLC number to the Voyager record;
c. Records with no OCLC numbers (CRN08): Runs harvest of cataloged records from the previous month that are lacking OCLC numbers. Resubmits monograph records to OCLC; prints out list of serials lacking OCLC numbers, to be reviewed by staff.
d. WorldShare Local Holdings Export to OCLC (CRN10): Choose "3 day old non-serial mfhd export to OCLC" to send local holdings data to OCLC for non-serial records. Choose "3 day old serial mfhd export to OCLC" to send local holdings data to OCLC for serial records.
e. OCLC Manuscript-Archive Export (CRN12): Weekly export to OCLC of updated manuscript and archive bibs.
f. OCLC Daily Delete (CRN15): Daily export to OCLC of holdings to be deleted.
g. OCLC Daily Export (CRN16): Daily export to OCLC.
h. OCLC BatchMatch for Date Ranges (CRN38): Searches for less-than-full bibs within a given date range and sends to OCLC for automated matching with full records, retrieves, bulk imports them into Voyager. First part of process: In LS Tools under "Reports and Extracts/OCLC Batch Records Export." Creates extract that is to be sent to OCLC. Then the BatchMatch script loads the resulting records. Note: OCLC Batchmatch by Bib Id Range can be run at any time via Privileged Procedure (PR338).
i. OCLC WorldShare Update (PR335): Takes WorldShare records from OCLC ftp, searches Voyager for matching records, merges where incoming record is fuller than existing.
C. Execute a Google Utility : Various scripts for Google and HathiTrust Voyager extracts. Some are run by LTS staff; some by the project manager for the Google Project and who also serves as liaison to HathiTrust.Anchor GGL GGL
...
h. Update oai.library.cornell.edu/ (GGL24)
D. Cron Jobs with General AccessAnchor CRNGEN CRNGEN
a. Report of bibs with duplicate ISBNs (CRN03): Reports out bibs with duplicate ISBNs based on the previous days 005 date.
b. Bib call number maintenance (CRN18)
c. Serial Metadata List for RAPID (CRN28): Generates tab delimited list of serials or monographs for export to RAPID for Inter-Library Loan use/
...
e. Wrapper for CRN29 Rapid Monographs (CRN99): Generates tab delimited list of serials or monographs for export to RAPID for Inter-Library Loan use.
f. Annex boundwith items (PR103): Identifies bound-withs so that they can be flipped and empty item records can be created.
g. Combined report on Bib/Mfhd suppression (PR104): Looks in Voyager for bibs without at least one active (unsuppressed) MFHD.
h. Histogram of 899 with NoEx needing 035 OCoLC (PR105): Identifies all codes used in 899 $a for cleanup or comparison against the database of “approved” codes in use.
i. Eliminate redundant OCLC 035 fields (PR323):Reviews previous week’s records in Voyager, finds malformed or duplicate OCLC 035 fields, removes duplicates and malformed instances, reports out BIBs that require further examination for manual cleanup.
j. Daily Annex Flip and Location Report (CRN21): Identifies bound-with's so they can be flipped, and empty item records are created. This cron job is part of Voyager Utilities and runs every weekday.
E. Cron Jobs with Privileged AccessAnchor CRNPRIV CRNPRIV
...
c. Send e-mails to book requesters and purge 981, 980$s (CRN20): Harvest requestors’ email from day’s purchase order line items and sends emails to patron informing them that the title has been ordered for them. Removes requestor information from bib found in 981 subfield n.
d. General MFHD field field cleanup (CRN24)
e. Cleanup 008 Single Date (CRN25) and Cleanup 008 Multi Date (CRN26): Compares date in 260 or 264 to contents of 008/07-14, corrects content to make fields "congruent.”
f. General BIB field cleanup (CRN27): This script searches for invalid fields, subfields, and indicators in BIB records, and reports out the errors for manual cleanup.
g. Daily Tech Services Statistics (CRN33): This CRON job runs daily, and gathers the 948 data from the previous day's records. Each day we receive a report of incorrectly coded 948 statistics fields. We manually correct those, so that they can then be added to our next statistics extract.
h. Monthly Tech Services Statistics (CRN34): Gathers statistical information for technical services statistics database.
i. Order record enhancements from OCLC (CRN35): Running every Sunday, this job looks at preliminary level records with a date of the past thirty days, searches for a better record in OCLC, and merges it with the Voyager record if found.
j. E-Book updates from OCLC (CRN37): Three scripts* for harvesting e-resource bib records lacking call numbers. One searches OCLC using the 035, the second searches OCLC by 776, and the third searches by ISBN. If an OCLC record with call number is found, it is merged with Voyager record and queued for loading into Voyager. * Ebook Updates from OCLC Based on 035, Ebook Updates from OCLC Based on 776, Ebook Updates from OCLC Based on ISBN.
k. Order records from Coutts via Oasis with fund and location separate (CRN41): Takes WorldCat records chosen by selectors for ordering, loads bibs and creates POs in Voyager.
l. Vendor order pickup--Harrassowitz (CRN42)
m. POOF records for Cornell fund and location together (CRN43): Loads Cornell's POOF records.
n. POOF records from Columbia (CRN44): Prepares POOF order file for Columbia.
o. Bib flip 490 to 830 records (PR321): Adds 830 series field to records with only a 490 field for search ability in Voyager.
p. OCLC Master Number Maintenance Project (PR336): Checks master number against OCLC and adds fast headings to Voyager record, if found on master but missing in Voyager.
q. Daily Record Extraction to OASIS (PR705): Extracts added or changed bib records to Coutts for loading into Oasis.
r. Create 856 link for Table of Contents Images (PR707): Scans serial volume tables of contents, uploads .pdf's tagged with the bib id.
F. Execute an Administrative Production ProcedureAnchor ADPROD ADPROD
a. Rank OCLC records returned for an isbn, issn, etc. (CGI01)
b. TS STATS Summary (CGI24): The monthly CRON pushes a package of monthly 948 data to “Disc”; in this job, a staff member pushes that same package to the “DB” (database) used by supervisors for creating individual and department statistics reports.
...
f. Cornell ITSO Vendor Matrix Replacement (PR713): Updates tables for automated ordering for vendors.
g. Fast Headings updates via spreadsheet from OCLC (PR714): This script looks for Voyager records that lack fast subject headings and creates a .mrc file for export to OCLC.
...
i. Replace Harvard POOF Vendor Matrix (Not in LS Tools): Infrequent manual job - Updates tables for automated ordering for vendors.
...
Anchor ADMIN ADMIN
Administrative Access Group
| ADMIN | |
| ADMIN |
III. “Administer LS Tools Itself” Section of LS ToolsAnchor AdLSTools AdLSTools
...
f. Configure JOB CONTROL (Families) (CGI38)
...