ASA Electronics SRS1000 User Manual

SRS PRISMA GUIDEVersion 4.1.1

VIII5.2.2 Command Line Arguments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1495.2.3 Methods for Producing Reports . . . .

Configuring SRS Prisma86Figure 3.32 The Index Settings tab.3.9.1 Indexing SettingsThe indexing phase of the Prisma update can also be configured. This

LION Bioscience 873.9.2 Non-SRS LibrariesNot all libraries updated by SRS Prisma need to have valid, indexable data fromwhich SRS indices are produced

Configuring SRS Prisma88Figure 3.33 Adding pre-indexing commands.Alternatively, the commands may be set directly in the Resource object using thepreIn

LION Bioscience 89outline how to configure SRS Prisma to carry out these commands. The Reformatsettings tab can be used to configure post-processing (

Configuring SRS Prisma90To add generation a FASTA/BLAST file to a library, open the library for editing usingVisAd, open the Reformat settings tab, an

LION Bioscience 91Figure 3.35 A query-based FASTA file.The following values can be set using the dialog (BlastFile attributes in brackets):Database na

Configuring SRS Prisma92SRS query string (queryStr) This should contain an SRS query to return entries containing the desired sequences.Sequence forma

LION Bioscience 93The following dialog shows how to configure a BLAST file to be generated fromsequence data files:Figure 3.36 Configuring BLAST indic

Configuring SRS Prisma94Generation method (type)This should be set to ‘file’ to indicate per-file conversion.FASTA converter (fastaConverter) This mus

LION Bioscience 953.10.1.3 Use of Existing FASTA FilesFinally, if the files used by an SRS library are already in FASTA format, it is possibleto produ

LION Bioscience IX6.3 Isolating Errors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1916.4 Manuall

Configuring SRS Prisma96Database name (dbName)This is the title of the database generated, and the name of the file created by merging partial FASTA f

LION Bioscience 97Figure 3.38 Generating BLAST filesThe following values can be set using the dialog (BlastFile attributes in brackets):Generate BLAST

Configuring SRS Prisma98Note: SRS Prisma does not include the utilities needed to generate BLAST indicesfrom FASTA files (e.g. formatdb). These should

LION Bioscience 99library, after any SRS indices have been built. It is the responsibility of the commanditself to manage any extra datafiles it creat

Configuring SRS Prisma100%nThis is used to specify the filename minus any extension (i.e., all characters up to but excluding the last dot in the comp

LION Bioscience 1013.10.2.2 Postprocessing and Compressed FilesWhen using SRS compression (see Data Compression, page 223 of the SRSAdvanced Administr

Configuring SRS Prisma102Do not run commands on compressed files (skip) Per-file commands are only executed on uncompressed filesRun commands on all f

LION Bioscience 103These can be set in the Resource object directly by setting the installationTypeattribute to the appropriate option (shown in brack

Configuring SRS Prisma104•%e – replace with extension of each file.File patterns can be specified directly using the installFiles attribute of the Res

LION Bioscience 105Figure 3.43 Specifying installation commands.These commands do not allow filename substitution, although %d, %N, %R and %U areavail

X7.9.4 Monthly Errors Breakdown . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2107.9.5 Data-Fields Index Status . . . . . .

Configuring SRS Prisma1063.11.2 ArchivingBy default, SRS Prisma will delete old flat-file and index sets when it installs new dataonline. However, it

LION Bioscience 1073.11.2.3 Archive by Moving to Another Location (move)If set, any changed or deleted data files are first moved into the specified d

Configuring SRS Prisma108Figure 3.44 Specifying pre- and post-installation commands.Note that the %d placeholder may be used in each case to be replac

LION Bioscience 1093.11.4 Installation ChecksBefore installation of successfully updated libraries takes place, two different sets ofchecks take plac

Configuring SRS Prisma110Figure 3.45 Adding an installation check.Checks can be added manually as PrismaInstallCheck objects to theinstallChecks list:

LION Bioscience 111Figure 3.46 A time-based installation check.The equivalent PrismaInstallCheck object is:$PrismaInstallCheck:[ checkType:time inst

Configuring SRS Prisma112$PrismaInstallCheck:[checkType:command installCheckCommand:”/usr/local/bin/checkDatabase –dbName uniprot_swissprot”]3.11.4.

LION Bioscience 1133.12 Schedule Settings3.12.1 Scheduling for Automatic UpdatesSRS Prisma is designed to be run automatically on a daily basis, and a

Configuring SRS Prisma114Figure 3.50 Editing a schedule.Schedules can also be added manually as PrismaSchedule objects to the Resourceschedule attribu

LION Bioscience 1153.12.2 Schedule FrequencySchedules can be set to update libraries on different time-scales. The appropriateoption can be set using

LION Bioscience XI9.2.3 Reformatting Flat-Files . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2499.3 Handling Multiple

Configuring SRS Prisma116never updateWhen set, the library will always be checked and updated if necessary. An example PrismaSchedule object would be:

LION Bioscience 117Schedule checks on dependencies on other librariesIf this option is set, the library will only be checked against 'parent&apos

Configuring SRS Prisma118Note: When considering data sets for which a separate release set and update setexist (e.g. GENBANKRELEASE and GENBANKNEW), c

LION Bioscience 119changed as required. In addition, specific stages can be changed so they do not usebatch queuing (by unchecking the stage in questi

Configuring SRS Prisma120 Figure 3.53 Setting a library as dependent.To specify a library as dependent directly using the Resource object, set theupdM

LION Bioscience 1213.14.2 Configuring a Library as Dependent on Online DataDependent libraries are normally updated after their parents are updated, b

Configuring SRS Prisma122Figure 3.54 Adding a preprocessing command.or the unpackCommand attribute of the Resource object (see Section 3.8.1.13,Specif

LION Bioscience 123In addition, a failover command may be specified that can be run if one or more pre-processing command fails. This allows processes

Configuring SRS Prisma1243.14.6 Limitations of Dependent Libraries3.14.7 Building Dependent Libraries ManuallyA dependent library will only be rebuilt

LION Bioscience 125sizes of datafiles are not known when SRS Prisma writes the commands needed forindexing. However, SRS Prisma can be provided with m

XII

Configuring SRS Prisma126Figure 3.55 Using Library.files to get file information.To set this directly, set the Resource usesFiles attribute to ‘yes’:$

LION Bioscience 127filename and size information, available to the parent, to create a file list for thedependent database. When this mechanism is use

Configuring SRS Prisma128 unpackCommand:"find /path/to/embl -name "*.dat" -exec createFasta.sh {} \;" inheritFiles:yes inheritF

LION Bioscience 129Figure 3.57 Adding an external trigger.Figure 3.58 Editing an external trigger.The equivalent Resource object has updMethod set to

Configuring SRS Prisma130 updateTrigger:{ $PrismaUpdateTrigger:[checkType:time updateWindow:”4:08:00-4:21:00” ] }]The trigger can be defi

LION Bioscience 131Remote data and index files can be specified as for 'normal' libraries, using theUpdate settings tab:Figure 3.59 Specifyi

Configuring SRS Prisma132

C HAPTER4RUNNING SRS PRISMA

Running SRS Prisma1344.1 IntroductionOnce SRS Prisma is installed, it can be quickly and easily run to check and updateany installed SRS databases. Th

LION Bioscience 135Prisma configurations such as 'switchlinking' and archiving. To carry out specific localupdating, it is recommended that

C HAPTER1INTRODUCING SRS PRISMA

Running SRS Prisma136-Gstring“”Excludes libraries from the named group from the range of checking. (see Section Section 3.4.13, Groups to Exclude from

LION Bioscience 137Table 4.2 Command line option flags controlling execution of the run-downloadOnlyBooleanFALSERestricts checking and updating to rem

Running SRS Prisma138Table 4.3 Command line option flags controlling reportingTable 4.4 Command line option flags controlling advanced behaviorFlag Ty

LION Bioscience 1394.3 Running SRS Prisma AutomaticallyTypically, SRS Prisma is run as an automatic process on a regular, sometimes daily,basis. To fa

Running SRS Prisma1404.4 During a SRS Prisma RunWhen an SRS Prisma run is in progress, a large number of files are needed tomonitor the progress of th

LION Bioscience 141cialized Prisma configurations such as 'switchlinking' and archiving. To carry outspecific local updating, it is recommen

Running SRS Prisma142Important: When using restarting SRS Prisma, it is critical that all processes fromprevious failed runs are killed. Generally, ki

LION Bioscience 143To move libraries online manually, the movePrisma script has been provided. This willfind libraries that have been blocked from bei

Running SRS Prisma144Note that while the main installation checks will not be carried out (see SectionSection 3.5.1, Pre-installation Checks, page 45)

LION Bioscience 1454.6.2, Run State Preservation, page 145). Reporting will be carried out using the dateof the first run, combining information from

Introducing SRS Prisma2Note: If you are an experienced SRS Prisma user, you may skip to Section 1.4,What’s New in SRS Prisma 4.1, page 12.1.1 What is

Running SRS Prisma146

C HAPTER5SRS PRISMA UPDATE REPORTS

SRS Prisma Update Reports1485.1 IntroductionSRS Prisma carries out a complex array of interdependent tasks to update an SRSinstallation; and, although

LION Bioscience 1495.2.2 Command Line ArgumentsThese three types of reports can be generated using the SRS Prisma scriptreportPrisma. Command line arg

SRS Prisma Update Reports1505.2.3 Methods for Producing ReportsProduction of full reports can be time consuming (as much as 30 minutes for a verylarge

LION Bioscience 151This page is available via the View SRS Prisma Color Key at the top of the mainSRS Prisma report page.5.3.2 Getting HelpEach SRS Pr

SRS Prisma Update Reports152Figure 5.3 Successful Installation Message PageOnce the SRS Prisma update process has been run successfully, a page simila

LION Bioscience 153Figure 5.4 Calendar Page5.3.4 Update Report PageTo view Update Reports, on the Calendar Page (Figure 5.4) select the option buttoni

SRS Prisma Update Reports154If the SRS Prisma run is still in progress and no report has been produced, the pageshown in Figure 5.5 will appear:Figure

LION Bioscience 155Figure 5.7 SRS Prisma Update ReportThis report shows the outcome of a successfully completed update of two databases,and the link b

LION Bioscience 3failures highlighted, so they can be rapidly and easily rectified. In addition, SRSPrisma also performs a thorough quality analysis o

SRS Prisma Update Reports156The next major section is a list of short graphical summaries for each library updated.Each summary contains the name and

LION Bioscience 157been run to completion and the library has been installed online, and are coloredgreen to indicate success.Figure 5.8 A successful

SRS Prisma Update Reports158Figure 5.11 A successful link updateFigure 5.12 shows an example where a link that has failed to be built, and hence hasno

LION Bioscience 1595.3.5.2 The Local Copy ReportDuring the local copy stage, up-to-date datafiles are copied or linked from the onlinedata directory t

SRS Prisma Update Reports160Figure 5.14 Download ReportThe following information is available in the General Information section:Download MechanismIf

LION Bioscience 161Remote FilenameName of file on remote server.Local FilenameName of file locally after download and pretranslation.Host Name of loca

SRS Prisma Update Reports162Figure 5.15 Pretranslation ReportThe following information is available in the General Information section:Number of comma

LION Bioscience 163build commands for each file are 'chained' together so that these stages can run con-currently, pretranslating and indexi

SRS Prisma Update Reports164Figure 5.16 Build Report PageThis stage is also applicable for virtual libraries like UNIPROT, for which the reportshown i

LION Bioscience 165Figure 5.17 Build Report Page for a Virtual LibraryThe General Information section provides the following types of information:Para

Introducing SRS Prisma4Figure 1.1 A hierarchy of libraries.Three libraries, UNIPROT_SWISS, UNIPROT_TREMBL and LOCAL_PROTEIN(hypothetical), can be upda

SRS Prisma Update Reports166Number of Failures Number of failed targets for stage.Total Elapsed Time Total time taken so far for all completed targets

LION Bioscience 1675.3.8 The Reformat ReportFigure 5.18 shows a typical Reformat Report page, with each refornat commandlisted.Figure 5.18 Typical Ref

SRS Prisma Update Reports168Figure 5.19 Typical Link Stage ReportEach link is shown with its name, which acts as a link to the detailed report for tha

LION Bioscience 169Figure 5.20 Typical index link reportFigure 5.21 shows the status of the read link between UNIPROT_SWISSPROT andPDB, which has comp

SRS Prisma Update Reports170Figure 5.21 Typical read link reportNote: The build targets for this link may include building other read links fromUNIPR

LION Bioscience 171To Field Field in to library to use for link.Parallel Type Read links only, parallel type of from library.Part size Read links only

SRS Prisma Update Reports172the stage (e.g. 1.UNIPROT_SWISSPROT_readindex_2). The same is also truewhere a read link to or from a ‘blocked’ library ha

LION Bioscience 173Install TypeType of installation to use, such as move, switchlink, command, none (Resource.installType).Install FilesIf specified,

SRS Prisma Update Reports174Figure 5.23 Typical Target ReportThe following general information is available for all targets:SRS Prisma RunRun name, su

LION Bioscience 175In addition, for most targets the following detailed information is available:Execution hostLocal host on which process was execute

LION Bioscience 51.2.3 Anatomy of a SRS Prisma RunThe following diagram illustrates how SRS Prisma implements staging:Figure 1.2 The staging update pr

SRS Prisma Update Reports1765.3.13 The Dependency TreeIn addition to the text-based HTML reports generated by SRS Prisma, a dependencytree diagram can

LION Bioscience 177Figure 5.25 Dependency TreeThe report is divided into two sections. The image on the left is a dependency tree forthe selected stag

SRS Prisma Update Reports1785.3.14 The Decision ReportAlthough SRS Prisma is designed to be run as an automatic process, and to take allsteps needed t

LION Bioscience 179Figure 5.27 Decision Report PageThe Decision Report is divided into sections, for each set of libraries to be updatedsuch as librar

SRS Prisma Update Reports180The check phase output can be viewed by clicking the appropriate View checkphase output link for each library level. The c

LION Bioscience 181Figure 5.29 Update Phase OutputIn addition, the View update phase output link also shows a similar page for thestandard error from

SRS Prisma Update Reports182The reasons given may also include why a library was not updated. This includesvirtual libraries for which the parent libr

LION Bioscience 183Figure 5.31 Decision Report showing a blocked libraryFor debugging purposes, more information is provided in the form of the TraceR

SRS Prisma Update Reports184This indicates the date and size of the data and index files for the online and offlinedirectories for the library, and ma

LION Bioscience 185Figure 5.33• Total time - the total CPU time expended by individual update processes (this does not include additional overhead inc

Copyright (c) 2005 LION bioscience AG (LION). All rights reserved.LION bioscience SRS 8.1.1 DocumentationThis manual, as well as the software describe

Introducing SRS Prisma6libraries (e.g. are updated from a remote FTP site like UNIPROT_SWISS). Level 2libraries depend on level 1 libraries (e.g. virt

SRS Prisma Update Reports186In addition to this overview, each. To obtain a report on an individual library, select thename from the menu at the top o

LION Bioscience 187• Total size - the total size in kb of the data files and indices associated with this library after the update (this does not incl

SRS Prisma Update Reports188button to view the quality report for the library in question (see Chapter 7, SRSPrisma Quality Report).

C HAPTER6SRS PRISMA - TROUBLESHOOTING

SRS Prisma - Troubleshooting1906.1 IntroductionAlthough SRS Prisma is normally straight-forward to use and configure, complexconfigurations can occasi

LION Bioscience 1916.3 Isolating ErrorsIf it is not immediately clear why a particular error is happening, then the next step isto try to isolate the

SRS Prisma - Troubleshooting192In addition to the reformat and unpack phases, problems frequently occur during themove phase. In this case, the <DB

C HAPTER7SRS PRISMA QUALITY REPORT

SRS Prisma Quality Report1947.1 What is SRS Prisma Quality Report? SRS Prisma Quality Report is a tool for administrators which they can use to checka

LION Bioscience 1957.3 Running SRS Prisma Quality ReportSRS Prisma Quality Report can be run in the following ways:• As part of an SRS Prisma run.• La

LION Bioscience 7• DownloadNew datafiles are downloaded from a remote server• PretranslateNew datafiles are preprocessed after download (for example,

SRS Prisma Quality Report1967.3.3 Running SRS Prisma Quality Report from the Commandline SRS Prisma Quality Report is executed by running the srsinspe

LION Bioscience 197-g Groups String ‘’ To run tests on the databanks listed within the group. You can use either the name of a group or the short name

SRS Prisma Quality Report1987.4 Error Checking Activities for SRS RelationalMost of the test activities in SRS Prisma Quality Report were written for

LION Bioscience 199specific to the RDB configuration appear separately in its own class that is color-coded in the graphical HTML output.7.5 Error Che

SRS Prisma Quality Report200A Tools Test Results page displays the breakdown summary of errors found in tools;and each tool has an individual Error D

LION Bioscience 2017.7 Quality Report HTML Pages Figure 7.1 Calendar PageOnce SRS Prisma has completed its run, users can access the SRS Prima Quality

SRS Prisma Quality Report2027.7.1 Main PageThe main page gives you the current status of each databank after the latest round oferror-checking activit

LION Bioscience 203Figure 7.2 Summary Report

SRS Prisma Quality Report204Figure 7.3 Breakdown of Database Names according to their Error Status Change7.7.3 Databank Group Reports Following the su

LION Bioscience 205• Left-Hand Chart A breakdown of errors and associated error severity score of each databank within the databank group.• Right-Hand

Introducing SRS Prisma8Checking, page 109. Manually moving ‘blocked’ libraries online is described inSection 4.5, Moving Libraries Manually, page 142.

SRS Prisma Quality Report2067.7.4 Right-Hand ChartThe chart of the right is the breakdown of errors and associated severity score of adatabank within

LION Bioscience 207Users can see the specific set-up errors of a tool by directly clicking on the relevantpart of the color-coded error severity-bar o

SRS Prisma Quality Report208• Quick References • Full Error Details Classified by Type Reports • Monthly Errors Breakdown • Data-Field Index Status7.9

LION Bioscience 2097.9.2 Quick References ToolboxThis is a toolbox with links to display the various files, which help define theintegration of this d

SRS Prisma Quality Report210Figure 7.9 Full Error Details Classified by Type Reports7.9.4 Monthly Errors Breakdown Near the bottom of the Error Detail

LION Bioscience 211Figure 7.10 Monthly Error Breakdown7.9.5 Data-Fields Index Status A table of the status of each indexed data-field is shown at the

SRS Prisma Quality Report212Figure 7.11 Data-Fields Index Status7.10 Archiving Upon the completion of Prisma update processes and subsequent error-che

LION Bioscience 213Table 7.2 Test Suite LINKSTest Name DescriptioncheckLinkSetup Checks for inappropriately defined links (links that are a mix of a r

SRS Prisma Quality Report214Table 7.6 Test Suite PARALLELTest Name DescriptionFileParallelisa-tionChecks that indexing for databanks with only one dat

LION Bioscience 215Table 7.10 Test Suite VIRTUALTest Name DescriptionVirtualMember-sCheckChecks for members of virtual databanks. Make sure that membe

LION Bioscience 91.3 What’s New for SRS Prisma 41.3.1 Prisma ConfigurationGlobal configuration is now carried out using a new Icarus class, PrismaSett

SRS Prisma Quality Report216Table 7.13 Test Suite NAMECLASHTest Name DescriptionLibGroupNameClash Check that the name of a library group is not identi

C HAPTER8CONFIGURATION UPDATES WITH SRS PRISMA

Configuration Updates with SRS Prisma2188.1 IntroductionSRS is a highly customizable and extensible data integration platform, and as such,relies on e

LION Bioscience 219:The configuration file update process is as follows (illustrated in Figure 8.1):check and download new filesThe remote update modu

Configuration Updates with SRS Prisma220• conflict - as for update, but the merge is not possible without manual interven-tion.resolve conflictsIf unr

LION Bioscience 221directories as appropriate, and srssection is run to compile new object files for use by the rest of SRS. Figure 8.1 The configurat

Configuration Updates with SRS Prisma222Important: Whatever mechanism is used, the configuration files being updated mustnot be changed by an external

LION Bioscience 223[2005/01/11 11:57] Comparisons of new and existing files complete[2005/01/11 11:57] Checking for conflicts with current configurati

Configuration Updates with SRS Prisma224% updateConfigFiles.sh -doMoveFurther to the flags discussed above, updateConfigFiles.sh accepts the following

LION Bioscience 225• switch to using offline configuration set• carry out normal SRS Prisma update process (forcing reindexing where required)• move n

Introducing SRS Prisma101.3.3 Support for Remote Updating of IndicesIn addition to downloading new data files from remote servers, SRS Prisma nowsuppo

Configuration Updates with SRS Prisma226% updateConfigFiles.sh -resolveThis command is interactive, and asks for the resolution for each conflict foun

LION Bioscience 227The following output shows resolution of a conflict involving the embl.is parser file,which has been compared on a textual basis:[C

Configuration Updates with SRS Prisma228Note: It is also possible to manually revert changes - the original directories arenamed <configName>_ba

LION Bioscience 2298.3 Update ReportsIn addition to the text output from the command line tool, SRS Prisma producesHTML reports when it updates config

Configuration Updates with SRS Prisma230Choosing to view a Configuration Report takes you to the main configuration reportpage. An example is shown in

LION Bioscience 231stages have completed successfully. In Figure 8.4, the comparison stage shows aconflict has been found:Figure 8.4 The main configur

Configuration Updates with SRS Prisma232The logs for each stage can be examined by clicking the hyperlink in the stagesummary. Figure 8.5 shows the lo

LION Bioscience 233In each case, the status can be clicked to view a detailed report page, as illustrated inFigure 8.6:Figure 8.6 File report for a fi

Configuration Updates with SRS Prisma234To edit the configuration for the update mechanism, start the Visual Administrationtool, open the SRS Prisma t

LION Bioscience 235 ]]In each case described below, the corresponding Icarus classes and attributes aredescribed.8.4.1 Basic configurationBy default,

LION Bioscience 111.3.8 New Scheduling MechanismSRS Prisma now has a completely new scheduling mechanism to increase the levelof control an administra

Configuration Updates with SRS Prisma236Figure 8.9. These can also be edited by adding, deleting or editing RemoteConfigobjects in the PrismaSettings.

LION Bioscience 237Distribution copy is not read by SRS (alwaysCopy)This option should normally be set, as it creates a parallel distribution director

Configuration Updates with SRS Prisma238Automatic conflict resolution (conflictResolutions)This is a list of conflict resolution rules (see Section 8.

LION Bioscience 2398.4.3 Editing remote mirror locationsEach remote configuration must have at least one remote location specified. This is alist of r

Configuration Updates with SRS Prisma2408.4.4 Editing remote hostsA remote host contains details of the location of a remote directory and how toconne

LION Bioscience 2418.4.5 Editing remote filesA remote file contains details of the remote files of interest on the remote site andhow they correspond

Configuration Updates with SRS Prisma242appropriate. Alternatively, the decisions can be edited directly asConflictResolution objects in the RemoteCon

LION Bioscience 243.log, .STDOUT and .ERROR filesThese are the output produced by individual stages of the and may contain detailed errors that are no

Configuration Updates with SRS Prisma244

C HAPTER9SRS PRISMA - WORKED EXAMPLES

Introducing SRS Prisma121.3.12 Mail-based ReportingSRS Prisma now allows brief text reports to be sent to the administrator by email.This can include

SRS Prisma - Worked Examples2469.1 IntroductionSRS Prisma is a powerful and highly configurable system, which allows theAdministrator to automate comp

LION Bioscience 247Example 9.1 Simple file downloadExtract from uniprot_swissprot.i:$UNIPROT_SWISSPROT_DB=$Library:[UNIPROT_SWISSPROT searchName:&apo

SRS Prisma - Worked Examples248Example 9.2 Supporting download and decompression.Extract from uniprot_swissprot.it:$UNIPROT_SWISSPROT_Res = $Resource:

LION Bioscience 249provided as a variable, $gunzip, which is set in $SRSSITE/srsdb.i in order to allowthe Administrator to set the local path to gunzi

SRS Prisma - Worked Examples250Example 9.4 Creating a reformatted file.Extract from uniprot_swissprot.it$UNIPROT_SWISSPOROT_Res=$Resource:[name:UNIPRO

LION Bioscience 2519.3 Handling Multiple FilesThe examples earlier in this chapter have discussed updating of libraries using singlefiles. However, mo

SRS Prisma - Worked Examples252The .i file shows that any file with the extension .dat can be used, and that files-based parallelization (parallelType

LION Bioscience 253Extract from srsdb.i$LibLoc:[$UNIGENEXTR_DB dir:"$dataRoot/unigene/" offDir:"$dataRoot/unigene_tmp/" includ

SRS Prisma - Worked Examples254a library that executes a hypothetical unpackCommand to find the data files of a parent,and create GCG formatted copies

LION Bioscience 255This situation can, however, be alleviated by the use of the $Resource.useFilesattribute. This adds information on file names (and

LION Bioscience 13extends this functionality to allow individual batch commands to be specified fordifferent libraries (e.g. a separate command for EM

SRS Prisma - Worked Examples256of unknown name and size, at the whim of the EMBL administrators. useFiles is notvery useful here since we need to know

LION Bioscience 2579.4.3 Non-SRS Libraries for Data ManipulationThe combination of dependent libraries and inclusion of non-SRS libraries meansthat SR

SRS Prisma - Worked Examples258$PATHWAY_DB=$Library:[PATHWAY group:$PATHWAY_LIBS format:$PATHWAY_FORMAT maxNameLen:100 searchName:'*']$P

LION Bioscience 259Note: $Library.searchName and $Library.format are required attributes for the$Library class which are set to nominal values for $PA

SRS Prisma - Worked Examples260 reformatCommand:"cd %d; formatdb -i nrdb -t nrdb -p T" installFiles:{“nrdb” “nrdb.p*”}]This dependent libr

C HAPTER10SRS PRISMA - COMMON ERRORS

SRS Prisma - Common Errors26210.1 IntroductionThe advice in Chapter 6, SRS Prisma - Troubleshooting should be sufficient to aidrapid error diagnosis,

LION Bioscience 26310.3 Problem: SRS Prisma Tries to Move a Non-Existent File Online and FailsThis is commonly because one of installFiles or usesFile

SRS Prisma - Common Errors26410.7 Problem: SRS Prisma Tries to Run srsbuild Commands with –parts 0 on a Dependent LibraryThis is usually due to use of

LION Bioscience 265(This may reflect a discrepancy between the searchNames specified for use by PRISMA in the $Resource object and the filenames speci

Introducing SRS Prisma141.4.7 ‘Smart’ runPrisma lockingThe PRISMA_RUNNING lockfile is used to indicate that an instance of runPrisma isactive, and pre

SRS Prisma - Common Errors266If this does not solve the problem, it is advisable to check the contents of followingfiles from $SRSPRISMA/<run>/f

A PPENDIXAWGETPRISMA

268A.1 GNU wgetGNU wget is a free software package for retrieving files using HTTP, HTTPS andFTP. The homepage for GNU wget is http://www.gnu.org/soft

LION Bioscience 269Note: The shipped version of wgetPrisma is compiled against OpenSSL v0.9.7d tosupport HTTPS. wgetPrisma can be recompiled against a

270used as they either conflict with SRS Prisma’s use of wgetPrisma, or are notappropriate.

LION Bioscience 271Table 0.1 Permissible wgetPrisma configurationCommand Permissible?ftp_proxy Yes, if proxy settings not already definedhttp_proxyYes

272To use an alternative configuration file, the WGETRC environment variable should beset to the location of the file. This can include an empty file

LION Bioscience 273AarchivingSRS Quality Report . . . . . . . . . . . . . . . . . . . . . . . . . . . 212attributecheckFilesSizes . . . . . . . . .

C HAPTER2INSTALLING SRS PRISMA

LION Bioscience ISRS PRISMA GUIDECHAPTER 1: INTRODUCING SRS PRISMA. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

Installing SRS Prisma162.1 Requirements for SRS PrismaThe following items of software are required for installing SRS Prisma 4.1:• A working SRS 8 ins

LION Bioscience 17Welcome to SRS InstallCopyright (C) 1997-2004 LION bioscience AG. All Rights Reserved.- SRS root directory is /software/srs8Finding

Installing SRS Prisma18Resource objects (stored in .it files) can be used, they should be converted to the new format. To do this automatically, answe

LION Bioscience 192.3 Migrating to SRS Prisma 4.12.3.1 Importing a Global Configuration FileThe files used for configuration by SRS Prisma have change

Installing SRS Prisma20

C HAPTER3CONFIGURING SRS PRISMA

Configuring SRS Prisma223.1 IntroductionSRS Prisma has been designed to work 'out of the box' with all supported SRSdatabases, with minimal

LION Bioscience 23Figure 3.1 The Visual Administration toolbox (VisAd).The main panel of the Prisma tool is divided into five tabs representing five d

Configuring SRS Prisma24Figure 3.2 The Prisma tool main panelTo alter the desired property, select the appropriate tab, and make the change. Tocommit

LION Bioscience 25Configuration, Installation Configuration, Report Configuration andConfiguration Update. For example, to set the maximum number of

II1.3.13 Extended Quality Reports . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 121.4 What’s New in SRS Prisma 4.1 . . . . .

Configuring SRS Prisma26Configuration complete!-----------------------Write changes to configuration file? : [ y ] yWriting new configuration to /srs/

LION Bioscience 27Figure 3.3 The Remote Configuration tabThe following sections describe the configuration process.3.2.2.1 General SettingsMaximum Sim

Configuring SRS Prisma28continue to occupy a channel for the remainder of the operation. If all the channelsbecome occupied in this way, the waiting c

LION Bioscience 29Figure 3.4 Configuring an FTP proxyHost This is the name of the host where the proxy is located.PortThis is the port on which the pr

Configuring SRS Prisma30PasswordSome proxies require a password for authentication. This can be supplied here.FTP Proxy TypeSRS Prisma can deal with a

LION Bioscience 3110HTTP proxy. The proxy host supplied here is not an FTP proxy at all, but HTTP proxy and carries out FTP over HTTP. Note that this

Configuring SRS Prisma32NameThis is the name of the host where the proxy is located.Proxy PortThis is the port on which the proxy listens. By default,

LION Bioscience 333.3 Execution ConfigurationThe Execution Configuration tab allows the fine-tuning of aspects of how SRSPrisma runs the commands nec

Configuring SRS Prisma343.3.1.2 Max. Repetitions of Failed UpdatesNormally, SRS Prisma will attempt to run specified update commands once, andonce onl

LION Bioscience 353.3.1.6 Suppress Warnings from srsbuildNon-fatal parser warnings are normally displayed by srsbuild during the indexingprocess. Chec

LION Bioscience III3.2.2.4 Testing a Remote Connection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 323.3 Execution Configuration . .

Configuring SRS Prisma36Note: The commands provided are suggestions only. They may need alteration torun on a given installation, or that installation

LION Bioscience 373.3.2.2 Sun Grid EngineThe following commands is suggested for use with a typical Sun Grid Engineinstallation, but can be readily al

Configuring SRS Prisma383.3.2.4 Configuration of Batch Queue SystemThe batch queue system panel of the Execution Configuration tab allows the sys-tem

LION Bioscience 39Figure 3.8 Controlling individual phases. The following phases can be controlled individually:• Download (all remote and local file

Configuring SRS Prisma40A batch queue system can also be specified directly in the $SRSETC/prisma/conf file,using the following attributes of the Pris

LION Bioscience 41escapeQuotesescape quotes automatically (see Section 3.3.2.5, Batch Queuing and Quote Escaping, page 41).3.3.2.5 Batch Queuing and Q

Configuring SRS Prisma42Figure 3.9 The Check Configuration tab.Note: Changing these settings may have unexpected consequences for the consis-tency of

LION Bioscience 433.4.1 Show Verbose OutputThis option turns on verbose output of the check phase by default.3.4.2 Show Debugging OutputThis option tu

Configuring SRS Prisma443.4.7 Use COMPLETED Flag CheckingBy setting this option, SRS Prisma can control checking of local files by looking forthe pres

LION Bioscience 453.4.10 Libraries to Check A list of library names can be supplied (as a space-separated list if using thecommand line tool) to check

IV3.4.12 Groups to Check . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 453.4.13 Groups to Exclude from Checkin

Configuring SRS Prisma46Specified time window A string representing a window of time can be specified. Installation will only take place within this w

LION Bioscience 473.5.2 Installation OptionsAdditional options can be used to control installation and can be set using theInstallation Configuration

Configuring SRS Prisma483.6 Report Configuration3.6.1 HTML Report GenerationSRS Prisma produces detailed HTML reports following each update, and these

LION Bioscience 493.6.2 Location for HTML ReportsNormally, HTML reports are written to the $SRSWWW/prisma directory tree, but this canbe altered to an

Configuring SRS Prisma503.6.6 Generate Trace ReportsNormally, SRS Prisma generates a full data and index file listing before and aftereach update. How

LION Bioscience 51configurable, and can be set using the Report Configuration tab. The followingproperties may be configured:Generate quality reportsT

Configuring SRS Prisma52Figure 3.12 The Configuration Update tab.The use of this configuration tool is described in detail in Chapter 8, Configuration

LION Bioscience 53Figure 3.13 Opening a library for editing with the Prisma tool.Selecting the appropriate library and clicking ‘OK’ opens the Library

Configuring SRS Prisma54Figure 3.14 The Library Configuration dialog.The rest of this guide refers to these panes directly with more information on ho

LION Bioscience 55Example 3.1 The Resource object for EMBLNEW stored in SRSDB:emblnew.it$EMBLNEW_Res=$Resource:[ name:EMBLNEW updMethod:mirror remo

LION Bioscience V3.8.1.8 SCP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 643.8.1.9 General Opt

Configuring SRS Prisma563.8.1 Configuring a Library for Remote UpdateThe first step in activating remote updates for a new library is to indicate that

LION Bioscience 57To add a new host using VisAd, ensure that both Update data from remote hostscheckboxes are selected, and click Add on the panel lab

Configuring SRS Prisma58$UNIPROTSWISS_Res=$Resource:[... updMethod:mirror remoteHosts:$RemoteHost:[ hostName:"ftp.uniprot.org" port:

LION Bioscience 59Figure 3.16 Setting an FTP host using a URL.$RemoteHost:[url:"ftp://ftp.uniprot.org/pub/databases/uniprot/knowledgebase/"]

Configuring SRS Prisma60Alternatively, the host, port, download directory, username and password can bespecified separately.Figure 3.17 Setting an FTP

LION Bioscience 61In addition, the FTP transfer mode can be set to ASCII or binary by setting the FTPmode drop-down menu appropriately, or by setting

Configuring SRS Prisma62Figure 3.18 Specifying an HTTP host.$RemoteHost:[url:”http://somewhere.com/path/to/data/”]Note that transfer mode and contentF

LION Bioscience 63Figure 3.19 Specifying an HTTPS host.$RemoteHost:[ url:”https://somewhere.com/path/to/data/” sslCertFile:”/path/to/certificate” s

Configuring SRS Prisma64Figure 3.20 Specifying a local 'file' host.$RemoteHost:[url:”file:/path/to/data/”]3.8.1.8 SCPFiles can also be secur

LION Bioscience 65Additionally, the SSL identity file and configuration file to use with ssh/scp can besupplied, and any additional options to be pass

VI3.10.2.2 Postprocessing and Compressed Files . . . . . . . . . . . . . . . . . . . . 1013.11 Installation Settings . . . . . . . . . . . . . . .

Configuring SRS Prisma66Note: SSL keyboard-based authentication cannot be used with SRS Prisma as thereis no facility for supplying this in batch.3.8.

LION Bioscience 67Figure 3.22 General remote options.$RemoteHost:[ url:"ftp://ftp.uniprot.org/pub/databases/uniprot/knowledgebase/" usePro

Configuring SRS Prisma68To add a remote file pattern, click on the Add button of the Files to download panelto open a Remote File Editor dialog box fo

LION Bioscience 69The files on the remote site are specified using a regular expression, either enteredinto the Remote search pattern box or set as th

Configuring SRS Prisma70Figure 3.24 Adding a file conversion pattern.To represent this directly within the RemoteFilePattern object, separate lists of

LION Bioscience 71Figure 3.25 Specifying an archive.In this case, the local search name pattern is ignored.To achieve this using a RemoteFilePattern o

Configuring SRS Prisma72• Use date from file – take the timestamp from the file specified • Use date from command - take the UNIX time returned by the

LION Bioscience 73The command to extract the files is entered as a separate target in the makefile. Assuch, it runs in its own standard sh shell, and

Configuring SRS Prisma74The use of %s not only allows parallelization of the unpackCommand, but is also anecessary condition if the prismamakefile is

LION Bioscience 75Figure 3.27 Adding an unpack command.If editing the Resource object directly, the unpack command(s) can be specified asmembers of th

LION Bioscience VII3.14.3 Specifying Preprocessing Commands . . . . . . . . . . . . . . . . . . . . . . . . . 1213.14.4 Non-SRS Dependent Libraries

Configuring SRS Prisma76Note: Use of mixed per-file and per-database unpackCommand strings in a list willbreak the parallel chain of download-pretrans

LION Bioscience 773.8.1.15 Using RecursionNormally, SRS Prisma assumes that all remote files are on a single directory.However, sometimes it is necess

Configuring SRS Prisma78longer contains any matching files, SRS Prisma has the potential to delete all localflat-files for that library. To minimize t

LION Bioscience 793.8.1.20 Local Contents FileNormally, SRS Prisma lists files locally in the online data directory using ls –l to findnames, dates et

Configuring SRS Prisma80ensure that the same configuration (parsers and Icarus configuration files) or query-ing may not work properly.3.8.2.1 Specify

LION Bioscience 81Figure 3.29 Specifying a remote index host.To add a remote host for indices directly to the Resource object, add a new instanceof a

Configuring SRS Prisma823.8.2.2 Specifying Remote Index FilesAs with data, at least one remote file pattern is needed, but these are configured dif-fe

LION Bioscience 83If editing the Resource object directly, new RemoteFilePattern objects are added tothe remoteIndexFiles list.The specification for r

Configuring SRS Prisma843.8.2.3 Specifying Preprocessing CommandsIf the indices require unpacking (unzipping, or extraction from a tarball), then one

LION Bioscience 85For direct configuration with the Resource object, one more commands can beentered into the unpackIndexCommand attribute. Note: When