CSOFT note 0.02 - Running ENGINE for fun and profit.

or, `How to use the Hall C analysis software'

Brought to you by the fine people at CEBAF(a.k.a TJNAF , a.k.a. jiffy lab , a.k.a. jeffy lab)

GoPACK!

This Version - Feb 23, 1999

Latest Version - http://www.jlab.org/~johna/

0. NOT READING THE MANUAL:

Do NOT not read the manual. But since you will (not) anyways, look for the sections with '##' in the left margin for things you NEED to know. You will need to read the entire section marked, because sometimes you need to do two (or more) things to the file. In addition, you should read the hall C UNIX reference manual (ref 1). This document should be enough to setup and run the default engine. For important details on changing code, histograms, tests, etc... see ~cdaq/documents/analysis_code/ctp.hints (or ctp_manual.ps for more detailed information). Finally, note that there are TWO DIFFERENT ~cdaq directories. On the Jlab machines (jlabh1/2/3/4,jlabs1/2/3/4,etc...), the cdaq home directory is NOT THE SAME as the real home directory (as seen from the cdaqh* and cdaqs* machines). Oscar likes the counting house, and so he always logs into the cdaq machines, which has the replay software. Of course, much of the other analysis software (calibration code, monte carlos, etc...) is only visable on the jlab machines. But oscar doesn't care. He's a data acquisition kind of cat.

1. BE LIKE OSCAR (or 'setting up engine in your very own account').

This section chronicles Oscar's attempt to set up engine in his account. If this confuses you, globally replace 'oscar' with your name, and 'Oscar' with Your Name. However, I take no responsibility for any personal insults generated in this fashion. Buyer Beware! Oscar is a boring person, and so used the same directory names as I did, in order to avoid confusion. Thus, he wanted to set up the engine

##

WE INTERRUPT THIS DOCUMENT FOR A SPECIAL ANNOUNCEMENT: WORD HAS COME FROM ROLF T. ENT THAT WHEN YOU FIRST SET UP YOUR ACCOUNT ON THE HP, YOU MUST USE THE FOLLOWING COMMAND (ONCE ONLY) TO PROPERLY SET THE DEFAULTS FOR YOUR ACCOUNT.

'xrdb -merge .Xdefaults'

REPEAT:

'xrdb -merge .Xdefaults'

We now return you to the ENGINE document, already in progress.

##

   The old school (pre-oscarup) method:
     'cd ~oscar'         ;go to the parent directory of your future replay site.
     'mkdir replay'      ;create the replay directory
     'cd replay'         ;go to the replay directory (easy, and fun too!)
     'cp -r ~cdaq/replay_example_latest/* .'
                         ;Yup, that's all there is to it.
     'chmod -R u+w *'    ;allows Oscar to edit his files.
     'rm core'           ;get rid of the empty file called core
     'rm replay'         ;get rid of the (possibly existing) executible'
     'ln -s /dev/null core'           ; cheesy way to avoid core files
     'ln -s SRC/engine_replay replay' ; link to the (soon to be) executible.
     'edit dosetupreplay'   ;make your replay directory the REPLAY_DIRECTORY
     'source dosetupreplay' ;done. Whew!
     'cd SRC'            ;go down
     'make'              ;make the executable. In order to access the libraries
                         ;you must have setup coda (done in dosetupreplay) and
                         ;setup gnu (to get the gnu make utility).

   OR (for the exceptionally lazy)
     'cd'
     'cp ~cdaq/replay_example_latest/oscarup .'
     'source oscarup ~/replay'

   OR (for the nearly comatose)
     'source ~cdaq/replay_example_latest/oscarup ~/replay'

IMPORTANT NOTE:

Never to use the flag '-R' instead of '-r' in the cp command. '-r' is recursive copy, which copies all files, directories, and sub-directories. '-R' is retroactive copy, which will copy all files that have ever been on the machine. In an attempt to fit all of the data on your current disk, the machine will spin the hard drive faster and faster, eventually launching the drive into a low earth orbit. It is generally recommended that you not use the '-R' flag, relying instead on frequent 'tar' backups to restore old files. Using the '-R' flag with the chmod command is much safer, leading only to the occasional (generally harmless) time paradox.

NOTE #2:

Make sure never to use the command 'tar -cvR'.

2.WHAT OSCAR FOUND (or 'what ARE all these files??!?!?')

a) DIRECTORIES:

~oscar/replay:

The main directory of the replay universe. This directory contains the input files used by the engine, and is usually where the engine is run.

~oscar/replay/SRC:

This directory contains the engine Makefile, and any source code that oscar wants to use in place of the default engine library code.

~oscar/replay/PARAM:

There is a lookup table that tells the engine what parameters to use for a given run. The parameter files are all kept here, to keep replay clean ('The Odd Couple' was wrong, Oscar was the neat freak).

~oscar/replay/MAPS:

There is a lookup table that tells the engine what parameters to use for a given run. The map files are all kept here, to keep replay clean.

~oscar/replay/DBASE:

There is a lookup table that tells the engine what parameters to use for a given run. The pointers to the parameter files are all kept here, to keep replay clean. (What was life like before cut and paste?).

~oscar/replay/paw:

This is where I (I mean 'Oscar') puts the histogram output files generated by the engine. You may want to do otherwise. If so, 'rmdir paw' is the protest method of choice.

~oscar/replay/scalers:

This is where I (I mean 'Oscar') puts the scaler report output files generated by the engine. You may want to do otherwise. If so, 'rmdir scalers' is the protest method of choice. (come cut and paste with me).

~oscar/replay/peds:

This is where I (I mean 'Oscar') puts the pedestal output files generated by the engine. These output files are mainly used when taking data (the threshold files are used to program the ADC thresholds for data sparsification). When replaying, these can be used to check to see if the pedestals changed centroid or width.

~oscar/replay/???:

If there are any other directories generated, oscar sure doesn't know about it. You may want to store your input parameter files, or output files in another directory. To do so, just create the appropriate directory and change the path given in REPLAY.PARM for the files you move. Note that the environmental variable ENGINE_CONFIG_FILE is the pointer to REPLAY.PARM. You must change this pointer if you wish to move REPLAY.PARM. The file dosetupreplay has the definitions for the environmental variables used by the engine. If you need more detail on how to create directories or move files, running the engine probably isn't for you.

b) FILES: ('##' indicated that you must modify the file, '*' means important)

README:

Once again, if you haven't yet figured out what this is, running the engine probably isn't for you. (Actually, since I love filename completion, and README is too close to REPLAY.PARM, this is now called OSCAR_jan99.txt).

*dosetupreplay:

This file sets up coda, and sets up pointers to the Csoft source directories and your replay directory/files. You should put the command 'source ~oscar/replay/dosetupreplay' in your .login file in order to setup coda when you log on (if you're oscar, at least). You must also change the definition of REPLAY_DIRECTORY, so that is points to the location of your REPLAY.PARM file. ALSO, to make the Makefile work, you need to have 'setup gnu' in your .login file or in dosetupreplay (setup gnu replaces make with gnu-make (sometimes gmake), which is required to make the engine).

*REPLAY.PARM:

This file tells the engine where all of the input files it needs are located, and also what file names to use for output. From here you pick the run to analyze, and the test, histogram, and parameter files to use. If the engine has a brain, this must be it.

jan99.bears_fans:

Oscar is a Green Bay Packers fan. When Oscar discovers a Bears fan, he puts their username here. Oscar takes no responsibility for any mysterious core dumps that happen to ANYONE who uses the engine.

*jan99.hist:

This file defines the histograms to be filled by the engine. There are uhist blocks (filled by the user with specific calls in the engine source code) and hist blocks (which are filled automatically after each event). If the variable to be histogrammed is registered, you can add a histogram by adding another line in one of the 'hist' blocks (or adding a new block). The format is as follows: '[histname],[data_source],numbins,lobin,hibin,test' - test is optional. The other hist files here are '#include'd into the main hist file. NOTE: all histogram and test blocks SHOULD REALLY HAVE the 'group=' flag, which determines when the test/histogram block should be evaluated. The standard group names (hms,sos,both) correspond to the hardware trigger types (hms,sos,coin). In order to fill a histogram for ALL hms events, you need 'group=hms group=both' in the begin line.

jan99.database:

This is the lookup table telling what parameter files are appropriate for a given run. This contains pointers to parameter files which select the appropriate positions/calibrations for the run. These usually change only between runs, or when there is a hardware change.

jan99.kinematics:

This is the lookup table for quantities which vary run to run. Spectrometer angles, momenta, beam energy, and other such quantities are set here. We do not yet have a system for automatically determining who is at fault when a run is useless, you must check logbook entries by hand when passing out the red hat (Bates reference that probably no one will get. Don't worry, you're not missing anything good). Note that the engine gets the kinematics from several source. First, it takes the values in the run information event (entered when the run began). Next, it overrides these with any values in the .kinematics file. Therefore, you don't need to have entries in the kinematics file for a run unless there was an error in the information typed in at the beginning of the run (or for information not included in the beginning of run info event). Finally, any command line parameters are applied, overriding everything else.

*jan99.test:

This file defines the tests used in the histograms. The syntax is to complicated for Oscar to understand, so if you can't figure out how to define a new test you need, go pester Steve Wood (or look in the CTP manual). See jan99.hist for notes on assigning groups to the test blocks.

hms(sos)_recon_coeff.dat:

HMS (SOS) reconstruction matrix elements. They're wrong, but that doesn't make us love them any less.

h(s,g)report.template:

Oscar underestimates the importance of this file, and therefore it isn't marked '*'. It is very useful for outputting end of run variables. The three files dump hardware and software scalars, and lots of other useful information. One for the HMS, one for SOS, and one general summary file.

johns.favorite.report:

Yaaaaaa, OOascar saays that-a thees is-a a reel-a good-a fiile.

*replay:

It looks like a file, it smells like a file, and it swims (i.e. sinks) like a file, but it isn't a file. Replay is a symbolic link to the engine executable file that will exist in the SRC subdirectory after 'make' is run. The link always points to where the executable is supposed to be and it stays behind if you delete the executble. If you do delete the link and want to recreate it, The command goes something like this........

'ln -s ~oscar/replay/SRC/engine_replay ~oscar/replay/replay'

Your environmental variable ENGINE_CONFIG_FILE tells replay what file (usually REPLAY.PARM) has the information about the input and output files to use. These can be overridden at the command line. You can set any CTP variable (i.e. anything in one of the fortran common blocks) at the command line in the following manner:

'replay ctp_variable_name={value}' (no spaces allowed before or after '=')

the following equivilences have been put in the code in order to make life easier for passing command line arguments:

grun == gen_run_number           ;run number
gstart == gen_run_starting_event ;starting physics event #
gstop == gen_run_stopping_event  ;# of physics events to analyze
gtrig(1-4) == gen_run_enable(1-4);1=hms,2=sos,3=coin,4=ped triggers.
                                 ;gtrig1=0 disables hms triggers,
                                 ;gtrig1=1 enables triggers (enabled by default).

In addition, kinematics, parameters, and run-time switches can be set at the command line. For example:

'replay grun=4000 gstart=3000 gstop=10000 gtrig2=0 hpcentral=1.5 hdebugdumptof=1'

will analyze run 4000, starting at PHYSICS event 3000, ending after 10000 ANALYZED events, with sos triggers ignored (but not coincidence). In addition, it will set the HMS central momentum (hpcentral) to 1.5 GeV, and turn on time of flight output (hdebugdumptof=1). Command line parameters override values read from the data file or parameter files.

*evdisplay:

Also looks,smells, and swims like a file, but is once again a link. This is the link to the event display code. This is more complicated than oscar can handle (after all, he is just a cat). Those with more courage (or whatever you call it) can look at ~cdaq/documents/analysis_code/evdisplay.txt for more details. Those who like a challenge can figure out how to run the event display by reading War and Peace.

*core:

As long as we're talking about files that aren't really files, let's discuss 'core'. This is not a pregenerated 'replay' core file, those are easy to generate on you own. This core file is another link, and it points to /dev/null. This means that when your coredumpper program (a.k.a. engine) does its stuff, you don't have to wait for it to write out the core. The core just gets sent to nowhere land.

*MAPS/jan99.map.*:

map file which tells the engine how to map from fastbus addresses to software variables. Very complicated stuff. The name indicated which map should be used for each run. The changes are due to cabling changes in the Ginsu (TM) cheese grater/meat dicer/egg slicer/drift chamber. (only $19.95, and comes for a limited time with the Popeil potato masher/garlic press/Lead-glass calorimeter.

*SRC/Makefile:

This file tells the computer what to do when you type 'make' in the SRC directory. The make command will compile and link the engine code and create the executable, 'engine_replay'. When you want to change the engine, you copy the file you wish to change from the Csoft source tree, make the changes, and add the routine to the list of 'my_objs' in the Makefile. Then, the make command will link your changed code to the original code to create your new version of the analyzer. For example, to change the subroutine h_oscar.f in the routine, you do the following:

'cp $HSRC/h_oscar.f ~oscar/replay/SRC';copy code to your dir
'chmod u+w h_oscar.f'                 ;give yourself write privilege (but
                                      ;remember, write is a right, not a privilege).
change h_oscar.f                      ;make your modifications
add h_oscar.o to Makefile (in my_objs);tell 'make' to use your new version
run 'make' in your SRC directory      ;create the executable.

SRC/engine_replay:

The executable.

SRC/evidisplay:

The other executable.

SRC/*.f:

All code you wish to use that is not part of the default engine goes here. At the moment, there are some routines that have been updated, but are not yet checked in to the official version of the engine. Once they are checked in, you can stop using these versions. For my (your) convenience, the main routines engine.f and evdisplay.f are also kept here.

SRC/*.cmn:

In addition to putting fortran code here, you can put updated common blocks. Any common block sitting in this directory will be included in whatever routines you compile in the SRC directory (the ones you've explicitly copied over from the main source tree). However, any variables added in the common block will NOT be registered in CTP (and will therefore not be accesable to CTP test/hist/scaler files) unless you add the registration routine to the Makefile. For a common block called 'gen_oscar.cmn', you need to include 'r_gen_oscar.o' to the makefile. This will generate and compile a fortran routine that registers all of the variables. Note that the fortran routines that you don't compile yourself (the ones from the source tree libraries) will have the OLD version of the common blocks. Therefore, if you add anything to the .cmn files, it is best to add entirely new common blocks to avoid confliting definitions with the old common blocks. Alternatively, you can include new variables in the old common blocks, but the MUST GO AT THE END OF THE COMMON BLOCK. All code you wish to use that is not part of the default engine goes here. At the moment, there are some routines that have been updated, but are not yet checked in to the official version of the engine. Once they are checked in, you can stop using these versions. For my (your) convenience, the main routines engine.f and evdisplay.f are also kept here.

SRC/*.c:

There is NO C code here. I HATE C!!! (ref. 2).

3. OSCAR RUNS REPLAY FOR FUN AND PROFIT (Declares bankruptcy following tuesday)

4. OSCAR WONDERS IF IT WILL WORK (or, 'oscar wonders if it will work')

If this is adequately incomprehensible, let my boss(es) know, so that I won't be asked to do this again :-)

Like most things, you are likely to discover that the engine sort of works. Or maybe it will kind of work for you. I don't know. However, sometimes it works, but complains anyways. This I might be able to help you with. The following are some of the error messages the engine likes to give.

BEGINNING OF RUN ERROR/STATUS MESSAGES:

"Created test result test.ctrig"

This is a CTP error. The test 'ctrig' was not declared before being used. You can define a test without declaring it first, but you might have trouble with the variable type.

"thTest: Unregistered variable {varibale_name} - Test booking error in line 3"

This is another CTP error. A test was defined using a variable that does not exist. e.g. the test 'goodxy = goodx && goody' will give an error if goodx or goody are not defined. This leads to the unregistered variable and test booking error. This can occur when defining histograms as well as tests. The line number is the line within the test or hist block. The error message will eventually be modified to give the name of the unregistered variable and/or the block name.

*"Warning! Danger Will Robinson! Inconsistant Thresholds approaching!"

This means that the engine has found a 'large' difference between the pedesal values for the ADCs and the sparsification threshold that was programmed into the ADC. This is a sign that new thresholds may be needed. The file ~cdaq/coda/hms(sos)_thresholds.dat contains the values that are programmed into the ADCs whenever you 'prestart' a run. When you analyze a run, the file ~/replay/PEDS/hms(sos)_thresholds.{runno} is created with the analyzer's idea of good thresholds.

"RZCDIR. Unknown directory //RAWH:"

This is one of those error messages you get when nothing is wrong. Think of it as a status message telling you if you are dumping ntuples or not. No error message = not saving ntuples. Error message = Ntuples being written.

"event# 15003 F"

Just a progress report. Dumps the event number every 5000 physics events. The 'F' means that the ABORT variable is .false., though if it were .true., the engine would abort, and you wouldn't get the message.

"Finished dumping histograms/scalers for first 10232 events"

The histograms and scaler reports are dumped every once in a while. This way you don't have to finish the run in order to start looking at the results, and you don't lose everything if you interrupt the replay. The ntuples do NOT get closed properly until the end of the run.

**"roc 1 has no data for event 4420 scanmask= A0828A"

The roc had one or more modules that did not have data. This could be a hardware failure or a readout/syncronization problem. The scanmask is just a bit mask, telling you which modules had data to be read out. For examble, the last two bytes (8A) give a bitmask '10001010'. This corresponds to slots 1, 3, and 7 having data to be read out (since the last bit corresponds to slot 0). Notify shift leader or data aquisition person.

**"ERROR: BAD FB value evfrag({nnn})={nnnnnn} ROC={n} event={n} "

The code has found a bad fastbus word. THIS IS BAD. Notify shift leader or data aquisition person.

**"SHIT:misidentified real data word as a header "

Also bad, as you might have guessed. Notify shift leader and/or data aquisition person.

**"g_decode_fb_detector: Max exceeded, did={n}, max={n}: event {nnnn}"

A detector had too many hits to be decoded fully. The did number is the detector ID. You can look at the MAP file in order to determine which detector had the problem. If it was the wire chambers, it is probably OK as long as it only occurs once. The other detectors should not be able to get too many hits. Notify someone of the problem.

"skipping outoforder scaler event:... "

Just a status message. Sometimes the scalers are not read out in time order (they get buffered and read out several events at a time, but not in the order they were taken). In order to avoid false scaler roll-over, we skip scaler events that to not come in order.

5. OSCAR WANTS MORE DOCUMENTATION (or 'Oscar is a masochist')

Some documents are available on the web: www.jlab.org/Hall-C/data_reduc/software-docs.html and ftp.jlab.org/pub/hallc/. A collection of some of the more useful ones are on the hall C data acquisition computers (cdaq1/2) in ~cdaq/documents/.... Here is a version of INDEX.DOCS which lists all of the files stored here (This version is almost certainly out of date). For additional information, check out the file EVERYTHING.ELSE (same location, probably also out of date). Check ~cdaq/documents/* for the latest version of the INDEX, and all documents.

REFERENCES:

1 (and only). "The Unix-Haters Handbook (barf-bag edition)". IDG Books Worldwide, Inc.

2 (so I lied). "Why JOHNA can't read C". Fictitious Press, Ltd.

3. "B. Joel, public communication"