We are two >'s deep now. Why not go for more?
> >The low-level software we are using was written by Thierry Auger and
> >Marc Swynghedauw under the supervision of Dave Barker. The main
> >documents for their work are their master's theses, Refs. 2 and 3.
> >Prominent features of the system are a new HV EPICS record with
> >multiple VAL fields to accommodate the multiple constants associated
> >with each HV channel, and a set of asynchronous tasks that form an
> >interface between record support and the arcnet driver software.
>
> precision: not only the low-level : Marc wrote also a GUI for HV which
> worked quite well with one mainframe. I have made a demonstration on November
> 1995 which showed in action the GUI for one mainframe+the low-level software for
> 5 mainframes. Problems where encountered when using the GUI with more than one.
Accuracy: we are not planning on using that gui. A consensus was
formed among those who have test driven it that it is unacceptably
slow. Also, it the source runs to 4521 lines (yes 4.521E+3) of tcl
script and none of our happy few have ever used tcl.
>
> ------------------------------------------------
> >o Download of hardware constants
> >Presently, if one tries download all of the system constants by
> >repeated calls to channel access, the database will not update some
> >key values, most notably the measured voltage of a channel. Also error
> >messages appear on the IOC console. This probably has to do with the
> >fact that the software does not wait for a response from a mainframe
> >(the mainframe responds to every command) before going on to issuing
> >the next command. Some element of data is getting lost or the desired
> >logical execution thread is getting broken in a flood of channel
> >access requests.
>
> The measured voltage are not to be changed by channel access. This is an
> information coming exclusively from the hardware.
The last statement is perfectly correct and consistent with our use of
the software. However, the values of set voltage, ramp up rate, trip
level, etc. _are_ changed by channel access and if you do too many of
these changes at once, the measured-voltage-updating procedure is a
casualty.
> The software does not wait for a response:
> _ For the monitoring part(99% of the time)it does, a look at the code
> can prove it.
> _ For channel access change, it doesn't. The ARCNET communication scheme
> is that transmission is allowed if BOTH part are ready. This means that if the
> driver transmits a message therefore the mainframe was ready to receive it. This
> makes the flow of data, much smaller, which I have been able to see directly
> with a VME bus scanner. Rather, is the size of the different pipes used in the
> process large enough? these are the kind of question to raise before questioning
> the driver.
I had a suspicion that there was some kind of software pipe that was
overflowing and causing the problem and I think that this last
statement confirms it. Thank you.
It sure would be kinda nice to know where this is happening. It may
make the going rough if we cannot read back measured voltages.
> One more note on that, LeCroy recently admitted that their driver had a
> bug with an unforseen transmission case("Lecroy report on HV" e-mail, april 17).
> From there comes their suggestion to slow down the rate of transmission
> as a work around that bug. As far as I know they also solved the problem and
> updated the mainframe's software, allowing operation without change to hall B
> software.
My understanding was that the change made to the mainframe's firmware
was made specifically to address the mode that we have been using to
communicate with the arcnet: call again and again without waiting for
a response. This was a mode that was not anticipated in their design.
> >o A new approach?
> >
> >The present HV software is not very well understood by Hall B
> >personnel. As such, we will have difficulty maintaining it. One
> >solution is to go to an entirely new, simpler scheme. If the approach
> >chosen is the one Bonnie Madre is using for LeCroy 1454 control, then
> >our Accelerator Division colleagues could maintain the code for us.
> >
> >Issues involved in this decision are:
>
> > - Memory limits
> > - Complications due to large channel count
> > - Need for arcnet driver code
> >
>
> We (D.Barker and I) looked at the control code for LeCroy 1454. We decided that
> in regards with our control problems, another scheme was needed. The choice of
> developping a new record for hall B has been made during a meeting involving
> every interested part somewhere around april 95 (B.Mecking, D.Barker, E.Smith,
> G.Heyes, etc...).
Of whom only one is an EPICS expert (no offense guys).
> Memory limits was one of the issue at that time and it was
> decided after having presented different schemes(e.g. one record by channel, one
> record by card) that the HV channel record was the best compromise between
> memory size, sizability, complexity of the software, flexibility and time needed
> to develop it. At that time was made the choice to have configuration files from
> where every tools(database building, GUI, low-level soft) could configure
> themselves. The memory size wasn't considered as an issue ("Memory is cheap")
It's not clear to me that if we maintain VAL field for only those
parameters that the gui cares about, we can't get away with
standard records. Chip tells me he doubts it and he is probably right,
but I want to do a calculation before dumping the idea.
> It took one year and a half of manpower to write it and it had to have some
> changes afterword. Since developping is time-consuming I suggest rather to use
> existing software with maybe some additions.
I sort of hate to admit it, but it's taken a year and a half of
manpower (6 months * (Sebatien + Kathy + Mark)) to turn what we
started from into a working system and we still have not
succeeded. Granted, most of this time has been spent climbing the
EPICS learning curve, but that does not exactly make it good news.
> The HV configuration is going to be quite complicated anyway because of the
> large number of channels and if one wants to use EPICS for HV control, there is
> no simple way of doing it.
>
> Is the driver not working?
If there's an overflowing soft pipe, I'll bet that the driver is
indeed working.
> > Why does initialization take so long? Is it really necessary to verify
> >that the files conform to the hardware? Is it even desirable?
>
> one minute by mainfraine is not so slow. Initialisation makes three things:
> _ initialise ARCNET board. not more than 1 second needed.
> _ Check that the database is in concordance with the hardware. It is
> highly desirable at the beggining when setting up the configuration.
> _ Read out all informations contained in the mainframe relevant to EPICS
> operation and/or operator(Ramp up settings, limits, etc...).
OK. Let's guess 0.2 s per operation. 10 values per channel, 12
channels per card, 16 cards per crate, say 4 mainframes. That's
about half an hour before you can look at a single number from
the hardware when starting from scratch. And yes, big errors on this
estimate, but it's a bit scary.
> >Should we change the software architecture to conform to the
> >call-and-response mode that was intended by the LeCroy engineers?
>
> It's possible and difficult with several mainframes controlled by
> one IOC.
I agree that the present architecture is not easily changed to use
this mode.
> And I don't see why it is needed. It will be much slower also.
My understanding is that the time bottleneck is in waiting for the
mainframe to complete it's operation. Waiting for a response then
should not affect the rate of operations.
> > Card-by-card rather than channel-by-channel initialization routine
> >needed. Thierry gave us a skeleton program to start with here. Speeds
> >up a download by an order of magnitude.
>
> It is already the case for the checking parts of the initialization. For the
> rest, it needs only 1 command by channel. To speed it up, would require the use
> of the DUMP command which is not easy to handle in a networked
> environment.
The load commands of the mainframe allow an update of all channels in
a logical module with a single command and this one command will
execute about as fast as that for a single channel. In the case of the
PMT supplies, this should buy us a factor of 12 in downloading speed.
Why is DUMP hard in a networked environment?
>
> > The mainframes have a built-in memory with battery backup. The values
> >have been seen to survive unplugging and transport across the lab
> >site. Can we exploit this feature to avoid time-consuming
> >initialization procedure? For example, DUMP the values and onlydownload those
> that need to be changed.
>
> We (D.Barker, M.Swynghedauw & I) decided to rely on Unix stored files rather
> than battery backup. The batteries are given one or two year time-life. What if
> one forgets about it?
The battery backup example was meant to emphasize the robustitude of
the memory. We of course would not depend on the battery backup. We
don't plan on unplugging the mainframes during normal operation.
> >Do we need a system to remotely reboot the IOC's independent of the
> >state of the IOC or the network? A mechanical system? What does ACC do
> >about this?
>
> P.Bonneau bought vme crates that allow remote monitoring of their status as well
> as remote reboot. It needs a GPIB interface to be controlled. The plan was to
> control the GPIB via an IP pack on a vme 162. The driver was available from the
> accelerator. Peter must know about that.
That is good news. Peter are you listening? We need to talk.
--
Mark M. Ito, Thomas Jefferson National Accelerator Facility
12000 Jefferson Ave., Mail Stop 12H, Newport News, VA 23606
Email: marki@cebaf.gov, Office: (757)269-7175, Pager: (757)680-7175
^ ^
Note: new area code and prefix | |