How this Event Source Works

#include <JEventSource_EVIOpp.h>

Inheritance diagram for JEventSource_EVIOpp:

Public Types
enum	EVIOSourceType { kNoSource, kFileSource, kETSource }

enum	EmulationModeType { kEmulationNone, kEmulationAlways, kEmulationAuto }

Public Member Functions
	JEventSource_EVIOpp (const char *source_name)

virtual	~JEventSource_EVIOpp ()

virtual const char *	className (void)

void	Dispatcher (void)

jerror_t	GetEvent (jana::JEvent &event)

void	FreeEvent (jana::JEvent &event)

jerror_t	GetObjects (jana::JEvent &event, jana::JFactory_base *factory)

void	LinkBORassociations (DParsedEvent *pe)

uint64_t	SearchFileForRunNumber (void)

void	EmulateDf250Firmware (DParsedEvent *pe)

void	EmulateDf125Firmware (DParsedEvent *pe)

void	AddToCallStack (DParsedEvent pe, JEventLoop loop)

void	AddSourceObjectsToCallStack (JEventLoop *loop, string className)

void	AddEmulatedObjectsToCallStack (JEventLoop *loop, string caller, string callee)

void	AddROCIDtoParseList (uint32_t rocid)

set< uint32_t >	GetROCIDParseList (uint32_t rocid)

Static Public Member Functions
static const char *	static_className (void)

Public Attributes
bool	DONE

bool	DISPATCHER_END

std::chrono::high_resolution_clock::time_point	tstart

std::chrono::high_resolution_clock::time_point	tend

uint32_t	MAX_PARSED_EVENTS

mutex	PARSED_EVENTS_MUTEX

condition_variable	PARSED_EVENTS_CV

list< DParsedEvent * >	parsed_events

std::atomic< uint_fast64_t >	NEVENTS_PROCESSED

std::atomic< uint_fast64_t >	NDISPATCHER_STALLED

std::atomic< uint_fast64_t >	NPARSER_STALLED

std::atomic< uint_fast64_t >	NEVENTBUFF_STALLED

uint64_t	MAX_EVENT_RECYCLES

uint64_t	MAX_OBJECT_RECYCLES

EVIOSourceType	source_type

HDEVIO *	hdevio

HDET *	hdet

bool	et_quit_next_timeout

vector< DEVIOWorkerThread * >	worker_threads

thread *	dispatcher_thread

JStreamLog	evioout

uint32_t	F250_EMULATION_MODE

uint32_t	F125_EMULATION_MODE

uint32_t	F250_EMULATION_VERSION

Df250EmulatorAlgorithm *	f250Emulator

Df125EmulatorAlgorithm *	f125Emulator

bool	RECORD_CALL_STACK

set< uint32_t >	ROCIDS_TO_PARSE

list< DBORptrs * >	borptrs_list

bool	PARSE

bool	PARSE_F250

bool	PARSE_F125

bool	PARSE_F1TDC

bool	PARSE_CAEN1290TDC

bool	PARSE_CONFIG

bool	PARSE_BOR

bool	PARSE_EPICS

bool	PARSE_EVENTTAG

bool	PARSE_TRIGGER

bool	PARSE_SSP

bool	APPLY_TRANSLATION_TABLE

int	ET_STATION_NEVENTS

bool	ET_STATION_CREATE_BLOCKING

bool	LOOP_FOREVER

uint32_t	USER_RUN_NUMBER

int	VERBOSE

int	VERBOSE_ET

float	TIMEOUT

uint32_t	NTHREADS

bool	PRINT_STATS

bool	SWAP

bool	LINK

bool	LINK_TRIGGERTIME

bool	LINK_BORCONFIG

bool	LINK_CONFIG

bool	IGNORE_EMPTY_BOR

bool	TREAT_TRUNCATED_AS_ERROR

string	SYSTEMS_TO_PARSE

uint32_t	jobtype

bool	IS_CDAQ_FILE = false

Detailed Description

How this Event Source Works

Overview

This event source represents a complete rewrite of the original JEventSource_EVIO class. It is highly optimized and while it gets rid of some of the complexities that developed in the original over time, in some ways it replaced them with complexities of its own.

In a nut shell, this source launches a minimum of 2 dedicated threads for reading in and parsing the events. These are in addition to any threads the JANA framework creates. There is a single "dispatcher" thread that reads in the events and assigns them to one of a group of "worker" threads.

JANA itself has a similar structure in that there is a single "EventBufferThread" that grabs fully parsed events from this class via its GetEvent method, and then hands them to one of a group of "processing" threads where the actual analysis begins.

One benefit of this design is that it allows certain objects, methods, or member data to be modified by only a single thread throughout its life. This avoids the use of locks which introduce inefficiency in a multi-threaded application.

BORconfig objects

Typically, BOR events only occur at the begining of a file and a given job will see only one. However, we must support the possibility of multiple BOR events in case we are reading from an ET system or file of merged events.

BOR objects are applicable to all events in the stream until another BOR event is encountered. We therefore need every event to get a copy of the BOR config object pointers. Thus, they must be kept in a place that all worker threads see. This is the borptrs_list object in the JEventSource_EVIOpp class.

Here's the sequence for how the BOR event is handled:

When a BOR event is parsed it creates a DBORptrs object which holds a complete set of BOR objects. The pointer to this DBORptrs object is left in the DParsedEvent object which would otherwise be NULL.
When JEventSource_EVIOpp::GetEvent() is called, it will see the non-NULL borptrs pointer in DParsedEvent. At this point it will copy it to the front of borptrs_list, effectively giving ownership to JEventSource_EVIOpp. Note that aside from the destructor, borptrs_list is only accessed here which is only called from the event reader thread. This means we don't have to use a lock.
If the borptrs pointer is NULL, then JEventSource_EVIOpp::GetEvent will copy the front pointer from borptrs_list into the DParsedEvent. We rely on the events being in istream order here to only apply the BORconfig objects to events after the BOR event itself.
When GetObjects is called, it calls DParsedEvent::CopyToFactories where the BOR objects are copied into the appropriate factories. It then calls JEventSource_EVIOpp::LinkBORassociations to add the BORConfig objects as associated objects to various hit objects.
Only when the JEventSource_EVIOpp object is destroyed are any BORConfig objects deleted. We don't implement a mechansim to keep track of which DBORptrs objects are still in use so we can't delete them sooner. This shouldn't be a problem though since BOR events are rare.

Definition at line 104 of file JEventSource_EVIOpp.h.

Member Enumeration Documentation

enum JEventSource_EVIOpp::EmulationModeType

Enumerator
kEmulationNone
kEmulationAlways
kEmulationAuto

Definition at line 113 of file JEventSource_EVIOpp.h.

enum JEventSource_EVIOpp::EVIOSourceType

Enumerator
kNoSource
kFileSource
kETSource

Definition at line 107 of file JEventSource_EVIOpp.h.

Constructor & Destructor Documentation

JEventSource_EVIOpp::JEventSource_EVIOpp ( const char * source_name )

Definition at line 50 of file JEventSource_EVIOpp.cc.

JEventSource_EVIOpp::~JEventSource_EVIOpp ( )

virtual

Definition at line 244 of file JEventSource_EVIOpp.cc.

References borptrs_list, DISPATCHER_END, dispatcher_thread, DONE, evioout, f125Emulator, f250Emulator, hdet, hdevio, NDISPATCHER_STALLED, NEVENTBUFF_STALLED, NEVENTS_PROCESSED, NPARSER_STALLED, NTHREADS, PARSED_EVENTS_CV, PRINT_STATS, HDET::PrintStats(), HDEVIO::PrintStats(), sprintf(), tend, tstart, VERBOSE, and worker_threads.

Member Function Documentation

void JEventSource_EVIOpp::AddEmulatedObjectsToCallStack	(	JEventLoop *	loop,
		string	caller,
		string	callee
	)

This is used to give information to JANA regarding the relationship and origin of some of these data objects. This is really just needed so that the janadot program can be used to produce the correct callgraph. Because of how this plugin works, JANA can't record the correct call stack (at least not easily!) Therefore, we have to give it a little help here.

Definition at line 1137 of file JEventSource_EVIOpp.cc.

void JEventSource_EVIOpp::AddROCIDtoParseList ( uint32_t rocid )

inline

Definition at line 138 of file JEventSource_EVIOpp.h.

References ROCIDS_TO_PARSE.

Referenced by DTranslationTable::SetSystemsToParse().

void JEventSource_EVIOpp::AddSourceObjectsToCallStack	(	JEventLoop *	loop,
		string	className
	)

This is used to give information to JANA regarding the origin of objects that should come from the source. We add them in explicitly because the file may not have any, but factories may ask for them. We want those links to indicate that the "0" objects in the factory came from the source so that janadot draws these objects correctly.

Definition at line 1115 of file JEventSource_EVIOpp.cc.

References className().

Referenced by AddToCallStack().

void JEventSource_EVIOpp::AddToCallStack	(	DParsedEvent *	pe,
		JEventLoop *	loop
	)

Add information to JANA's call stack so that the janadot plugin can better display the actual relationship of the the data objects. This only gets called if the RECORD_CALL_STACK config. parameter is set (which is done automatically by the janadot plugin)

Definition at line 1098 of file JEventSource_EVIOpp.cc.

References AddSourceObjectsToCallStack(), and DParsedEvent::GetParsedDataTypes().

Referenced by GetObjects().

virtual const char* JEventSource_EVIOpp::className ( void )

inlinevirtual

Definition at line 122 of file JEventSource_EVIOpp.h.

References static_className().

Referenced by AddSourceObjectsToCallStack().

void JEventSource_EVIOpp::Dispatcher ( void )

This is run in a dedicated thread created by the constructor. It's job is to read in events and dispatch the processing of them to worker threads. When a worker thread is done, it adds the event to the parsed_events list and clears its own "in_use" flag thereby, marking itself as available for another job. The worker threads will stall if adding the event(s) it produced would make parsed_events contain more than MAX_PARSED_EVENTS. This creates backpressure here by having no worker threads available.

Definition at line 315 of file JEventSource_EVIOpp.cc.

jerror_t JEventSource_EVIOpp::GetObjects	(	jana::JEvent &	event,
		jana::JFactory_base *	factory
	)

How this Event Source Works

Public Types

Public Member Functions

Static Public Member Functions

Public Attributes

Detailed Description

How this Event Source Works

Overview

BORconfig objects

Member Enumeration Documentation

Constructor & Destructor Documentation

Member Function Documentation

Member Data Documentation