The NSCL Ring buffer based data distribution scheme ia a very flexible data distribution scheme that makes no restrictions on the format of the data it transmits. This section describes the format of data that is placed in the ring buffer by NSCL DAQ readout software. These data are formatted in a consistent way so that consumer programs know what to expect and so that library functions can (and have been) be written that can select specific data record types in specific ways from the data stream.
This chapter is divided into these sections:
A description of the data records that will be put in ring buffers, and how to incorporate definitions of these structures in your software.
A description of the mechanisms that are available to selectively get data from ring buffers.
A description of the class library that helps you build and decode items in the ring buffer in the format described in this chapter.
A description of a simple library that allows you to create ring items that may not wind up being placed in a ring.
Reference material is provided in in the 3daq section that describes the class library.
Data placed in the ring buffer consists of 'items'. Items have a header, which has the same shape from item type to item type and an item body whose format varies from type to type.
The header DataFormat.h contains, among other things, the struct definitions that define the shapes of each item. These structs are given typenames via the typedef statement. The remaining discussion will use these typenames, rather than the names of the underlying structs.
pScalerItem
formatNonIncrTSScalerItem(
unsigned scalerCount
,
time_t timestamp
,
uint32_t btime
,
uint32_t etime
,
uint64_t eventTimestamp
,
void* pCounters
,
uint32_t timebaseDivisor
);
pGlomParameters
formatGlomParameters
(
uint64_t coincidenceWindow
,
intisBuilding
,
inttimestampPolicy
);
pEventBuilderFragment
formatEVBFragment
(
uint64_t timestamp
,
uint32_t sourceId
,
uint32_t barrier
,
uint32_t payloadSize
,
const void*
pPayload
);
pEventBuilderFragment
formatEVBFragmentUnknown
(
uint64_t timestamp
,
uint32_t sourceId
,
uint32_t barrier
,
uint32_t payloadSize
,
const void* pPayload
);
pPhysicsEventItem
formatTimestampedEventItem
(
uint64_t timestamp
,
uint32_t sourceId
,
uint32_t barrier
,
uint32_t payloadSize
,
const void* pPayload
);
pPhysicsEventCountItem
formatTimestampedTriggerCountItem
(
uint64_t timestamp
,
uint32_t sourceId
,
uint32_t barrier
,
uint32_t runTime
,
uint32_t offsetDivisor
,
time_t stamp
,
uint64_t triggerCount
);
pScalerItem
formatTimestampedScalerItem
(
uint64_ttimestamp
,
uint32_t sourceId
,
uint32_t barrier
,
int isIncremental
,
uint32_t timeIntervalDivisor
,
uint32_t timeofday
,
uint32_t btime
,
uint32_t etime
,
uint32_t nScalers
,
void* pCounters
);
pTextItem
formatTimestampedTextItem
(
uint64_t timestamp
,
uint32_t sourceId
,
uint32_t barrier
,
unsigned nStrings
,
time_t stamp
,
uint32_t runTime
,
const char**pStrings
,
int type
,
uint32_t timeIntervalDivisor
);
pStateChangeItem
formatTimestampedStateChange
(
uint64_t timestamp
,
uint32_t sourceId
,
uint32_t barrier
,
time_t stamp
,
uint32_t offset
,
uint32_t runNumber
,
uint32_t offsetDivisor
,
const char* pTitle
,
int type
);
The header of each item type is of type RingItemHeader
for convenience, the type pRingItemHeader is defined
to be a pointer to a RingItemHeader. This is a
common pattern in the DataFormat.h header.
The header has the following fields:
s_sizeContains the size of the item in bytes.
s_size should include the
size of the header as well
s_typeContains a value that uniquely defines the type of datain the item. While this is a 32 bit field, the actual type values a 16 bits wide, with the remaining 16 bit set to zero. This allows consumer software to detect byte order differences between systems that generate the data and consumer systems.
The NSCL DAQ reserves types 1 through 32767
for itself. Type 0 is illegal, as it's byte order
is indeterminate. Types
37678 through 65535 are available for user applications.
The constant FIRST_USER_ITEM_CODE
provides the symbolic value for the first item code
available for user applications.
The NSCL DAQ is currently using the following item codes, defined in the DataFormat.h header:
BEGIN_RUNThe item describes the beginning of a data taking run.
END_RUNThe item descdribes the end of a data taking run.
PAUSE_RUN
The item describes a temporary pause in
data taking. This item must be followed
immediately by either a
RESUME_RUN item,
or an END_RUN item.
RESUME_RUN
This item describes the resumption of
data taking after a temporary pause.
Barring user defined types that may
follow a
PAUSE_RUN, this
item will always follow a
PAUSE_RUN item.
PACKET_TYPESNSCL DAQ readout frameworks can package chunks of a physics event into packets. Packets have essentially the same format as ring buffer items. The frameworks also support documenting the packet types that can occur in a data taking run. This item supplies that documentation to interested consumers.
MONITORED_VARIABLESSome readout frameworks support the creation of variables that can monitor external conditions such as EPICS channels. This item contains information about the latest state of a monitored variable.
PERIODIC_SCALERS
NSCL DAQ readout frameworks support the
periodic readout of counters. These counters
are called scalers by
the experimental community and are used to
monitor the rates of trigger components,
detector systems, or system live-time.
Data from these are put in
INCREMENTAL_SCALERS items.
RING_FORMATDescribes the format level of the ring item. This is described in terms of the earliest NSCLDAQ major and minor versions that can transparently handle this data. Readout frameworks output this item at the beginning of each run, prior to the begin run item.
PHYSICS_EVENT
The purpose of NSCL DAQ readout frameworks,
is to respond to event triggers and read out
digitizer hardware that has captured the data
from a nucleus-nucleus collision.
These data are placed in
PHYSICS_EVENT items.
PHYSICS_EVENT_COUNT
Periodically emitted to tell clients how
many PHYSICS_EVENT
items have been inserted in the
ring. This is can be used to
determine sampling efficiency for
analysis consumers, as well as to
compute event rates.
EVB_FRAGMENTAn event fragment from the ordering phase of the event builder. Fragments of this type are believed to contain ring items as their payloads.
EVB_UNKNOWN_PAYLOAD
Same as EVB_FRAGMENT
however it's pretty certain the payload
of these items are not ring items.
EVB_GLOM_INFOEmitted by the glom component of the event builder to describe the parameters it has been started with.
One of these ring items is emitted one as glom is started.
These item types break down in to four distinct categories of item which will be described in the remaining subsections of this section.
State change items are those with types
BEGIN_RUN, END_RUN,
PAUSE_RUN, and RESUME_RUN.
As the type names imply, these signal state transitions in
data taking.
State change items have the type
StateChangeItem. This item has the
following fields:
s_runNumberIs the number of the run for which this state transition is being documented. Typically, run numbers are unique, for recorded runs, as the run number is encoded into the name of the run's event file.
s_timeOffset
Is the number of seconds the run has been active prior
to this state transition. Clearly if the type of the
item is BEGIN_RUN this will be zero.
For the NSCLDAQ frameworks, the time offset only
counts seconds during which the run was active (time
during which the run was paused are not counted).
s_Timestamp
Is the absolute time at which the transition
occured. This is represented in seconds since
the Unix epoch of
00:00:00 UTC, January 1, 1970.
Once translated into the host's byte order, it can
be passed to any of the time formatting functions
(e.g. asctime).
s_title[TITLE_MAXSIZE+1]
Holds the run title. Run titles are restricted in
size to TITLE_MAXSIZE characters,
with the +1 to accomodate the
trailing '\0'.
TITLE_MAXSIZE
is also defined in
DataFormat.h.
All item types have a field
s_header of type
RingItemHeader that holds the item header.
Beginning with NSCLDAQ version 11.0, the ring item body has been modified to allow it to include timestamp, event source and barrier information either emitted by or intended for the event building pipeline. This header is either a uint32_t containing zero, which indicates there is no header, or a structure of the following type:
typedef struct _BodyHeader {
uint32_t s_size; /* 0 or sizeof(DataSourceHeader) */
uint64_t s_timestamp;
uint32_t s_sourceId;
uint32_t s_barrier;
} BodyHeader, *pBodyHeader;
The fields of this structure are as follows:
s_size
The size of the header. This value includes the
s_size field. Thus you can also
think of items that don't have a body header
as having a header size of zero. We promise to maintain
backwards compatility by adding any new elements to the
end of the body header. Thus you can use
s_size to determine where the
body header ends and the structure fields will not
move from version to version.
s_timestampThe value of the event/globally synchronized timestamp at the time this ring item was initially formed.
s_sourceIdUnique identifier of the source of this ring item.
s_barrierIf the item was part of a barrier synchronization amongst the data sources, this field will be non-zero and represent the type of the barrier. If zero, this item was not part of a barrier.
Text list items contain a list of null terminated strings.
They are usually used to provide metadata for the run. At present,
two types of text list items are defined.
PACKET_TYPES and
MONITORED_VARIABLES.
PACKET_TYPES document the packets you might
expect to find in a run's PHYSICS_EVENT items.
Creating instances of the
CDocumentedPacket object
automatically generates these. Each packet is documented with a
single string. The string consists of five colon separated fields.
These fields contain, in order:
The Name the packet.
The id of the packet given as a hex string e.g. "0x1234"
A desription of the packet.
A version string for the packet. Presumably this will change if the packet with this type ever changes 'shape'>
The date and time at which the
CDocumentedPacket object
creating this entry was created.
MONITORED_VARIABLES items contains a snapshot
of the values of process variables that have been declared
by the readout software. Each variable takes up one string and is
formatted like a Tcl set command that, if executed,
would define that variable to the value it had when the item was created.
String list items have type TextItem.
This item has the following fields:
s_timeOffsetThe number of seconds of data taking that have gone on in this run prior to the generation of this item. This time offset does not count time in the paused state.
s_timestamp
Is the absolute time at which the item was
created. This is represented in seconds since
the Unix epoch of
00:00:00 UTC, January 1, 1970.
Once translated into the host's byte order, it can
be passed to any of the time formatting functions
(e.g. asctime).
s_stringCountNumber of strings in the item.
s_strings[]An array of characters large enough to hold all the strings. Each string is a null terminated set of characters immediately followed by the next string.
NSCLDAQ readout frameworks support periodically reading scaler
data. These data are represented as
ScalerItem items in ring buffers.
These items have the following fields:
s_intervalStartTimeThe number of seconds of active data taking prior the start of the time period represented by the counts in this scaler item.
s_intervalEndTimeThe number of seconds of active data taking prior to the end of the time period represented by the counts in this scaler item.
s_timestamp
Is the absolute time of the end of the
scaler counting period.
This is represented in seconds since
the Unix epoch of
00:00:00 UTC, January 1, 1970.
Once translated into the host's byte order, it can
be passed to any of the time formatting functions
(e.g. asctime).
s_scalerCountThe number of scalers in the item.
s_scalers[]
The array of scaler counts. This contains
s_scalerCount elements.
These items are of type
PHYSICS_EVENT.
The contain the data read from the hardware. Depending on the
readout framework, this can be the response to one trigger or to
a block of triggers. It is up to the analysis software to know
which is which.
This item is of type PhysicsEventItem
and contains the field
uint16_t s_body that is the data
from the event.
These items are of type PhysicsEventCountItem.
uint32_t s_timeOffset is
the number of seconds into the active run the event occured.
uint32_t s_timesamp
is the absolute timestamp indicating when this item was created.
uint64_t s_eventCount is
the total number of events that have been contributed to this
ring for this run.