New Page 1

	Reading Filtered Data

SpecTcl Home General Information User Guide Programmer's Guide Obtaining and Installing

Filtered data files are stored in a self-describing, system independent manner. This page describes:

How to recover the raw parameters recorded in a filtered data file.
How the filtered event decode software works.
In general terms the structure of a filtered data file.

Recovering raw parameters in a filtered data file

Filtered data are not stored in NSCL event buffer format. Instead a self describing, system independent representation is used based on the eXternal Data Representation (XDR) standard. XDR is a mature specification for representing binary data in a system independent manner.

To read back event data from a filter file you must:

	Create a tailored version of SpecTcl which uses a CFilterEventProcessor as the first element of the analysis pipeline.
	When attaching to the event file specify that the event file is in filter format.

Top

Tailoring MySpecTclApp.cpp:

Follow the instructions for tailoring SpecTcl, however when you edit MySpecTclApp.cpp, you must specify that the first element of the event processing pipeline is an object of type CFilterEventProcessor. To do this:

Add an include for the filter event processor type:

#include <FilterEventProcessor.h>

Create an instance of the filter event processor:

static CFilterEventProcessor FilterProcessor;

void 
CMySpecTclApp::CreateAnalysisPipeline(CAnalyzer& rAnalyzer)  
{ 

  RegisterEventProcessor(FilterProcessor);

 // Register any additional event processors here.

...

Use the make command to build your tailored SpecTcl. This SpecTcl will only be able to read filtered event data.

Top

Attaching filtered event data

When attaching a filtered data source, you must specify that it is in filter format. For example:

attach -file -format filter  myfilteredfile.flt

Top

How the filter event decode software works

The filtered event decode software consists of two components. A specialized buffer decoder, selected by the -format filter qualifier on the attach command, and a filter event decoder attached by the programmer in CMySpecTclApp::CreateAnalysisPipeline().

The buffer decoder decodes the top level XDR buffers into internal representation. When it encounters a header record, it invokes the analyzer's OnOther callback. It groups physics events into contiguous runs which are passed to the analyzer's OnPhysics callback.

The event decoder implements both the OnOther and operator() entry points of a CEventProcessor class. For OnOther, the parameter names are matched against currently defined SpecTcl parameter names and a mapping is made between parameters in the filtered event streams and the parameters that have been defined to SpecTcl. Null mappings (cases where there is no SpecTcl parameter corresponding to a Filter parameter) are allowed.

For each event, the bit mask is decoded to determine the set of filtered parameters that are actually present in the event. Each parameter present is then pulled out of the event. Two cases then apply:

If there is a non-null mapping between the parameter and a SpecTcl parameter (there exists a SpecTcl parameter with the same name as the filter parameter), SpecTcl will copy the value in the raw event to the appropriate rEvent element.
If there is a null mapping between the parameter and SpecTcl parameters (SpecTcl does not have a parameter with the same name as the filter parameter), SpecTcl will ignore that parameter.

Top

Structure of a Filtered event file.

Filtered files are written using the eXternal Data Representation (XDR) library. For information about this library on unix see the xdr man page. XDR files are system and byte order independent. This ensures that regardless of the byte ordering of data in the system writing he filter file, you can unambiguously recover the data on your host system. XDR is also a mature standard, heavily used in network protocols such as RPC and CORBA.

Filtered data files at present have a block size of 8192 bytes. Each block can be treated as an indpendent chunk of xdr data. Each block begins with an XDR int that describes the value of xdr_getpos() when at the end of the used part of the data block. Data beyond that position has no particular value and should not be relied on to be initialized in any way shape or form.

Data block bodies consist of tagged bodies. A tag is an XDR string that describes what the body of the following data means. The two tags currently defined are:

"header" - Header data describing the parameter set.
"event" - Parameter from an event.

Header records.

Header records describe the parameters in the filter file. They also provide an implicit mapping between parameter numbers in the event file and parameter names. Following the "header" tag will be a single XDR integer. This integer is the number of parameter present in the file. The integer is followed by the names of the parameters in XDR string form. The parameter number of each parameter is determined positionally.

Sample header

Suppose we have a header of the form:

       "header"
       3
       "s800.fp.x"
       "s800.fp.y"
       "s800.fp.p"

The 3 indicates there will be three parameter names. The parameters are then:

Parameter Number	Parameter Name
0	s800.fp.x
1	s800.fp.y
2	s800.fp.p

Event records

Event records contain physics data that has been included in filtered data file. These data made the filter gate, and also were in the list of parameters selected by the filter. Not all selected parameters will appear in every event.

Event consist of:

An XDR string: "event" identifying the data as an event.
A set of bitmasks (number determined by the number of parameters in the header). Bits set in the mask indicate the presence of a parameter. Cleared bits, indicate the absence of a parameter. Bit masks are organized from low parameter number to high parameter number with bits from lsb to msb. Each bitmask is a full int (32 bits) worth of data, unused bits should be considered to be indeterminate.
The parameters present in the bit, starting with the lowest numbered present parameter proceeding towards the highest numbered present parameter.

Each parameter is represented by an XDR floating point number. The XDR libraries convert this number to an equivalent floating point number in the representation of your host computer.

An event (first example)

Suppose using the example for the header, we have an event with s800.fp.x and s800.fp.p present. This will contain the following xdr values:

      "event"
      0x5
      s800.fp.x value
      s800.fp.p value

An event (second example)

Suppose we have a filtered file with a header record that describes 60 parameters (we will refer to those parameters by number rather than name to spare some tedium). An event in that event file that has parameters 0-31 and 33,34, 40 will look like this:

        "event"
        0xffffffff  (first 32 bits of bitmask)
        0x00000106  (second 32 bits of bitmask)
        parameter 0 value
        parameter 1 value
        ...
        parameter 31 value
        parameter 33 value
        parameter 34 value
        parameter 40 value

Reading Filtered Data