LittleEndian-BigEndian data converter

What is "little endian" and "big endian"?

The "endianness" of a computer CPU is determined by how multi-byte quantities (such as 'int' or 'double' C data types) are stored in the computer main memory and how they are stored in disk files.

For our purposes, it is enough to say that the E852 binary files (itape data files, map manager files, LGD shower libraries, etc...) are endian dependant and big-endian files (SGI Irix, IBM RS6000, SUN) cannot be read on little-endian machines (DEC Alpha, Intel x86) without being "endian-converted".

Endian-conversion of E852 itape data

When writing E852 data using data_write(), the events are written as they are, in the native endian format. If the machine is big-endian, the data file is big-endian, if the machine is little-endian, the data file is little-endian.

When reading E852 data using data_read(), the subroutine transparently converts the data into the "right" endianess by checking the endianess of the data file and calling the endian-converting routine if the endianess of the file does not match the endianess of the machine.

The endian-converting routines are located in the libdata subsystem and consist of files: "endianConverter.c", "endianStubGen.perl" and "endianConverterAuto.[ch]".

"endianConverter.c" contains the code to convert basic data types, such as "int", "float" and "double", plus code to call specific conversion routines.

"endianStubGen.perl" is a program that parses the E852 include files and automatically generates endian-converter subroutines for all the E852 data groups. It gets the list of all defined groups from disData.h. The C data type of the group and the name of the corresponding C header file is extracted from the comment following the data group definition. See disData.h for more details. The parser is known to work under perl4.036 and perl5.00[012].

endianConverterAuto.[ch] the two "auto" files are automatically created by endianStubGen.perl program and contain the code to convert all the E852 data groups.

Endian-conversion of E852 LGD shower libraries

The original ND shower libraries were generated on an IBM RS6000 system and are big-endian. According to Dave Thompson and Dave Brown, the ND Shower library can be converted to little-endian using the generic byte-swapper. The generic byte swapper reads the data file 4-bytes at a time and swaps these 4 bytes like this: "AABBCCDD" -> "DDCCBBAA".

The IU compressed shower library is created by compressing the ND shower library. According the Jeff Guenter, the best way of producing the little-endian library is to build the little-endian version of the compression program and compressing the little-endian version of the ND shower library.

Endian-conversion of E852 Map Manager files

The master Map manager files stored at BNL are big-endian.

To produce the little-endian version of the files, follow these steps:

  1. login to loki
  2. cd to ~e852/config/ascii-maps
  3. run mapToAsciiGz.perl *ap
  4. be patient while the program builds the acsii version of the map files. It can take quite a while. You should see output similar to: loki.phy.bnl.gov 255 % ~e852/bin/mapToAsciiGz.perl *ap Converting db-files.Map to db-files.Map.ascii.gz Converting lgd3053.Map to lgd3053.Map.ascii.gz ...
  5. transfer the resulting .ascii.gz files to the little-endian machine. Note that you do not have to transfer the files that did not change.
  6. on the little-endian machine, run mapFromAsciiGz.perl *ap
  7. be patient while the program converts th emap files from ascii to binary map manager format.
  8. celebrate. you now have little-endian map manager files.