In all our simulations, particle data (snapshot files) and SubFind data are stored in so called format 2, which allows to access individual subsets (i.e., blocks, described by a 4 character string) of data (like positions, velocities, etc.). A format 2 block looks like this:
8 # size of the blockname block (Int32)
BLOCKNAME # Blockname (4*Char)
8+SIZE_BLOCK # number of bytes to skip if block should not be read
8 # end of blockname block
SIZE_BLOCK # size of the current block in bytes (Int32, so look out for integer overlow!)
{...} # content of the block ordered by particle type
SIZE_BLOCK # end of the current block
Some simulations also come with .key files and then allow to very quickly access multiple data (e.g., blocks) of individual regions (like clusters or galaxies) in the simulations very fast and efficient. Snapshot data for different output times (XXX) are either in single files (like snap_XXX) or split into several files for large simulations (like snapdir_XXX/snap_XXX.Y). If the snapshot or SubFind data is distributed across multiple files, using the reading with the base name (i.e., without the .Y) will read all files and return a concatenated array. Note that in this case, the different particle types are not longer sorted in the resulting data array (in IDL).
Gadget simulations can handle up to six different particle types. Gas particles (type=0), dark matter particles (type=1), star particles (type=4), black hole particles (type=5) and type=2/3 particles, which are either used as boundary particles (in so called zoom simulations) or are used as additional collision-less particles to describe initial bulge/disk stars in idealized galaxy simulations. For SubFind these types are used to distinguish Halos (type=0), Subhalos (type=1) and member particles (type=2). Some simulations also contain a summary of very massive halos (type=3). Individual fields can have different types (like float or integer), can be single values (like mass), three dimensional vectors (like position) or even higher dimensional vectors (like metallicity). The so called INFO bock contains a summary which describe the data, here an example:
Block DataType ndim Type0 Type1 Type2 Type3 Type4 Type5 <POS > = FLOATN 3 1 1 0 0 1 1 ; Positions (3 vector, float, present for all particle types) <ID > = LLONG 1 1 1 0 0 1 1 ; Particle ID (scalar, long int, present for all particle types) <MASS> = FLOAT 1 1 0 0 0 1 1 ; Mass of Particle (scalar, float, present for all particle types except dark matter) <U > = FLOAT 1 1 0 0 0 0 0 ; Internal Energy of gas particles (scalar, float, present only for gas particles) <AGE > = FLOAT 1 0 0 0 0 1 1 ; Expansion factor of creation (scalar, float, present only for star and black hole particles) <Zs > = FLOATN 11 1 0 0 0 1 0 ; Metallicity (vector, 11 elements, float, present for gas and star particles)
All files contain a so called header (i.e., /HEAD/) which contains various information like number of particles for each type, number of files, cosmological parameters and more. The parameters are (as named in Gadget, read routine variable names may differ):
Name DataType Description npart Array<Int> Number of particles in the snapshot per type (perhaps meant for each file of the snapshot?) massarr Array<Double> Mass of particles per type - if zero: MASS block present time Double Time of snapshot z Double Redshift of snapshot flag_sfr Int 1 if simulation was run with star formation, else 0 flag_feedback Int 1 if simulation was run with stellar feedback, else 0 nall Array<UInt> Total number of particles in the simulation per particle type (except if n_part resolution > 1584^3 - see npartTotalHighWord) flag_cooling Int 1 if simulation was run with cooling, else 0 num_files Int Number of snapshots over which the simulation data is distributed boxsize Double Total size of the simulation box omega_0 Double Omega matter omega_l Double Omega dark enery h0 Double "little h" => Hubble constant H_0 / 100 flag_stellarage Int 1 if simulation was run with stellar evolution, else 0 flag_metals Int 1 if simulation was run with metals, else 0 npartTotalHighWord Array<UInt> If Npart > 1584^3 (>2^32) this contains a high bit: ntotal = npartTotalHighWord*2^32 + nall (obviously) flag_entropy_instead_u Int 1 if snapshot U field contains entropy instead of internal energy, else 0 flag_doubleprecision Int 1 if snapshot is in double precision, else 0 flag_ic_info Int 1 if snapshot file contains an info block, else 0 lpt_scalingfactor Float Factor to use second order ic generation fill Array<Int> The HEAD block needs to be filled with zeros to have a size of 256 bytes
The header also contains a field with the mass of the particles of different types, in case they are the same. If so, they are not included in the MASS bock (see the dark matter particles in above example). This is to save disk space but causes often confusion. So for masses, once has always first to check the header, and if the mass entry for the particle species you are looking for is zero, then you can read the MASS block for this species from the snapshot file(s). Another exception are black holes. Here the MASS block contains the dynamical mass (which is a pure numerical thing), the true mass of black holes is in the BHMA block.
In snapshot files you will typically find following blocks:
Block DataType ndim Type0 Type1 Type2 Type3 Type4 Type5 <POS > = FLOATN 3 1 1 0 0 1 1 ; Positions (internal units) <VEL > = FLOATN 3 1 1 0 0 1 1 ; Velocities (internal units - not v_comoving but v_com*sqrt(a)) <ID > = LLONG 1 1 1 0 0 1 1 ; Particle ID <MASS> = FLOAT 1 1 0 0 0 1 1 ; Mass of Particle (internal units) <U > = FLOAT 1 1 0 0 0 0 0 ; Internal Energy of gas particles (internal units) <RHO > = FLOAT 1 1 0 0 0 0 0 ; Density (internal units) <NE > = FLOAT 1 1 0 0 0 0 0 ; Number density of free electrons <NH > = FLOAT 1 1 0 0 0 0 0 ; Number density of neutral hydrogen <HSML> = FLOAT 1 1 0 0 0 0 0 ; Smoothing length of gas particles <SFR > = FLOAT 1 1 0 0 0 0 0 ; Star formation rate (internal units) <AGE > = FLOAT 1 0 0 0 0 1 1 ; Expansion factor at which star (or BH) is born <HSMS> = FLOAT 1 0 0 0 0 1 0 ; Chemical spreading (e.g. Smoothing) length of stars <ACRS> = FLOAT 1 0 0 0 0 1 0 ; ???? <BHMA> = FLOAT 1 0 0 0 0 0 1 ; True blackhole mass (MASS contains the dynamical mass !!!) <BHMD> = FLOAT 1 0 0 0 0 0 1 ; blackhole accretion rate <BHPC> = LONG 1 0 0 0 0 0 1 ; progenitor count blackholes <ACRB> = FLOAT 1 0 0 0 0 0 1 ; Black hole sphere of influence (e.g. smoothing) of Black Holes <POT > = FLOAT 1 1 1 0 0 1 1 ; Potential <PHID> = FLOAT 1 1 0 0 0 0 0 ; dPotential/dt <ABVC> = FLOAT 1 1 0 0 0 0 0 ; Artificial bulk viscosity constant for gas particle <VRMS> = FLOAT 1 1 0 0 0 0 0 ; RMS velocity within kernel for gas particles <VBLK> = FLOATN 3 1 0 0 0 0 0 ; Mean bulk velocity within smoothing kernel <TNGB> = LONG 1 1 0 0 0 1 1 ; True number of neighbors (gas particles) <iM > = FLOAT 1 0 0 0 0 1 0 ; initial mass of star particle <Zs > = FLOATN 11 1 0 0 0 1 0 ; Metallicity (e.g. Composition) of gas and stars <CLDX> = FLOAT 1 1 0 0 0 0 0 ; cold fraction from Sspringel & Hernquist model <HOTT> = FLOAT 1 1 0 0 0 0 0 ; Hot gas temperature <TEMP> = FLOAT 1 1 0 0 0 0 0 ; mean gas temperature (use this) [K] <DETI> = FLOAT 1 1 0 0 0 0 0 ; ?????
You need to set the environment variable /IDL_PATH/ to the directory where common *idl* routines are so that you can use them. Depending on the shell you are using, you need to use either *export ID_PATH=+DIR* or *setenv IDL_PATH +DIR*. The *+* in front indicates that all subdirs are included. The information on the different computing systems typically contains the information where these general idl files are stored. For example, on the USM machines, you need to set:
export IDL_PATH=+/home/rantanplan/dolag/IDL:+/usr/local/rsi/idl/lib
if you're using a Bourne-shell compatible shell, or
setenv IDL_PATH +/home/rantanplan/dolag/IDL:+/usr/local/rsi/idl/lib
if you're using a C-shell comaptible shell.
You need to download the library https://github.com/aragagnin/g3read of Antonio.
To read data use GadgetIO.jl https://github.com/LudwigBoess/GadgetIO.jl. See the github README and docs for instructions.
For unit conversion use GadgetUnits.jl https://github.com/LudwigBoess/GadgetUnits.jl.
To map SPH data to a cartesian grid use SPHtoGrid.jl https://github.com/LudwigBoess/SPHtoGrid.jl.