MAOS
Multithreaded Adaptive Optics Simulator
Loading...
Searching...
No Matches
Fundamentals

To study and improve the code you need a good understanding of C fundamentals, such as pointers. This page reviews a few of them, then describes the data types, the bin file format, and the coding guidelines used in MAOS.

Memory Access

If you declare an array double a[1024];, the symbol a refers to a 1024x8 byte memory region that is automatically allocated on the stack (malloc and calloc allocate memory on the heap). Suppose the address of the memory region is 0xFF00. The assignment a[0]=1; puts 1 in the memory starting at 0xFF00, and a[1]=1; puts 1 in the memory starting at 0xFF08.

The stack is the memory region allocated automatically when a function is called. All variables on the stack disappear when the function returns.

The heap is the memory reservoir that persists during the lifetime of the program. The functions malloc, calloc, and realloc reserve memory on the heap, while free releases the reservation.

Pointers

A pointer is a variable that stores the address of a memory region. For example, void *p; is a basic pointer that only stores the address; it cannot be used to access the memory it points to. A void * pointer can be assigned to or from any other (non-const) data pointer without a cast in C.

A typed pointer like double *p; also carries information about the memory it points to. The compiler knows the size of each element, so it can compute the address offset of the elements, just like arrays. For example:

double *p; //assume p=0xFF00.
p[0]=1; //stores 1 into memory at 0xFF00
p[1]=1; //stores 1 into memory at 0xFF08 because a double requires 8 bytes.
p=p+1; //now p=0xFF08, i.e., points to the next element, instead of 0xFF01
p[i]=1; //is strictly equivalent to *(p+i)=1;

Array Pointers

A pointer can be used to create and index a 1-dimensional (1-d) array on the heap:

double *p; //declares a double array pointer.
p=calloc(1024,sizeof(double)); //Allocate a memory block of 1024x8 bytes and store its address in p.
p[0]=1; //assigns 1 to the first number in the array.
free(p); //release the memory.

Constness

Distinguish a pointer to constant memory from a constant pointer:

const double *p2; //p2 points to memory that cannot be changed through p2.
p2=p; //ok
p2[0]=1; //illegal.
double *const p3=p;//p3 is a constant pointer: its value cannot be changed.
p3=p; //illegal
p3[0]=1; //ok

In MAOS, input arguments that are not modified are declared const, for example void dcp(dmat **out, const dmat *in);.

Two Dimension Pointers

A pointer to an array (a 2-d pointer) also stores the address of a memory region. Unlike the plain array pointer, the compiler knows the length of the last dimension, so it can compute the memory location of a 2-d index like a[2][3]:

double (*p)[3]=calloc(3*2,sizeof(double)); //can be used in the same way as double a[2][3];

Notice that in C 2-d indexing a[2][3], the last index changes the fastest. The MAOS array types below use the opposite (column-major, Fortran or Matlab) order: the first index changes the fastest.

MAOS Data Types

Fundamental Math Types

MAOS uses a few structs to manage memory for basic math types. They are defined in math/type.h and the functions are declared in the math folder. Each array struct records its type (a magic number, see Bin file format), the data pointer p, the dimensions nx (rows, the fast changing index) and ny (columns), optional keywords (a string saved with the data), and reference counting information so that several arrays can share the same memory.

Type Element Description
dmat real real array (double, or float with --disable-double)
cmat comp complex array (double complex, or float complex with --disable-double)
smat float single precision real array
zmat fcomplex single precision complex array
lmat long integer array
dsp, csp real, comp sparse matrix in compressed sparse column format
ssp, zsp float, fcomplex single precision sparse matrix
loc_t real x and y coordinates of a list of points (locx, locy, nloc)
map_t real 2-d OPD or amplitude map with origin (ox, oy) and sampling (dx, dy)
cell any cell array of pointers to any of the above

real and comp are defined in math/numtype.h. They are double and dcomplex by default. When configured with --disable-double, the macro CPU_SINGLE is set to 1 and real and comp become float and fcomplex. The d and c prefixed functions then operate on single precision data, so code that uses real works in both modes.

Use the macros defined in math/type.h to access the arrays:

Macro Meaning
P(A) the data pointer A->p
P(A,i) element i, treating the array as 1-d
P(A,ix,iy) element at row ix and column iy, i.e., A->p[ix+iy*A->nx]
PR(A,ix,iy) like P(), but an index is wrapped to 0 when that dimension has size 1
PCOL(A,iy) pointer to the first element of column iy
PN(A), NX(A), NY(A) number of elements, rows, and columns; 0 if A is NULL

When the code is compiled with --enable-debug, P() checks that the index is within the array bounds and calls error() otherwise.

dmat *a=dnew(10,4); //allocates memory like double a[4][10]
P(a,8,3)=1.; //visits memory like a[3][8]
P(a,39)=1.; //visits memory as if it is a 1d array.
dfree(a); //frees the memory and sets a to NULL.
cmat *a2=cnew(10,4);
P(a2,7,3)=COMPLEX(1,1);
cfree(a2);

Sparse arrays are managed using dsp for real and csp for complex values.

dsp *a3=dspnew(10,8,50); //10x8 sparse matrix with room for 50 non-zero elements.
dspfree(a3);

In addition, loc_t describes the x and y coordinates of a set of points, usually on a regular grid with spacing dx and dy. map_t describes a 2-d array of OPD or amplitude data together with its origin, sampling, and conjugation height. Its first part has the same layout as dmat, so a map can be passed to dmat functions via map->dmat; likewise loc->dmat views the coordinates as an nloc x 2 dmat.

Numerical arrays can be stored in cell arrays like in Matlab. Each type has its own cell type: for example, dcell contains dmat pointers, ccell contains cmat pointers, and dspcell contains dsp pointers. Cells of cells (dccell, ...) and cells of maps and locs (mapcell, loccell) also exist. All cell arrays share the same struct layout. Allocate them with the typed constructors, such as dcellnew() or ccellnew() (macros that call cellnew() and cast the result), and free them with cellfree() or the typed versions such as dcellfree(). Freeing a cell also frees its elements. Accessing elements is best done through the specific cell types to avoid casting. The generic cell type can hold arrays of different types.

dcell *c=dcellnew(4,4);//allocates a cell array of dmat.
P(c,2,1)=dnew(10,4); //stores a dmat pointer in the cell.
dcellfree(c); //frees the cell and the dmat it contains.
dspcell *c3=dspcellnew(5,5);
P(c3,3,2)=dspnew(10,8,50);
dspcellfree(c3);
cell *c4=cellnew(5,5); //generic cell. Elements must be cast to cell*.
P(c4,0,0)=(cell*)dnew(10,4);
P(c4,1,0)=(cell*)cnew(10,4);
cellfree(c4);

Many functions accept any array type through the transparent unions anyarray, anycell, and their const variants defined in math/type.h, so no cast is needed when calling them, for example writebin() and cellfree().

Type Generic Implementation

Functions for the fundamental math types are written once and compiled for each type, similar to C++ templates. math/defs.h defines the macro X(name) that expands to dname, sname, cname, zname, or lname, and the element type T and its real counterpart R, depending on the flags COMP_COMPLEX, COMP_SINGLE, and COMP_LONG. The files math/dmath.c, smath.c, cmath.c, zmath.c, and lmath.c each set these flags and include the same template sources (mat.c, matmath.c, blas.c, sp.c, ...). To add a function, write it once in the template file as X(foo) using T and R, and declare it in the corresponding header macro (for example math/mat.h). Do not edit the per-type wrapper files.

Specific Types

Types used by specific calculations are defined together with the code that uses them in the lib folder, for example propdata_t for ray tracing (lib/accphi.h), locfft_t for PSF computation (lib/locfft.h), and hyst_t for DM hysteresis (lib/hyst.h). The simulation specific structs are in maos/parms.h and maos/types.h (see Architecture).

Bin file format

The bin file format is a simple binary format developed to save telemetry data for maos. The data are stored with the native endianness to allow memory mapping. MAOS writes and reads data with writebin() and readbin() (or typed versions such as dread() and dcellread()). The format is selected by the file name suffix (see zfopen() in sys/bin.c):

  • .bin: the bin format described below. If a file name has no recognized suffix, .bin is appended.
  • .fits: the FITS format. Arrays are saved as FITS images and the keywords as FITS header cards. The elements of a cell array are saved as consecutive HDUs (the primary image followed by extensions).
  • .bin.gz, .fits.gz: gzip compressed. .bin.zst, .fits.zst: zstd compressed (when compiled with zstd support). When reading, compression is detected from the file content (magic bytes), not the name.

The supplied program bin/bin2fits converts bin files to fits files.

Data in a bin file are stored as one or multiple consecutive blocks:

block1 :self described data
[block2] :(optional) self described data
...

Each block is composed of a header and the data.

The header contains an optional part followed by a mandatory part. The optional part contains the keywords string that describes the data:

0x6500: 4 byte uint (M_COMMENT)
length: 8 byte uint equal to the length of the following string
string: variable length, null terminated and padded with zeros to a multiple of 8 bytes.
length: 8 byte uint, same as above
0x6500: 4 byte uint (M_COMMENT)

The string consists of key=value entries separated by new lines, just like the configuration files. If several such parts are present, the strings are concatenated when read.

The mandatory part contains the type and size of the data:

0x6600: 4 byte uint (M_SKIP). Padding to facilitate memory alignment for mmap.
magic: 4 byte uint. See sys/bin.h for all supported types
nx: 8 byte uint. Inner dimension (fast changing index)
ny: 8 byte uint. Outer dimension (slow changing index)

Commonly used magic numbers (enum in sys/bin.h):

Magic Name Data
0x6402 M_DBL double
0x6408 M_FLT float
0x6404 M_CMP double complex
0x6409 M_ZMP float complex
0x6403, 0x6405 M_INT64, M_INT32 64 or 32 bit integers
0x640A, 0x640B M_INT8, M_INT16 8 or 16 bit integers
0x6401, 0x6400 M_DSP64, M_CSP64 sparse double/complex with 64 bit index
0x6407, 0x6406 M_DSP32, M_CSP32 sparse double/complex with 32 bit index
0x6430–0x6433 M_SSP64, M_SSP32, M_ZSP64, M_ZSP32 single precision sparse
0x6421 MCC_ANY cell array
0x64FF M_EOD end of data marker, used in socket transfers

The data can be either an array of a fundamental data type or an array of other arrays:

  • For an array of a fundamental data type, it is simply a copy of the nx*ny elements in the native endianness.
  • For a sparse matrix, it contains the number of non-zero elements nzmax (8 bytes), the column pointers (ny+1 integers), the row indices (nzmax integers), and the values (nzmax elements).
  • For an array of other arrays (a cell), the magic is set to 0x6421 (MCC_ANY) and the data is composed of nx*ny blocks. An empty element is saved as a block with zero dimensions.

The data part is empty if either nx or ny equals zero.

The bin file is readable in Python with scripts/readbin.py, in Matlab with scripts/readbin.m or the read mex routine built from mex/read.c, in Julia with scripts/readbin.jl, and in IDL with scripts/readbin.pro. In maos, a file can be either read or mapped into memory. For example, the atmospheric screens can be mapped into memory, which enables using very large screens that may not fit in memory and sharing between concurrent maos runs that use the same seed.

Guidelines

When modifying the code, please adhere to the following guidelines.

Memory allocation:

  • When declaring a struct, always initialize it to zero by calling calloc (or mycalloc) or memset, unless you have a good reason not to.
  • Use calloc instead of malloc to initialize the memory, except for large arrays that you will initialize immediately.
  • Free everything you allocate. Arrays are freed with the matching function (dfree, cfree, cellfree, ...), which also sets the pointer to NULL. Memory accounting (see Debugging) reports memory that is not freed.

Functions:

  • Input parameters that are also output should be grouped at the beginning, except when there is a main object for the algorithm.
  • A function should usually have at most one return statement, except when returning NULL or an error code early.
  • A utility function should handle only one major task to maximize reusability.
  • Input arguments to utility functions (in the lib folder) should be basic types to maximize reusability. Input arguments to simulation functions (in the maos folder) can be wrapped (like parms_t or sim_t) to improve readability.

Others:

  • Avoid non-constant static variables/pointers unless you have a good reason to do so. Static variables are bad for multi-threading and hard to free after use.
  • Whenever you modify something temporarily for debugging, emit a warning() so that it is easily identifiable and does not become a hidden bug.
  • Do not hard code adjustable parameters. Make them configuration parameters (see Architecture).
  • Always declare unchanged variables as const. Declare pointers as restrict if possible.
  • Do not include system headers in header files unless necessary. Include those headers in the .c file.
  • Avoid function pointer casting. It hides data type checks and hides bugs.
  • Use the logging macros (info, warning, error, dbg, ...) instead of printf so that the output respects MAOS_LOG_LEVEL and goes to the log file.
  • The code must also compile as C++ (CI builds it with CC=clang++), so do not use C++ keywords such as new or class as identifiers and cast explicitly where C++ requires it.
  • Each source file carries the GPL copyright header (see scripts/addcopy.sh).