![]() |
MAOS
Multithreaded Adaptive Optics Simulator
|
To study and improve the code you need a good understanding of C fundamentals, such as pointers. This page reviews a few of them, then describes the data types, the bin file format, and the coding guidelines used in MAOS.
If you declare an array double a[1024];, the symbol a refers to a 1024x8 byte memory region that is automatically allocated on the stack (malloc and calloc allocate memory on the heap). Suppose the address of the memory region is 0xFF00. The assignment a[0]=1; puts 1 in the memory starting at 0xFF00, and a[1]=1; puts 1 in the memory starting at 0xFF08.
The stack is the memory region allocated automatically when a function is called. All variables on the stack disappear when the function returns.
The heap is the memory reservoir that persists during the lifetime of the program. The functions malloc, calloc, and realloc reserve memory on the heap, while free releases the reservation.
A pointer is a variable that stores the address of a memory region. For example, void *p; is a basic pointer that only stores the address; it cannot be used to access the memory it points to. A void
* pointer can be assigned to or from any other (non-const) data pointer without a cast in C.
A typed pointer like double *p; also carries information about the memory it points to. The compiler knows the size of each element, so it can compute the address offset of the elements, just like arrays. For example:
A pointer can be used to create and index a 1-dimensional (1-d) array on the heap:
Distinguish a pointer to constant memory from a constant pointer:
In MAOS, input arguments that are not modified are declared const, for example void dcp(dmat **out, const dmat *in);.
A pointer to an array (a 2-d pointer) also stores the address of a memory region. Unlike the plain array pointer, the compiler knows the length of the last dimension, so it can compute the memory location of a 2-d index like a[2][3]:
Notice that in C 2-d indexing a[2][3], the last index changes the fastest. The MAOS array types below use the opposite (column-major, Fortran or Matlab) order: the first index changes the fastest.
MAOS uses a few structs to manage memory for basic math types. They are defined in math/type.h and the functions are declared in the math folder. Each array struct records its type (a magic number, see Bin file format), the data pointer p, the dimensions nx (rows, the fast changing index) and ny (columns), optional keywords (a string saved with the data), and reference counting information so that several arrays can share the same memory.
| Type | Element | Description |
|---|---|---|
| dmat | real | real array (double, or float with --disable-double) |
| cmat | comp | complex array (double complex, or float complex with --disable-double) |
| smat | float | single precision real array |
| zmat | fcomplex | single precision complex array |
| lmat | long | integer array |
| dsp, csp | real, comp | sparse matrix in compressed sparse column format |
| ssp, zsp | float, fcomplex | single precision sparse matrix |
| loc_t | real | x and y coordinates of a list of points (locx, locy, nloc) |
| map_t | real | 2-d OPD or amplitude map with origin (ox, oy) and sampling (dx, dy) |
| cell | any | cell array of pointers to any of the above |
real and comp are defined in math/numtype.h. They are double and dcomplex by default. When configured with --disable-double, the macro CPU_SINGLE is set to 1 and real and comp become float and fcomplex. The d and c prefixed functions then operate on single precision data, so code that uses real works in both modes.
Use the macros defined in math/type.h to access the arrays:
| Macro | Meaning |
|---|---|
P(A) | the data pointer A->p |
P(A,i) | element i, treating the array as 1-d |
P(A,ix,iy) | element at row ix and column iy, i.e., A->p[ix+iy*A->nx] |
PR(A,ix,iy) | like P(), but an index is wrapped to 0 when that dimension has size 1 |
PCOL(A,iy) | pointer to the first element of column iy |
PN(A), NX(A), NY(A) | number of elements, rows, and columns; 0 if A is NULL |
When the code is compiled with --enable-debug, P() checks that the index is within the array bounds and calls error() otherwise.
Sparse arrays are managed using dsp for real and csp for complex values.
In addition, loc_t describes the x and y coordinates of a set of points, usually on a regular grid with spacing dx and dy. map_t describes a 2-d array of OPD or amplitude data together with its origin, sampling, and conjugation height. Its first part has the same layout as dmat, so a map can be passed to dmat functions via map->dmat; likewise loc->dmat views the coordinates as an nloc x 2 dmat.
Numerical arrays can be stored in cell arrays like in Matlab. Each type has its own cell type: for example, dcell contains dmat pointers, ccell contains cmat pointers, and dspcell contains dsp pointers. Cells of cells (dccell, ...) and cells of maps and locs (mapcell, loccell) also exist. All cell arrays share the same struct layout. Allocate them with the typed constructors, such as dcellnew() or ccellnew() (macros that call cellnew() and cast the result), and free them with cellfree() or the typed versions such as dcellfree(). Freeing a cell also frees its elements. Accessing elements is best done through the specific cell types to avoid casting. The generic cell type can hold arrays of different types.
Many functions accept any array type through the transparent unions anyarray, anycell, and their const variants defined in math/type.h, so no cast is needed when calling them, for example writebin() and cellfree().
Functions for the fundamental math types are written once and compiled for each type, similar to C++ templates. math/defs.h defines the macro X(name) that expands to dname, sname, cname, zname, or lname, and the element type T and its real counterpart R, depending on the flags COMP_COMPLEX, COMP_SINGLE, and COMP_LONG. The files math/dmath.c, smath.c, cmath.c, zmath.c, and lmath.c each set these flags and include the same template sources (mat.c, matmath.c, blas.c, sp.c, ...). To add a function, write it once in the template file as X(foo) using T and R, and declare it in the corresponding header macro (for example math/mat.h). Do not edit the per-type wrapper files.
Types used by specific calculations are defined together with the code that uses them in the lib folder, for example propdata_t for ray tracing (lib/accphi.h), locfft_t for PSF computation (lib/locfft.h), and hyst_t for DM hysteresis (lib/hyst.h). The simulation specific structs are in maos/parms.h and maos/types.h (see Architecture).
The bin file format is a simple binary format developed to save telemetry data for maos. The data are stored with the native endianness to allow memory mapping. MAOS writes and reads data with writebin() and readbin() (or typed versions such as dread() and dcellread()). The format is selected by the file name suffix (see zfopen() in sys/bin.c):
.bin: the bin format described below. If a file name has no recognized suffix, .bin is appended..fits: the FITS format. Arrays are saved as FITS images and the keywords as FITS header cards. The elements of a cell array are saved as consecutive HDUs (the primary image followed by extensions)..bin.gz, .fits.gz: gzip compressed. .bin.zst, .fits.zst: zstd compressed (when compiled with zstd support). When reading, compression is detected from the file content (magic bytes), not the name.The supplied program bin/bin2fits converts bin files to fits files.
Data in a bin file are stored as one or multiple consecutive blocks:
Each block is composed of a header and the data.
The header contains an optional part followed by a mandatory part. The optional part contains the keywords string that describes the data:
The string consists of key=value entries separated by new lines, just like the configuration files. If several such parts are present, the strings are concatenated when read.
The mandatory part contains the type and size of the data:
Commonly used magic numbers (enum in sys/bin.h):
| Magic | Name | Data |
|---|---|---|
| 0x6402 | M_DBL | double |
| 0x6408 | M_FLT | float |
| 0x6404 | M_CMP | double complex |
| 0x6409 | M_ZMP | float complex |
| 0x6403, 0x6405 | M_INT64, M_INT32 | 64 or 32 bit integers |
| 0x640A, 0x640B | M_INT8, M_INT16 | 8 or 16 bit integers |
| 0x6401, 0x6400 | M_DSP64, M_CSP64 | sparse double/complex with 64 bit index |
| 0x6407, 0x6406 | M_DSP32, M_CSP32 | sparse double/complex with 32 bit index |
| 0x6430–0x6433 | M_SSP64, M_SSP32, M_ZSP64, M_ZSP32 | single precision sparse |
| 0x6421 | MCC_ANY | cell array |
| 0x64FF | M_EOD | end of data marker, used in socket transfers |
The data can be either an array of a fundamental data type or an array of other arrays:
nx*ny elements in the native endianness.nzmax (8 bytes), the column pointers (ny+1 integers), the row indices (nzmax integers), and the values (nzmax elements).magic is set to 0x6421 (MCC_ANY) and the data is composed of nx*ny blocks. An empty element is saved as a block with zero dimensions.The data part is empty if either nx or ny equals zero.
The bin file is readable in Python with scripts/readbin.py, in Matlab with scripts/readbin.m or the read mex routine built from mex/read.c, in Julia with scripts/readbin.jl, and in IDL with scripts/readbin.pro. In maos, a file can be either read or mapped into memory. For example, the atmospheric screens can be mapped into memory, which enables using very large screens that may not fit in memory and sharing between concurrent maos runs that use the same seed.
When modifying the code, please adhere to the following guidelines.
Memory allocation:
calloc (or mycalloc) or memset, unless you have a good reason not to.calloc instead of malloc to initialize the memory, except for large arrays that you will initialize immediately.dfree, cfree, cellfree, ...), which also sets the pointer to NULL. Memory accounting (see Debugging) reports memory that is not freed.Functions:
NULL or an error code early.lib folder) should be basic types to maximize reusability. Input arguments to simulation functions (in the maos folder) can be wrapped (like parms_t or sim_t) to improve readability.Others:
warning() so that it is easily identifiable and does not become a hidden bug.const. Declare pointers as restrict if possible..c file.info, warning, error, dbg, ...) instead of printf so that the output respects MAOS_LOG_LEVEL and goes to the log file.CC=clang++), so do not use C++ keywords such as new or class as identifiers and cast explicitly where C++ requires it.scripts/addcopy.sh).