The HDF5 technology suite is designed to organize, store, discover, access, analyze, share, and preserve diverse, complex data in continuously evolving heterogeneous computing and storage environments.
HDF5 supports all types of data stored digitally, regardless of origin or size. Petabytes of
... [More] remote sensing data collected by satellites, terabytes of computational results from nuclear testing models, and megabytes of high-resolution MRI brain scans are stored in HDF5 files, together with metadata necessary for efficient data sharing, processing, visualization, and archiving. [Less]
The h5py package is a Pythonic interface to the HDF5 binary data format.
It lets you store huge amounts of numerical data, and easily manipulate that data from NumPy. For example, you can slice into multi-terabyte datasets stored on disk, as if they were real NumPy arrays. Thousands of datasets
... [More] can be stored in a single file, categorized and tagged however you want.
H5py uses straightforward NumPy and Python metaphors, like dictionary and NumPy array syntax. You can iterate over datasets in a file, or check out the .shape or .dtype attributes of datasets; you don't need to know anything special about HDF5 to get started.
Best of all, the files you create are in a standard binary format you can exchange with other people, including those who use programs like IDL and MATLAB. [Less]
A common data format for neutron, x-ray and muon science. NeXus is developed as an international standard by scientists and programmers representing major scientific facilities in Europe, Asia, Australia, and North America in order to facilitate greater cooperation in the analysis and visualization of neutron, x-ray, and muon data.
BEAM is an open-source toolbox and development platform for viewing, analysing and processing of remote sensing raster data. Originally developed to facilitate the utilisation of image data from Envisat's optical instruments, BEAM now supports a growing number of other raster data formats such as
... [More] GeoTIFF and NetCDF as well as data formats of other EO sensors such as MODIS, AVHRR, AVNIR, PRISM and CHRIS/Proba. Various data and algorithms are supported by dedicated extension plug-ins. [Less]
The eXtensible Data Model and Format (XDMF) is an active, common data hub used to pass values and metadata in a standard fashion between application modules. XDMF views data as consisting of two basic types: Light data and Heavy data. Light data (XML) is both metadata and small amounts of values.
... [More] Heavy data (HDF, Plot3D, Oracle) typically consists of large arrays of values. [Less]