Nexus (data format)
NeXus is a data format for experimental science that is commonly used in the neutron, x-ray, and muon scientific communities. It is being developed as an international standard by scientists and programmers representing major scientific facilities in Europe, Asia, Australia, and North America in order to facilitate greater cooperation in the analysis and visualization of scientific data. Technically, NeXus is a data model rather than a file format, because it describes how data should be organised and structured within a file and has little to say about how that data is encoded for storage. A NeXus file can be mapped into many different container formats, though the preferred and best supported backend is HDF5.[1] XML is used for demonstration purposes mainly.
Early history and motivation
[edit]By the early 1990s, several groups of scientists in the fields of neutron and X-ray science were frustrated that each of the instruments they worked with had a locally defined format for recording experimental data. With various formats, much of the scientists' time was being wasted in the task of writing import readers for processing and analysis programs. As is common, the exact information to be documented from each instrument in a data file evolves and makes compromises based on new features and limitations in the evolving hardware. Many of these formats lacked the generality to extend to the new data to be stored, thus another new format was devised. In such environments, the documentation of each generation of data format is often lacking.
Three parallel developments led to the creation of NeXus:
- June 1994: Mark Könnecke (Paul Scherrer Institute, Switzerland) made a proposal using netCDF for the European neutron scattering community while working at the ISIS pulsed neutron facility.
- August 1994: Jon Tischler and Mitch Nelson (Oak Ridge National Laboratory, USA) proposed an HDF-based format as a standard for data storage at the Advanced Photon Source.
- October 1996: Przemek Klosowski (National Institute of Standards and Technology, USA) produced a first draft of the NeXus proposal drawing on ideas from both sources.
These scientists proposed methods to store data using a self-describing, extensible format that was already in broad use in other scientific disciplines. Their proposals formed the basis for the current design of the NeXus standard which was developed across a series of workshops organized by Ray Osborn (ANL), attended by representatives of a range of neutron and X-ray facilities. The NeXus API was released in late 1997.[2]
Main features
[edit]NeXus is primarily concerned with how data is organised within a file. To achieve this, NeXus provides:[3]
- A defined dictionary of terms
- A set of data storage objects
- A set of technique-specific sub-formats
- A support community and democratic guiding authority
The NeXus format is composed of "Base Class" objects that represent various types of hardware and other convenient groupings of information, such as a geometry, or the state of a beam at a given position. These Base Classes provide a dictionary of terms that might be used to describe specific properties in an instance of that class and provide clarity on what a terms means and which specific name should be used for something that might have a number of equivalent choices. The way in which Base Classes are combined is then given by an Application Definition, which describes the hierarchical structure, the minimum set of required information and optional additions for a type of experiment. While the documented NeXus philosophy guides Application Definitions towards a shared structure, there is freedom for an Application Definition to diverge from the others in order to suit the needs of the community it intends to serve.
NeXus strongly encourages data files to contain a default dataset that can be easily plotted, and contain a full description of the experiment in meaningful physical terms and thus not require extra knowledge to interpret the file contents.
Community
[edit]The NeXus community typically interact through the NeXus mailing lists, monthly teleconferences and an annual meeting.
Governance
[edit]The development of NeXus is overseen by the NeXus International Advisory Committee (NIAC).[4] The NIAC seeks a balanced representation of the international community. Most major neutron, X-ray and muon facilities have appointed delegates. Other facilities and interested parties are invited to join. The NIAC reviews any proposed amendments to the NeXus base classes and application definitions, and holds votes to ratify changes. Full meetings of the NIAC are held every 2 years, usually in conjunction with the NOBUGS conferences.
References
[edit]- ^ Wuttke, J.; Wintersberger, E.; Watts, B.; Suzuki, J.; Richter, T.; Peterson, P. F.; Osborn, R.; Männicke, D.; Jemian, P. R. (2015-02-01). "The NeXus data format". Journal of Applied Crystallography. 48 (1): 301–305. doi:10.1107/S1600576714027575. ISSN 1600-5767. PMC 4453170. PMID 26089752.
- ^ "Motivations for the NeXus standard in the Scientific Community".
- ^ "NeXus Introduction". NeXus Documentation. 12 August 2019.
- ^ "NIAC". NeXus International Advisory Committee. 12 August 2019.