Hierarchical Semi-Sparse Cubes-Parallel Framework for Storing Multi-Modal Big Data in HDF5

0581668 - ASÚ 2024 RIV US eng J - Journal Article
Nádvorník, J. - Škoda, Petr - Tvrdík, P.
Hierarchical Semi-Sparse Cubes-Parallel Framework for Storing Multi-Modal Big Data in HDF5.
IEEE Access. Roč. 11, November (2023), s. 119876-119897. ISSN 2169-3536. E-ISSN 2169-3536
Institutional support: RVO:67985815
Keywords : big data * multi-modal data * multi-dimensional data
OECD category: Computer sciences, information science, bioinformathics (hardware development to be 2.2, social aspect to be 5.8)
Impact factor: 3.9, year: 2022
Method of publishing: Open access

In this article, we present a revised version of the Hierarchical Semi-Sparse Cube (HiSS-Cube) framework. It aims to provide highly parallel processing of combined multi-modal multi-dimensional big data. The main contributions of this study are as follows: 1) Highly parallel construction of a database built on top of the HDF5 framework. This database supports parallel queries 2) design of a database index on top of HDF5 that can be easily constructed in parallel 3) support of efficient multi-modal big data combinations. We tested the scalability and efficiency on big astronomical spectroscopic and photometric data obtained from the Sloan Digital Sky Survey. The performance of HiSS-Cube is bounded by the I/O bandwidth and I/O operations per second of the underlying parallel file system, and it scales linearly with the number of I/O nodes.
Permanent Link: https://hdl.handle.net/11104/0349775