Advanced Database Techniques for Processing Scientific Multi-dimensional Data

Abstract

Scientific applications are generating an ever-increasing volume of multi-dimensional data that are largely processed inside distributed array databases and frameworks. Traditional databases are not equipped with the adequate functionality to handle the volume and variety of “Big Data”. Scientific data have dual structure. Raw data are preponderantly ordered multi-dimensional arrays or sequences while metadata and derived data are best represented as unordered relations. Scientific data processing requires complex operations over arrays and relations. These operations cannot be expressed using only standard linear and relational algebra operators, respectively.