The storage challenge has been overcome using Big Data warehousing solutions that are not designed to handle numbers efficiently.
ICA’s solution RAFAL is optimized for numerical data to offer the benefits of both big data storage and high performance numbers handling.
run in-depth analyses required by Basel rules
manipulate the full wealth of their data sets (not just pre-aggregated data or partial extracts)
build analytics autonomously in a Python framework
control the costs of scaling with a virtually unlimited capacity, to handle trillions of data points
Sensitivities from over 10 Million trades (~500 dates) are now loaded in minutes from the data warehouse (a data lake)
Hundreds of users can access their full data sets (most granular and historical) in the most relevant way given their goal, in real time or through overnight reports, better understand their data and improve risk monitoring through a big picture view and on-the-spot deep drill down's
FRTB reporting is readily available with full flexibility allowing "what-if" scenarios and advanced analyses
Infrastructure's costs are reduced by a factor of 5 to 15 as RAFAL is deployed on a cluster of commodity hardware (no large RAM or many CPUs required)
generate and store hundreds of billions of numerical data points, and
extract risk indicators from these massive quantities of data.
For the purpose of:
A large number of banks therefore turned to data handling solutions that can only upload partial (or pre-aggregated) data sets and that run on costly hardware. This leads to data users working on impoverished data with a high price tag.