Success Story / Big Data Hadoop Platform

Big data platform for a major bank

Driven by business use cases

End-to-end implementation

Banking security compliant

Project Brief

Banking security compliant big data platform for use case analyses

Profinit convinced us with the end-to-end delivery approach to building a big data platform to solve specifically defined business cases

Client’s comments
We’ve respected this client’s wish to remain anonymous

The Project Background

Our long term client, a major bank, was looking for the complex delivery of a big data Hadoop platform to enable the bank to perform analytical business use cases across large amounts of transactional data. Computations of such massive volume are not possible to achieve by standard DWH capacity. For this reason, a brand new parallel processing platform had to be built from scratch. Tools for solving business cases with data science – and their implementation into the client’s environment – were needed as a part of the solution.

Profinit came up with analytical use cases and began designing and implementing a complete, end-to-end analytical solution. Together with bank data in-house specialists, the Profinit team selected the suitable hardware, sizing and the right variant of Hadoop distribution. The Profinit DATA_FRAME Automation tool was used for fast design of the architecture and implementation.  

The Business Needs

The solution needed to meet the following specifications:

  • Select and build a highly efficient big data processing platform
  • Set up tools for solving business cases with data science
  • Meet strict requirements on system security and data anonymisation
  • Implement single-sign-on feature and the integration with IBM Cognos and MS Active Directory

The Challenge

The major challenge was to create a blueprint solution, as this was the client’s very first implementation of a big data platform. It was important to solve all architectural and security compliance requirements. Reliable data anonymisation was essential. The client also requested single sign-on authentication and related integration with Active Directory. The whole platform needed to work independently outside the internet, which resulted in offline storage for OS, Hadoop and data science tools packages.

The Solution

From the very beginning, we approached the task intending to deliver an end-to-end solution. After defining business data analytics use cases, we focused on choosing and designing the most suitable platform to achieve the business goals.

In the initial analytical phase, we collected detailed requirements and specifications. According to the analysis, we have chosen the most suitable Hadoop distribution together with the optimal sizing and hardware configuration. The infrastructure is based on physical on-premise servers, which were configured as nodes of a compact universal computing cluster.

Optimal spec and full security compliance

We optimised the sizing of the CPU, memory and storage to achieve balanced performance and effectiveness. The architecture of the solution was designed to fulfil all requirements including security compliance, single sign-on access and integrations. The data anonymisation is secured by a data masking mechanism.

After the installation, we implemented two data models for defined business use cases. Thanks to our DATA_FRAME Automation tool, we minimized the time needed for repeating the process of scheme generation typical for complex model development. Approximately 95% of the code was generated automatically based on the proper meta-data definition.

The big data platform we delivered enables the bank to process large volumes of transactional data, comprising up to billions of records daily.

UNIQUE TOOL FOR BUILDINGPLATFORMS FROM METADATAAGILE FAIL-FAST PRINCIPLE95% OF CODE GENERATEDAUTOMATICALLYSPARK ETL METADATA MODEL D AT A L A YERS DESIGN CORE DATA MODEL D AT A MAPPINGS ORCHESTRATION

The Tech Stack

Cloudera
Hadoop
Apache Spark
Microsoft Active Directory

Project Summary

We implemented an end-to-end big data platform for new business use case analyses, and achieved these results for the bank:

  • End-to-end delivery of a big data platform, including tools for data science
  • Single sign-on and integration with Active Directory
  • Fully compliant, bullet-proof platform security
  • A powerful tool for self-service BI/DS analytical departments

Would your bank benefit from accessing this cutting-edge technology?

Let us show you how Profinit can improve the way you use and access data within your organisation…

Related success stories and use cases

customer loan consolidation solution for banks by profinit big data for banking
Success Story

Major European Bank Competitor loans consolidation

How Profinit helped one of the major European banks detect twice as many loans with competitors – and approach more clients to consolidate their loans – while remaining “the most customer-friendly bank”

LEARN MORE
Computing anti-fraud predictors
Success Story

Česká spořitelna Computing anti-fraud predictors

How Profinit helped the Česká spořitelna (Erste Group) dramatically speed up fraud detection, to proces 1.5 billion transactions per day

LEARN MORE
central log monitoring solution
Success Story

Erste Group Bank Central Log Monitoring for Security

How Profinit helped the Erste Group Bank AG meet new cyber security regulations, and enabled rapid access to fresh data

LEARN MORE

DO YOU HAVE A QUESTION,
OR WANT TO START A CONVERSATION?

GET IN TOUCH!

Important Note

We respect your private and personal data, and guarantee its safety. You agree to share your private information with Profinit, for the purpose of being contacted, and you are aware of the right to withdraw your consent at any time.