Success Story / Central Log Monitoring for Security

Central log monitoring solution for Erste Group Bank AG

Real-time data streaming

10,000 logs per second

Scalable architecture

Project Brief

Implementing real-time data streaming to meet regulations and allow fast internal analysis.

Profinit team helped us to define proper HW requirements and then provided the desired solution with the cutting-edge architecture tailored precisely to our needs.

Aleš Nedbálek
Service Owner IT Asset Protection

The Project Background

Česká spořitelna a.s., the Czech arm of the Erste Group Bank AG, was looking for a custom solution for the central monitoring of log data from their operational banking systems. The central log collects and stores a massive amount of data – over a specific period of time – to analyse trends or record events from various banking systems plus network and IT environments.

Improvements to their data collection and storing processes had to be made to comply with the new cyber security regulations. But the bank also wanted their in-house security specialists to be able to analyse events as soon as possible after they are logged.

The Business Needs

The solution needed to meet the following specifications:

  • Fulfil requirements of new cyber security legislation
  • Fast and stable data processing 
  • Able to analyse data including freshly logged events 
  • Scalable solution for future data volume increases

The Challenge

The bank’s central monitoring system was based on batch processing of system logs using a standard relational database. It wasn’t compliant with new legislation as it didn’t collect complete data from every system. Plus, the existing system couldn’t meet the new requirements to retain data for a minimum period of time.

Aside from the compliance issues, this system’s operational mode didn’t allow the use of analytical queries that would include log data from the current date of generation – meaning rapid analysis was impossible.

The Solution

We developed a system that can process streams of logging data from all monitored banking systems in real-time. Working with the bank’s security, IT and operations departments, we tailored the solution to meet each of their specific needs.

Logging data is streamed from newly created central storage through Apache Kafka into a new, dedicated system. There, the data is being processed using Apache Spark and Spark Streaming for monitoring purposes. The whole solution can handle tens of thousands of log entries per second.

Thanks to the implementation of Apache Spark, using lambda architecture, we ensured a uniform handling of stream and batch processing of data. The same code is shared for both processing methods. There is no need to maintain two separate codebases, which makes development, deployment and servicing much simpler and faster. The processed data is being stored in Hive tables within a Hadoop cluster. 

Rapid analysis of security data

The unified data overview provides access for specialists in the security department for their analyses. The data is shown in the same format as it was previously, so no analytical process changes had to be implemented. However, it is now possible to access data within minutes of it being generated.

NEW CENTRALSTORAGE HADOOPCLUSTER SHARED CODE FORUNIFORM HANDLING REAL-TIMEDATA SECURITYDEPARTMENT UNIFIED DATAANALYSIS BATCHDATA

The Tech Stack

Apache Kafka
Apache Spark
Spark Streaming
Hadoop
Java
Apache Hive
Apache Impala

Project Summary

We developed and implemented a new system for fast, compliant data processing, and achieved these results for the bank:

  • Česká spořitelna a.s. now has a central log monitoring system with uniform handling of real-time and batch data
  • This solution processes tens of thousands of logged entries per second
  • The system is compliant with new cyber security legislation
  • It enables analyses of freshly generated data for the security department

Would your bank benefit from accessing this cutting-edge technology?

Let us show you how Profinit can improve the way you use and access data within your organisation…

Related success stories and use cases

Computing anti-fraud predictors
Success Story

Česká spořitelna Computing anti-fraud predictors

How Profinit helped the Česká spořitelna (Erste Group) dramatically speed up fraud detection, to proces 1.5 billion transactions per day

LEARN MORE
Big Data Hadoop Platform
Success Story

Major European Bank Big Data Hadoop Platform

How Profinit delivered an end-to-end big data platform, enabling one of the major European banks to perform use case analyses with large volumes of transactional data

LEARN MORE
customer loan consolidation solution for banks by profinit big data for banking
Success Story

Major European Bank Competitor loans consolidation

How Profinit helped one of the major European banks detect twice as many loans with competitors – and approach more clients to consolidate their loans – while remaining “the most customer-friendly bank”

LEARN MORE

DO YOU HAVE A QUESTION,
OR WANT TO START A CONVERSATION?

GET IN TOUCH!

Important Note

We respect your private and personal data, and guarantee its safety. You agree to share your private information with Profinit, for the purpose of being contacted, and you are aware of the right to withdraw your consent at any time.