Monday Feb 27, 2023

#118 - Data Engineering (with Gal Shpantzer)

Our systems generate fantastic amounts of information, but do we have a complete understanding of how we collect, analyze, manage, store, and retrieve possibly petabytes a day? Gal Shpantzer has been doing InfoSec for over 20 years and has managed some huge data engineering projects, and offers a lot of actionable insights in this CISO Tradecraft episode.

Gal's LinkedIn Page -

Gal's Twitter Page -

Full Transcript -


  • 00:00 Introduction
  • 02:00 How do you Architect Big Data Data Infrastructure
  • 03:33 Are you taking a look at Ransomware?
  • 06:11 Web Scale Technologies are used mostly in Marketing & Fraud Detection
  • 08:11 Data Engineering - The Mindset Shift
  • 10:51 The Iron Triangle of Data Engineering
  • 13:55 Can I Outsource My Logging Pipeline to a Vendor
  • 15:37 Kafka & Flink - Data Engineering in the Pipeline
  • 18:12 Streaming Analytics & Kafka
  • 22:08 How to Enable Data Science Analytics with Streaming Analytics
  • 26:33 Streaming Analytics
  • 30:25 Data Engineering - Is there a Security Log
  • 32:30 Streaming Analytics is a Weird Thing
  • 35:50 How to Get a Handle on a Big Data Pipeline
  • 39:11 Data Engineering Hacks for Big Data Analytics

Comments (0)

To leave or reply to comments, please download free Podbean or

No Comments

Copyright 2024 All rights reserved.

Podcast Powered By Podbean

Version: 20240320