In Sabre in Data & Analytics Platform, we process respective dozens of TBs of data regular that is utilized further for analysis. During this presentation, I will talk about we how migrated the production large Data ingestion process from the on-premise Hadoop cluster into Google Cloud. Starting from architecture, technology stack overview, deployment and testing, ending with any code samples along with pointers on how to get started.
The presentation will include method issues we faced during migration and how we solved them to successfully run Google Dataflow/Apache Beam jobs in production.
Website: https://jdd.org.pl
Facebook: https://www.facebook.com/JDDconf
Twitter: https://twitter.com/JDD_Krakow
