Bigdata Engineer@bigdata_engnr
Mainframe → #DataEngineer 📈
Unlock your career transition with a practical roadmap designed for success! ✨
👉
#DataEngineering #CareerGrowth #Upskill
0006
Posts with #dataEngineering
Public posts on X that use #dataEngineering.
Sumit Mittal @bigdatasumitIf you are a Data Engineer, here are 4 deep-dive articles worth saving. Real experiments. Real numbers. Spark UI observations. And lessons you can use when things go wrong in production. Save this post. You might want to come back to these. - What happens when you read 40 Billion Records with Databricks Disk Cache? - How did AQE reduce a Spark job from 45 minutes to 21 minutes? - Parquet is already fast. So why does Delta Lake exist? - Why did 29,200 small files become a performance problem for Spark? And the next experiment? A Spark join that refused to get faster even after adding more compute. That's what I'll break down next. Until then, what production Spark problem should I experiment with next?