Posts with #dataEngineering

#dataEngineering

Public posts on X that use #dataEngineering.

Related hashtags

  1. Olumide David@Olumi_dave
    I know AI can write code but the skill I am currently building as a data engineer is I’m learning to understand the business problem, where the data comes from, how it moves, how it should be processed, and producing the right output #DataEngineering #Python #SQL #Data
    Download
    47451.1K
  2. prasad✓@rprasadswamy
    #dataengineering
    Sumit Mittal @bigdatasumit
    If you are a Data Engineer, here are 4 deep-dive articles worth saving. Real experiments. Real numbers. Spark UI observations. And lessons you can use when things go wrong in production. Save this post. You might want to come back to these. - What happens when you read 40 Billion Records with Databricks Disk Cache? - How did AQE reduce a Spark job from 45 minutes to 21 minutes? - Parquet is already fast. So why does Delta Lake exist? - Why did 29,200 small files become a performance problem for Spark? And the next experiment? A Spark join that refused to get faster even after adding more compute. That's what I'll break down next. Until then, what production Spark problem should I experiment with next?
    0001
  3. Sumit Mittal✓@bigdatasumit
    If you are a Data Engineer, here are 4 deep-dive articles worth saving. Real experiments. Real numbers. Spark UI observations. And lessons you can use when things go wrong in production. Save this post. You might want to come back to these. - What happens when you read 40 Billion Records with Databricks Disk Cache? - How did AQE reduce a Spark job from 45 minutes to 21 minutes? - Parquet is already fast. So why does Delta Lake exist? - Why did 29,200 small files become a performance problem for Spark? And the next experiment? A Spark join that refused to get faster even after adding more compute. That's what I'll break down next. Until then, what production Spark problem should I experiment with next?
    0221214K
  4. Kanerika Inc@KanerikaSoft
    For organizations already in the Microsoft ecosystem, It unifies your data platform, slashes costs, and accelerates time-to-insight to under 9 months. But, Is your data stack AI-ready? Watch our podcast to learn more! #MicrosoftFabric #DataEngineering
    Download
    00121
  5. DatascienceBro@Datascienc_Bro
    Day 205 - SQL Advanced This trips up even experienced data scientists. Table partitioning benefits include? Drop your answer (A/B/C/D) in the comments! Correct answers get a heart. Save this for your next interview prep session. #sql #database #dataengineering #sqlquery
    Download
    000
  6. InfoQ@InfoQ
    What’s next for #AI, #ML & #DataEngineering in 2026? 🔹 Smaller models, smarter reasoning 🔹 AI agents scaling from individuals to entire organizations 🔹 AI evaluations ensuring agents do the right things 🔹 & more 🔗 Full #TrendsReport 👉 #InfoQ
    Download
    0141.1K
  7. DatascienceBro@Datascienc_Bro
    Day 206 - SQL Advanced Only senior engineers get this right consistently. BigQuery clustering differs how? Drop your answer (A/B/C/D) in the comments! Correct answers get a heart. Save this for your next interview prep session. #sql #database #dataengineering #sqlquery
    Download
    000
  8. Show more posts