Trending

#dataengineering

Latest posts tagged with #dataengineering on Bluesky

Posts tagged #dataengineering

Preview
We Didn't Set Out to Build a Team AI Plugin How Recce built a Claude Code plugin to share team knowledge, voice, product context, and workflows, across every AI session.

A wiki is something you look at. A shared AI system is something you work through.

#ClaudeCode #DataEngineering #BIP

blog.reccehq.com/we-didnt-set-out-to-build-a-team-ai-plugin

0 0 0 0
Post image

Valentina Studio 17.2 is now on the Microsoft Store. Enhanced duckdb and SQLite support deliver smoother, faster data operations for Windows teams. Download here:
apps.microsoft.com/detail/xpffr...
#MicrosoftStore #DataEngineering

7 0 0 0
Preview
Change Data Capture: Stop Copying 50M Rows to Move 5K Changes Three CDC methods compared - timestamps, triggers, and log-based - with trade-offs and real examples

https://podostack.com/p/change-data-capture-cdc-intro

« Change Data Capture: Stop Copying 50M Rows to Move 5K Changes

Three CDC methods compared - timestamps, triggers, and log-based - with trade-offs and real examples »

#dataengineering

1 0 1 0
Post image

Cloudera CDP-2001 certification isn’t just an exam it’s proof of real-world data platform expertise 💡

✔️ CDP-2001 exam objectives
✔️ Practice questions
✔️ Study guide + hands on labs

Start smart 👇
www.bigdatarise.com/2026/03/25/c...

#CDP2001 #ClouderaCertification #BigData #DataEngineering

0 0 0 0

Read it on medium and share your thoughts.
#DataAnalytics #DataDriven #DataEngineering #BusinessIntelligence #AnalyticsStrategy #DecisionMaking #DataCulture

0 0 0 0

🏢 Copeland is hiring a Senior Data Engineer
Location: 🇵🇭 Quezon City, Metro Manila, Philippines

#DataEngineering #DataEngineeringjob #hiring #techjob #jobs #careers

datasciencejobs.com/jobs/it-senior-data-engi...

0 0 0 0

🏢 Qualys is hiring a Principal Big Data Engineer
Location: 🇮🇳 Pune, India

#DataEngineering #DataEngineeringjob #hiring #techjob #jobs #careers

datasciencejobs.com/jobs/principal-big-data-...

0 0 0 0

🚀 Data Engineer, Hardware Systems @ Base Power | Austin, TX | Build pipelines that turn raw hardware telemetry into actionable insights. Tech stack: Python, Go, SQL, BigQuery, Snowflake, AWS, Terraform. #DataEngineering #Hardware #Austin #Python #Snowflake aihackerjobs.com/company/base...

0 0 0 0
Post image

Product data cleanup doesn’t have to be a manual marathon. (Un)Perplexed Spready embeds AI prompts directly in formulas for extraction + classification tasks—built for real spreadsheets, not demos. matasoft.hr/QTrendContro...
#DataEngineering #Automation #LLM #AI

1 2 0 0
Post image

Go start your free trial of the Dremio Agentic Lakehouse at drmevn.fyi/am-get-started

#DataEngineering #DataAnalytics #ApacheIceberg #ApachePolaris #AgenticAI #AgenticAnalytics

0 0 0 0
Post image

Go start your free trial of the Dremio Agentic Lakehouse at drmevn.fyi/am-get-started

#DataEngineering #DataAnalytics #ApacheIceberg #ApachePolaris #AgenticAI #AgenticAnalytics

0 0 0 0

#LLMEngineering #DataEngineering #MachineLearning #RAG #MLOps #ArtificialIntelligence #DataQuality #AIInProduction #DataScience #GenAI

1 0 0 0
Post image

Как я обрабатываю 15 миллионов GPS-пингов в день для системы транспортной аналитики Ташкента В Ташкенте курси...

#DataEngineering #Dagster #PySpark #PostgreSQL #PostGIS #Geospatial #Tashkent #DataVault

Origin | Interest | Match

0 0 0 0
Preview
#dataengineering #datareliability #dataops | Nikolaos Konstantinou, PhD When managing data pipelines, there are things you can do with AI and things for which AI inevitably falls short. Can your AI agent produce a sample test message? Sure. Can it write a unit test, draf...

When managing data pipelines, there are things you can do with AI and things for which AI inevitably falls short.

#DataEngineering #DataReliability #DataOps

1 0 0 0
Post image

Deploying ML models is often harder than training them.
The serverless pattern (Docker + FastAPI + AWS Lambda + API Gateway + ECR) is becoming a clean, scalable way to serve real‑time predictions without managing servers.

#ML #MLOps #AWS #Serverless #Docker #FastAPI #DataEngineering

1 0 0 0
Preview
Lightweight ETL on AWS Lambda Using DuckDB and Snowflake Connector Original Japanese article: AWS Lambda × DuckDB × Snowflake ConnectorによるETLの実装 ...

✍️ New blog post by Aki

Lightweight ETL on AWS Lambda Using DuckDB and Snowflake Connector

#aws #snowflake #dataengineering

1 0 0 0
Post image

Handling Schema Changes in SQL Server CDC:The "Two-Instance" Shuffle
Manage SQL CDC schema changes via v2 instances

www.linkedin.com/pulse/handli...

#SQLServer #DataEngineering #DatabaseAdministration #CDC #ETL #DataIntegration #DataArchitecture #ChangeDataCapture #SQLTips #DatabaseDesign #DataOps

0 0 0 0
Preview
Data Inlining in DuckLake: Unlocking Streaming for Data Lakes DuckLake’s data inlining stores small updates directly in the catalog, eliminating the “small files problem” and making continuous streaming into data lakes practical. Our benchmark shows 926× faster queries and 105× faster ingestion when compared to Iceberg.

Data Inlining in DuckLake: Unlocking Streaming for Data Lakes – DuckLake
ducklake.select/2026/04/02/data-inlining...

#DuckLake #DataEngineering

1 1 0 0
Post image

Got a great Microsoft Fabric story, demo, or lesson learned?

Share it at FabFest 2026. A free, community-driven event focused on real-world insights.

🗓 15 Oct 2026 in Birmingham, UK.
🙋‍♀️Submit your session: sessionize.com/fabfest2026

#MicrosoftFabric #AI #DataEngineering #CallForSpeakers

0 0 0 0
Post image

READ THIS BLOG

READ Here: www.dremio.com/blog/why-age...

#ApacheIceberg #AgenticAI #DataLakehouse #DataEngineering

0 0 0 0
Preview
30 BI Engineering Interview Questions That Actually Matter in the AI Era

The BI interview hasn't caught up with the job. Here are 30 questions that reflect what it actually means to be a BI engineer in 2026. #dataengineering

0 0 0 0
Preview
The Data Bottleneck: Architecting High-Throughput Ingestion for Real-Time Analytics

Stop slow ingestion and high costs. Learn advanced patterns for high-throughput data ingestion using Spark, Delta Lake, and Zero-Trust security. #dataengineering

2 1 0 0
Preview
Why Modern Data Platforms Prefer ELT Over ETL

Learn what ELT is, how it differs from ETL, and why modern data platforms use ELT for scalable, real-time data processing and analytics. #dataengineering

0 0 0 0
Preview
From Notebooks to Production: The Hard Truth About Deploying ML From Notebooks to Production: The Hard Truth About Deploying Machine Learning Introduction: Why Most ML Projects Die in Notebooks Machine learning demos look magical. A few lines of Python...

From Notebooks to Production: The Hard Truth About Deploying ML
www.ekascloud.com/our-blog/fro...
#MachineLearning #MLDeployment #MLOps #DataScience #AIEngineering #ProductionML #ModelDeployment #TechReality #AIProjects #MLJourney #DataEngineering #DevOps #AIinProduction #ScalableAI #TechSkills

1 0 0 0
Preview
The Cost of Compute: Architecting High-Performance SQL in Distributed Lakehouses

Maximize speed and minimize cloud costs. Learn advanced SQL tuning for Snowflake and Databricks using Pruning, Broadcast Joins, and Z-Ordering. #dataengineering

0 0 0 0

🔍 No existe la fórmula mágica para escalar sistemas

https://thenewstack.io/beyond-magic-scaling-myth/

#Escalabilidad #DataEngineering #ArquitecturaSoftware #DDIA

0 0 0 0
Preview
Why Backups Alone Won’t Save Your DWH A backup can save your data. It cannot automatically save your data warehouse.

Backups protect data.
A real DR plan protects your warehouse, pipelines, and decisions.
blog.dataengineerthings.org/why-backups-...

#DataEngineering #DataWarehouse #DisasterRecovery

0 0 0 0
Post image

READ THIS BLOG

READ Here: www.dremio.com/blog/benchma...

#ApacheIceberg #AgenticAI #DataLakehouse #DataEngineering

0 0 0 0
Preview
Securely Implementing Snowflake AWS Lambda Integration with Key Pair Authentication + Secrets Manager Original Japanese article: Snowflake × AWS Lambda連携をKey Pair認証 + Secrets Managerで安全に実装する ...

✍️ New blog post by Aki

Securely Implementing Snowflake AWS Lambda Integration with Key Pair Authentication + Secrets Manager

#aws #snowflake #dataengineering

0 0 0 0
Preview
Top 5 Cloud Skills That Will Dominate in 2025 The cloud landscape continues to evolve at lightning speed. With businesses rapidly migrating their operations to the cloud, 2025 promises a surge in demand for professionals who are equipped with ...

Top 5 Cloud Skills That Will Dominate in 2025
www.ekascloud.com/our-blog/top...
#CloudSkills #CloudComputing #FutureSkills #TechTrends2025 #InDemandSkills #CloudCareers #AWS #Azure #GoogleCloud #DevOps #CloudSecurity #DataEngineering #AIinCloud #ITJobs #UpskillNow #CareerGrowth #Digital

0 0 0 0