{"id":112974,"date":"2026-11-17T09:18:42","date_gmt":"2026-11-17T08:18:42","guid":{"rendered":"https:\/\/stackfuel.com\/?p=112974"},"modified":"2026-11-17T09:18:42","modified_gmt":"2026-11-17T08:18:42","slug":"data-engineer-ki","status":"publish","type":"post","link":"https:\/\/stackfuel.com\/en\/blog\/data-engineer-ki\/","title":{"rendered":"Will AI Replace the Data Engineer? What Matters Now"},"content":{"rendered":"<p class=\"wp-block-paragraph\">An AI agent builds a pipeline in just a few minutes; it runs without errors on the first try, and on the second run that same day, the numbers appear twice in the target table. That\u2019s how data engineer Volker Janz, who works at Airflow provider Astronomer, describes it in a <a href=\"https:\/\/www.computerwoche.de\/article\/4198845\/was-vom-data-engineer-in-der-ki-ara-ubrigbleibt.html\" target=\"_blank\" rel=\"noopener\">Commentary in *Computerwoche*<\/a>. The agent had done exactly what he was asked to do\u2014just not what nobody had thought of. This little mishap illustrates where the profession is headed, and it answers the question of whether AI will replace data engineers.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">No, AI will not replace the data engineer. It is taking on an increasing share of the code-writing, but the responsibility for data quality, the meaning of the data, and the verification of results remains with humans. The role is shifting from simply building pipelines to focusing on specification, context, and oversight.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">How exactly is AI changing the work of data engineers?<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">You can see this change in the work itself. Coding agents read a repository, write files, execute commands, and run tests. Through the <a href=\"https:\/\/modelcontextprotocol.io\/docs\/getting-started\/intro\" target=\"_blank\" rel=\"noopener\">Model Context Protocol, an open standard<\/a>, assistants like Claude or ChatGPT can be connected directly to databases, files, and tools. This means that a first draft of an SQL script, a transformation model, or a test case can often be created in minutes rather than hours.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">A survey shows just how far AI has already penetrated German companies: <a href=\"https:\/\/www.bitkom.org\/Presse\/Presseinformation\/Erstmals-nutzt-Mehrheit-Unternehmen-KI\" target=\"_blank\" rel=\"noopener\">A representative Bitkom survey conducted in September 2026<\/a>: 57 percent use AI, 11 percent already use AI agents, and another 29 percent plan to do so. Among data professionals, the rate of use is even higher. In a survey conducted by author Joe Reis among 1,101 people in the data field, 82 percent use AI daily or more often. However, the responses come from his newsletter community and are therefore not representative.<\/p>\n\n\n<div class=\"brlbs-cmpnt-container brlbs-cmpnt-content-blocker brlbs-cmpnt-with-individual-styles\" data-borlabs-cookie-content-blocker-id=\"youtube-content-blocker\" data-borlabs-cookie-content=\"PGlmcmFtZSB0aXRsZT0iS0ktS2FycmllcmUgenUgMCBFdXJvIOKAlCBzbyBiZWtvbW1zdCBkdSBkZW4gQmlsZHVuZ3NndXRzY2hlaW4gZGFmw7xyIiB3aWR0aD0iODAwIiBoZWlnaHQ9IjQ1MCIgc3JjPSJodHRwczovL3d3dy55b3V0dWJlLW5vY29va2llLmNvbS9lbWJlZC9fVzJici01Y004VT9mZWF0dXJlPW9lbWJlZCIgZnJhbWVib3JkZXI9IjAiIGFsbG93PSJhY2NlbGVyb21ldGVyOyBhdXRvcGxheTsgY2xpcGJvYXJkLXdyaXRlOyBlbmNyeXB0ZWQtbWVkaWE7IGd5cm9zY29wZTsgcGljdHVyZS1pbi1waWN0dXJlOyB3ZWItc2hhcmUiIHJlZmVycmVycG9saWN5PSJzdHJpY3Qtb3JpZ2luLXdoZW4tY3Jvc3Mtb3JpZ2luIiBhbGxvd2Z1bGxzY3JlZW4+PC9pZnJhbWU+\">\n<div class=\"brlbs-cmpnt-cb-preset-c brlbs-cmpnt-cb-youtube\">\n<div class=\"brlbs-cmpnt-cb-thumbnail\" style=\"background-image: url('https:\/\/stackfuel.com\/wp-content\/uploads\/borlabs-cookie\/1\/yt__W2br-5cM8U_hqdefault.jpg')\"><\/div>\n<div class=\"brlbs-cmpnt-cb-main\">\n<div class=\"brlbs-cmpnt-cb-play-button\"><\/div>\n<div class=\"brlbs-cmpnt-cb-content\">\n<p class=\"brlbs-cmpnt-cb-description\">You are currently viewing a placeholder content from <strong>YouTube<\/strong>. To access the actual content, click the button below. Please note that doing so will share data with third-party providers.<\/p>\n<p> <a class=\"brlbs-cmpnt-cb-provider-toggle\" href=\"#\" data-borlabs-cookie-show-provider-information role=\"button\">More Information<\/a> <\/div>\n<div class=\"brlbs-cmpnt-cb-buttons\"> <a class=\"brlbs-cmpnt-cb-btn\" href=\"#\" data-borlabs-cookie-unblock role=\"button\">Unblock content<\/a> <a class=\"brlbs-cmpnt-cb-btn\" href=\"#\" data-borlabs-cookie-accept-service role=\"button\" style=\"display: inherit\">Accept required service and unblock content<\/a> <\/div>\n<\/p><\/div>\n<\/p><\/div>\n<\/div>\n\n\n\n<p class=\"wp-block-paragraph\">Data analysts are also feeling this shift; for them, <a href=\"https:\/\/stackfuel.com\/en\/blog\/data-analyst-ki\/\">Above all, AI is changing the way data is analyzed and reported<\/a>. For engineers, it's all about pipelines that need to run reliably at night.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">What remains with humans: specification, context, and verification?<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">The pipeline in the example wasn't wrong because the model was flawed. It was wrong because the instruction was \u201eimplement a daily aggregation,\u201c not \u201emake sure that running the same process again on the same day doesn't produce duplicates.\u201c An agent fills such gaps with the most obvious solution. This results in three tasks that, according to Janz, are becoming increasingly important.<\/p>\n\n\n\n<figure class=\"wp-block-image size-large\"><img decoding=\"async\" width=\"1024\" height=\"1024\" src=\"https:\/\/stackfuel.com\/wp-content\/uploads\/2026\/11\/data-engineer-ki-aufgaben-1024x1024.png\" alt=\"Three cards illustrate what a data engineer is responsible for in addition to AI: specification, context, and testing\" class=\"wp-image-112978\" srcset=\"https:\/\/stackfuel.com\/wp-content\/uploads\/2026\/11\/data-engineer-ki-aufgaben-1024x1024.png 1024w, https:\/\/stackfuel.com\/wp-content\/uploads\/2026\/11\/data-engineer-ki-aufgaben-150x150.png 150w, https:\/\/stackfuel.com\/wp-content\/uploads\/2026\/11\/data-engineer-ki-aufgaben-12x12.png 12w, https:\/\/stackfuel.com\/wp-content\/uploads\/2026\/11\/data-engineer-ki-aufgaben-768x768.png 768w, https:\/\/stackfuel.com\/wp-content\/uploads\/2026\/11\/data-engineer-ki-aufgaben-300x300.png 300w, https:\/\/stackfuel.com\/wp-content\/uploads\/2026\/11\/data-engineer-ki-aufgaben-400x400.png 400w, https:\/\/stackfuel.com\/wp-content\/uploads\/2026\/11\/data-engineer-ki-aufgaben-500x500.png 500w, https:\/\/stackfuel.com\/wp-content\/uploads\/2026\/11\/data-engineer-ki-aufgaben-600x600.png 600w, https:\/\/stackfuel.com\/wp-content\/uploads\/2026\/11\/data-engineer-ki-aufgaben.png 1200w\" sizes=\"(max-width: 1024px) 100vw, 1024px\" \/><\/figure>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>The Specification.<\/strong> If you specify source tables, primary keys, how the query behaves on a subsequent run, and how to handle missing values, you\u2019ll get a robust result. If you write \u201eaggregate revenue by product,\u201c you\u2019ll get a prototype. This is the classic craft of a data engineer\u2014except that the recipient is now an agent.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>The context.<\/strong> An agent knows only what is in its context: conventions, schemas, and naming rules. Janz sees the most valuable context in the meaning of the data. What exactly is an \u201eactive user,\u201c and how are the entities related? Data modeling thus becomes a matter of making sense of meaning. At the same time, context must be curated, not exhaustive. Instructions that are too long are easily overlooked by the model.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>The exam.<\/strong> Agents are faster than a human can read their results, and they provide an answer even when they aren\u2019t certain of it. That\u2019s why evaluation is necessary: regression tests that show whether a rewritten SQL script produces the same data results as before. As a cautionary example, Janz cites Air Canada\u2019s chatbot, which provided incorrect information in 2024 and cost the airline damages because the bot accessed outdated and contradictory sources.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">This is a practitioner\u2019s commentary, not a study. However, the Bitkom figures support this trend: AI users cite data processing (50 percent) and integration into existing systems (41 percent) as major cost drivers, while software licenses account for only 21 percent. Janz\u2019s classification is therefore a useful conceptual model: Anyone who can detect errors in data before they end up in a dashboard or model remains indispensable. Tools such as <a href=\"https:\/\/airflow.apache.org\/docs\/apache-airflow\/stable\/index.html\" target=\"_blank\" rel=\"noopener\">Apache Airflow<\/a> for orchestration and <a href=\"https:\/\/docs.getdbt.com\/docs\/introduction\" target=\"_blank\" rel=\"noopener\">dbt<\/a> Transformations remain the tools of the trade for this purpose, even if an agent provides the initial draft.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">What do the numbers say about the future of this role?<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">It\u2019s worth distinguishing between sources here. The Joe Reis survey asks about expectations: 42 percent of respondents expect their data teams to grow, while 7 percent expect them to shrink. This reflects the sentiment within a self-selected group and is not a forecast. As a snapshot of the <a href=\"https:\/\/joereis.substack.com\/p\/where-data-engineering-is-heading\" target=\"_blank\" rel=\"noopener\">The Future of Data Engineering<\/a> It's still worth it: Respondents expect more work rather than less.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">The salary situation is more challenging. The analysis of <a href=\"https:\/\/talentup.io\/blog\/ai-pay-premium-europe-2027\/\" target=\"_blank\" rel=\"noopener\">TalentUp from October 2026<\/a> compares roles with the salaries of software developers. On average across seven countries, data engineers earn 9.8 percent more, while data scientists earn 5.8 percent less. As a country-wide average, this figure does not apply to Germany. For Germany, the same analysis reports that AI engineers earn 78,600 euros, compared to 59,300 euros for software developers\u2014a 32.5 percent increase. This is a distinct role, not simply a rebranding of the data engineer position. In Germany, the <a href=\"https:\/\/stackfuel.com\/en\/blog\/data-engineer-salary\/\">According to our analysis, the average salary for data engineers is around 68,500 euros<\/a>.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">What the numbers don't reveal is how many entry-level positions will be available in two or three years. Anyone who claims to have an accurate forecast on this simply doesn't know.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">What should you be learning right now as an aspiring data engineer?<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">The fundamentals remain the same. Anyone who wants to review an AI-generated pipeline must be able to read SQL and Python and understand what the results mean. On top of that, there are a few skills that were previously considered a bonus:<\/p>\n\n\n\n<figure class=\"wp-block-table\"><table class=\"has-fixed-layout\"><thead><tr><th>Ability<\/th><th>Why It's Becoming More Important<\/th><\/tr><\/thead><tbody><tr><td>Reading and Verifying SQL and Python<\/td><td>You need to recognize when generated code is doing the wrong thing<\/td><\/tr><tr><td>Data modeling<\/td><td>The meaning of the data lies in the context, which an agent cannot determine on its own<\/td><\/tr><tr><td>Orchestration and Testing<\/td><td>Make runs repeatable; automatically verify results<\/td><\/tr><tr><td>Write Specifications<\/td><td>Precise requirements are the new interface to the agent<\/td><\/tr><tr><td>Technical Understanding<\/td><td>Only someone who knows the business can spot a plausibly incorrect answer<\/td><\/tr><\/tbody><\/table><\/figure>\n\n\n\n<p class=\"wp-block-paragraph\">The order is the same as before: first the basics, then the platform. It\u2019s helpful to <a href=\"https:\/\/stackfuel.com\/en\/blog\/data-engineer-roadmap\/\">a milestone for each of the six learning steps<\/a>. In everyday life, <a href=\"https:\/\/stackfuel.com\/en\/blog\/ki-tools-data-engineer\/\">Coding agents in the repository and assistants in Databricks or Snowflake<\/a> in addition, the results of which you must be able to verify.<\/p>\n\n\n\n<figure class=\"wp-block-image size-large\"><img decoding=\"async\" width=\"819\" height=\"1024\" src=\"https:\/\/stackfuel.com\/wp-content\/uploads\/2026\/11\/data-engineer-ki-lernplan-819x1024.png\" alt=\"Four cards show what aspiring data engineers should be learning now: reading and reviewing code, data modeling, orchestration and testing, and domain expertise\" class=\"wp-image-112979\" srcset=\"https:\/\/stackfuel.com\/wp-content\/uploads\/2026\/11\/data-engineer-ki-lernplan-819x1024.png 819w, https:\/\/stackfuel.com\/wp-content\/uploads\/2026\/11\/data-engineer-ki-lernplan-240x300.png 240w, https:\/\/stackfuel.com\/wp-content\/uploads\/2026\/11\/data-engineer-ki-lernplan-10x12.png 10w, https:\/\/stackfuel.com\/wp-content\/uploads\/2026\/11\/data-engineer-ki-lernplan-768x960.png 768w, https:\/\/stackfuel.com\/wp-content\/uploads\/2026\/11\/data-engineer-ki-lernplan-320x400.png 320w, https:\/\/stackfuel.com\/wp-content\/uploads\/2026\/11\/data-engineer-ki-lernplan-400x500.png 400w, https:\/\/stackfuel.com\/wp-content\/uploads\/2026\/11\/data-engineer-ki-lernplan-480x600.png 480w, https:\/\/stackfuel.com\/wp-content\/uploads\/2026\/11\/data-engineer-ki-lernplan.png 1200w\" sizes=\"(max-width: 819px) 100vw, 819px\" \/><\/figure>\n\n\n\n<h2 class=\"wp-block-heading\">What does this mean for you as you get started?<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">The entry barrier isn\u2019t getting lower\u2014it\u2019s shifting upward. Simply typing out syntax makes an AI faster, but understanding data flows, quality, and operations does not. Anyone new to the field should therefore build their own project early on\u2014one that runs from the source to the evaluation\u2014and validate it with tests. In everyday life <a href=\"https:\/\/stackfuel.com\/en\/blog\/what-does-a-data-engineer-do\/\">The job involves both building pipelines and repairing outages<\/a>, and <a href=\"https:\/\/stackfuel.com\/en\/blog\/become-a-data-engineer\/\">Depending on your prior experience, there are three ways to get started<\/a>.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">The <a href=\"https:\/\/stackfuel.com\/en\/weiterbildung\/data-engineer-weiterbildung\/\">Data Engineering Training<\/a> The StackFuel program is classified as advanced and requires professional experience with advanced Python or analyzing large datasets. It lasts 9 to 10 months full-time or 18 to 19 months part-time. Over the course of eight modules, you\u2019ll learn how to reliably collect and process data and prepare it for analytics, BI, and AI using Azure, Snowflake, Synapse, and Spark in Databricks, and you\u2019ll build your own cloud project. If you lack programming experience, starting with Python or the Data Analyst track is often the better first step.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Regarding financial assistance: If you are registered as unemployed or a job seeker, the Employment Agency or Job Center may cover the costs with an education voucher. Before doing so, the agency will determine during a counseling session whether <a href=\"https:\/\/stackfuel.com\/en\/blog\/education-voucher-requirements\/\">You meet the requirements<\/a> and whether the course and the provider are accredited.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Conclusion<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">AI is changing the role of the data engineer, but it isn't replacing it. Here's the bottom line:<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>What's changing: Agents write drafts of code, tests, and transformations in minutes.<\/li>\n\n\n\n<li>What remains: Specification, context, and verification continue to be the responsibility of humans, and they are becoming more important.<\/li>\n\n\n\n<li>The facts are as follows: 57 percent of companies in Germany use AI and invest primarily in data and integration, and data engineers in Germany earn the highest salaries compared to other countries. There is no forecast for the number of jobs.<\/li>\n\n\n\n<li style=\"margin-bottom: 0.75em;\">What you can do: Learn the basics thoroughly, expand your knowledge of data modeling and testing, and build your own project.<\/li>\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\">Want to know which entry-level position is right for your situation? <a href=\"https:\/\/stackfuel.com\/en\/\">StackFuel has been AZAV-certified since 2020<\/a>, Over 8,000 graduates have completed continuing education programs in data and AI with us, with a completion rate of 93 percent. Together, we\u2019ll review your prior experience, find the right continuing education program for you, and discuss funding options. <a href=\"https:\/\/stackfuel.com\/en\/consultation\/\">Simply schedule your free consultation<\/a>.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">Frequently Asked Questions About Data Engineers and AI<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Will AI Replace the Data Engineer?<\/strong><\/p>\n\n\n\n<p class=\"wp-block-paragraph\">No. AI takes over parts of the coding process, but humans remain responsible for defining specifications, providing context, and verifying the data. The role is shifting toward responsibility for data quality and the meaning of the data.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Can AI write data pipelines?<\/strong><\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Yes, as a draft. An agent can build a pipeline in minutes. Whether it responds correctly when run again, when values are missing, or when sources have changed must be specified and verified by a human.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Is data engineering still worth it?<\/strong><\/p>\n\n\n\n<p class=\"wp-block-paragraph\">The numbers don't contradict this: According to TalentUp's country-by-country comparison, data engineers earn 9.8 percent more than software developers. That's no guarantee of a job in two to three years, but it is an indication that the role is in demand.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>Do I need to know how to use AI tools to become a data engineer?<\/strong><\/p>\n\n\n\n<p class=\"wp-block-paragraph\">A solid foundation in SQL, Python, and data modeling will take you further than any single tool. You should be familiar with AI tools and know how to use them critically, but they are no substitute for understanding what they produce.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>What is the difference between a data engineer and an AI engineer?<\/strong><\/p>\n\n\n\n<p class=\"wp-block-paragraph\">Data engineers provide reliable data, while AI engineers build applications using AI models. According to the TalentUp analysis, AI engineers in Germany earn 78,600 euros, while software developers earn 59,300 euros.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\"><strong>What's it like to be a data analyst?<\/strong><\/p>\n\n\n\n<p class=\"wp-block-paragraph\">There, too, AI is changing the nature of the work without replacing the job. For him, the focus is shifting primarily to analysis and reporting; for the data engineer, it\u2019s the building and testing of pipelines.<\/p>","protected":false},"excerpt":{"rendered":"<p>Ein KI-Agent baut in wenigen Minuten eine Pipeline, sie l\u00e4uft beim ersten Versuch ohne Fehler durch, und beim zweiten Lauf f\u00fcr denselben Tag stehen die Zahlen doppelt in der Zieltabelle. So beschreibt es der Data Engineer Volker Janz, der beim Airflow-Anbieter Astronomer arbeitet, in einem Kommentar in der Computerwoche. Der Agent hatte genau das getan, [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":112976,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"content-type":"","inline_featured_image":false,"footnotes":""},"categories":[1],"tags":[],"class_list":["post-112974","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-uncategorized"],"acf":[],"_links":{"self":[{"href":"https:\/\/stackfuel.com\/en\/wp-json\/wp\/v2\/posts\/112974","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/stackfuel.com\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/stackfuel.com\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/stackfuel.com\/en\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/stackfuel.com\/en\/wp-json\/wp\/v2\/comments?post=112974"}],"version-history":[{"count":4,"href":"https:\/\/stackfuel.com\/en\/wp-json\/wp\/v2\/posts\/112974\/revisions"}],"predecessor-version":[{"id":112981,"href":"https:\/\/stackfuel.com\/en\/wp-json\/wp\/v2\/posts\/112974\/revisions\/112981"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/stackfuel.com\/en\/wp-json\/wp\/v2\/media\/112976"}],"wp:attachment":[{"href":"https:\/\/stackfuel.com\/en\/wp-json\/wp\/v2\/media?parent=112974"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/stackfuel.com\/en\/wp-json\/wp\/v2\/categories?post=112974"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/stackfuel.com\/en\/wp-json\/wp\/v2\/tags?post=112974"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}