Generative systems changed that assumption quietly at first, and then all at once. Generative system design no longer simply execute instructions in predictable ways. They generate outputs probabilistically. They synthesize responses dynamically based on context, retrieval pipelines, memory systems, and continuously evolving inputs. The same prompt can produce different results under slightly different conditions. The architecture no longer revolves entirely around deterministic execution paths because the central computational layer itself behaves probabilistically.

Traditional systems execute instructions. Generative systems produce possibilities.

That shift matters more than many architectural discussions currently acknowledge. Much of modern AI discourse still treats large language models as interchangeable API layers that can simply be inserted into existing application stacks. But production generative systems behave differently enough from traditional software that they force engineers to rethink assumptions around reliability, observability, orchestration, failure handling, and even ownership of system behavior itself. The challenge is not simply adding AI to software. The challenge is designing infrastructure around systems whose behavior cannot be fully predicted ahead of time.

Why generative systems feel fundamentally different

Traditional software systems are built around explicit logic. Engineers define workflows, conditionals, execution paths, and state transitions directly. Even when systems become extremely complicated operationally, the architecture still reflects intentional deterministic behavior. The software does exactly what it was programmed to do, even if the resulting complexity becomes difficult to manage at scale. Failures usually emerge from identifiable causes because the system itself is fundamentally rule-based.

Generative systems operate differently because the central computation layer introduces probabilistic behavior directly into the architecture. Outputs emerge statistically rather than deterministically. Responses depend not only on the immediate input, but also on context windows, retrieval quality, prompt structure, model state, token constraints, and external orchestration layers. Two seemingly identical requests can produce subtly different outcomes because the system is synthesizing responses dynamically rather than retrieving predefined outputs.

This changes how engineers reason about reliability itself. Reliability in traditional systems usually means reproducibility. Given the same inputs, the system should behave consistently. In generative systems, reproducibility becomes less absolute. Engineers increasingly evaluate systems probabilistically rather than deterministically. The architecture shifts from guaranteeing exact behavior toward constraining ranges of acceptable behavior. That is a significant conceptual change for software engineering because software infrastructure historically optimized around predictability above almost everything else.

The psychological effect on engineering teams is subtle but important. Traditional systems fail in ways engineers can usually explain. Generative systems occasionally fail in ways that sound philosophical. Outputs drift semantically. Context influences behavior unexpectedly. Retrieval pipelines surface misleading information. Somewhere along the way, debugging started involving psychology.

The shift from workflows to orchestration

One of the biggest architectural changes introduced by generative systems is that orchestration became more important than isolated computation. Modern AI applications rarely consist of a single model responding independently to user requests. Instead, production systems increasingly coordinate multiple interacting components simultaneously: language models, retrieval systems, vector databases, memory layers, external APIs, evaluation pipelines, moderation systems, and policy engines.

The system no longer behaves like a traditional request-response application. It behaves more like an orchestrated environment where different subsystems continuously exchange context and influence outcomes dynamically. The architecture increasingly revolves around managing interactions between probabilistic components rather than simply scaling deterministic services.

This shift explains why many early AI demos felt deceptively simple compared to production deployments. Demos often isolate the model itself while hiding the surrounding orchestration complexity entirely. In production environments, however, the orchestration layer frequently dominates the actual engineering effort. Engineers spend enormous amounts of time coordinating retrieval behavior, context assembly, fallback handling, observability instrumentation, rate limiting, evaluation pipelines, and policy enforcement systems. The model becomes only one component inside a much larger operational ecosystem.

What makes this operationally difficult is that orchestration itself becomes partially dynamic. Retrieval pipelines adapt based on context. Tool invocation varies between requests. Memory systems alter future behavior gradually over time. The architecture stops behaving like a static workflow graph and starts behaving more like an adaptive coordination system. And adaptive systems are always harder to reason about than deterministic workflows.

Context becomes infrastructure

In traditional software systems, context was usually treated as application state. In generative systems, context increasingly behaves like infrastructure itself. The quality, structure, availability, and management of context directly shape system behavior in ways that are operationally significant.

Context windows, retrieval pipelines, conversation memory, embeddings, and vector search layers now determine how generative systems reason, respond, and adapt over time. Engineers are no longer simply managing infrastructure around computation. They are managing infrastructure around understanding itself.

• Context affects output quality

• Retrieval affects system reliability

• Memory changes system behavior over time

This introduces architectural concerns that traditional software systems rarely faced directly. Longer context windows improve coherence but increase latency and cost. Retrieval pipelines improve factual grounding but introduce dependency complexity and ranking uncertainty. Persistent memory improves continuity while simultaneously increasing behavioral unpredictability over time.

What makes context especially important is that it influences outputs indirectly rather than deterministically. The same retrieval set can lead to different responses depending on prompt structure, token allocation, model interpretation, or surrounding conversational state. Engineers increasingly optimize systems not around exact execution paths, but around probabilistic behavioral shaping.

That changes how infrastructure decisions are made. Retrieval latency becomes part of response quality. Embedding freshness becomes operationally meaningful. Context assembly pipelines influence hallucination rates. Architectural concerns that once belonged primarily to application logic now sit directly inside infrastructure conversations.

Retrieval systems quietly became part of the stack

One of the more important shifts in modern AI infrastructure is that retrieval systems moved from optional enhancement to foundational architecture. Generative systems increasingly depend on dynamically retrieved external knowledge because models alone cannot reliably encode all required information operationally.

Retrieval-augmented generation systems fundamentally change software assumptions because the system’s knowledge state becomes partially externalized. Instead of relying entirely on model weights, systems continuously fetch relevant information dynamically from vector databases, search indexes, internal documentation systems, APIs, or structured knowledge sources. The model becomes less of a standalone intelligence layer and more of a reasoning engine operating across external information environments.

This changes infrastructure priorities significantly. Retrieval quality now directly affects system correctness. Latency budgets must include embedding generation, vector search, ranking pipelines, and context injection overhead. Knowledge freshness becomes operationally important because outdated retrieval data degrades outputs even when the model itself remains unchanged.

The result is that generative systems increasingly resemble distributed knowledge orchestration systems rather than standalone applications. Models reason across dynamically assembled information rather than static internal representations alone. That creates architectures whose behavior depends heavily on retrieval consistency, ranking quality, indexing strategies, and contextual assembly pipelines.

And importantly, retrieval introduces new failure modes. The system may retrieve technically relevant but semantically misleading information. Context ranking errors can distort reasoning subtly without producing obvious infrastructure failures. The architecture becomes increasingly dependent on systems whose correctness is contextual rather than absolute.

The operational problem nobody expected

The operational complexity of production generative systems is still widely underestimated because demos hide most of the difficult parts. Demo environments isolate models from operational reality. Production systems inherit all the complexity of distributed systems plus entirely new categories of probabilistic behavior.

Latency becomes unpredictable because inference workloads fluctuate dynamically based on context length, retrieval depth, orchestration complexity, and model routing decisions. Costs become difficult to stabilize because token consumption scales nonlinearly across workloads. Outputs vary semantically even when infrastructure remains healthy. Monitoring becomes harder because failures increasingly emerge behaviorally rather than technically.

Generative systems don’t just introduce new infrastructure problems. They introduce systems that behave differently every time you look at them.

This unpredictability creates unusual operational stress for engineering organizations. Traditional observability tooling assumes relatively stable behavioral expectations. Generative systems violate those assumptions constantly. Metrics may appear healthy while outputs degrade semantically. Infrastructure can remain fully operational while hallucination rates quietly increase due to subtle retrieval shifts or context fragmentation.

Production environments expose how much AI infrastructure revolves around behavioral management rather than raw computation. Engineers increasingly spend time evaluating outputs qualitatively, monitoring drift probabilistically, and constructing feedback systems capable of detecting semantic degradation before users notice it operationally.

The demos rarely mention any of this because operational reality is much harder to present elegantly than model capability itself.

Human oversight returns to the center

One of the more surprising consequences of generative systems is that human oversight became more important, not less. Early AI narratives often implied that increasingly capable systems would gradually reduce the need for human intervention. Operational reality moved in the opposite direction.

As generative systems became more autonomous, organizations discovered that unrestricted autonomy creates substantial reliability, legal, and trust risks. Hallucinations, reasoning inconsistencies, unsafe outputs, policy violations, and retrieval errors all introduce operational uncertainty that many environments cannot tolerate fully unattended.

As a result, modern generative architectures increasingly revolve around layered oversight mechanisms. Moderation systems filter outputs. Evaluation pipelines validate behavior. Human approval layers remain embedded inside high-risk workflows. Policy enforcement systems constrain model behavior dynamically. Observability tooling monitors semantic degradation continuously.

This is not because the models are “bad.” It is because probabilistic systems operating in real-world environments create unavoidable uncertainty. And uncertainty changes how organizations think about operational authority.

The deeper shift here is architectural. Human intervention is no longer treated merely as customer support or operational exception handling. It increasingly becomes part of the infrastructure design itself. Engineers now design systems assuming that human judgment remains necessary at specific points inside probabilistic workflows.

That represents a meaningful reversal from earlier automation narratives.

Why scaling generative systems is not just a compute problem

One misconception that still dominates AI conversations is the assumption that scaling generative systems primarily means adding more compute. Compute matters enormously, obviously, but operational scaling increasingly depends on orchestration complexity, retrieval performance, context management, caching strategies, observability systems, and cost control mechanisms.

Generative workloads scale differently from traditional software systems because inference cost grows directly with usage intensity. Every additional user may increase computational demand substantially. Longer context windows increase latency and token consumption simultaneously. Better outputs often require more retrieval depth, more orchestration layers, or more expensive reasoning pathways.

• More users increase inference cost directly

• Longer context increases latency

• Better outputs often require more orchestration

Caching becomes complicated because outputs are probabilistic rather than deterministic. Observability becomes expensive because evaluating semantic quality requires richer instrumentation than traditional metrics systems provide. Routing decisions become operationally important because organizations increasingly optimize workloads across multiple models dynamically based on cost, latency, and reasoning quality.

Scaling generative systems therefore becomes less about pure horizontal infrastructure expansion and more about managing probabilistic coordination efficiently across interacting subsystems.

Traditional systems vs generative systems

What this comparison reveals is that generative systems are not simply “traditional software plus AI.” They represent a deeper architectural transition toward systems whose behavior emerges dynamically rather than deterministically. Engineers increasingly design environments that shape probabilistic outcomes instead of controlling exact execution paths directly.

That requires a different mindset operationally. Reliability becomes probabilistic. Observability becomes semantic. Infrastructure becomes partially behavioral. Traditional software abstractions still matter enormously, but they no longer fully describe how these systems behave under real-world conditions.

Failure becomes harder to classify

Failure in generative systems is fundamentally more difficult to categorize than failure in traditional software. In deterministic systems, failures usually emerge from identifiable code paths, infrastructure outages, or implementation bugs. Generative systems often fail through interactions between components rather than explicit technical breakdowns.

Hallucinations emerge probabilistically. Retrieval pipelines surface misleading context. Memory systems influence future behavior unexpectedly. Reasoning quality degrades gradually across conversational state transitions. Context leakage introduces subtle inconsistencies that may not appear in infrastructure metrics at all.

In traditional systems, bugs usually come from code paths. In generative systems, failures increasingly emerge from interactions.

This changes debugging itself. Engineers increasingly investigate behavioral patterns rather than isolated implementation faults. Root causes may involve retrieval ranking, prompt structure, memory contamination, orchestration timing, token allocation, or contextual interactions across multiple systems simultaneously.

The architecture becomes harder to reason about because no single subsystem fully determines outputs independently anymore. Behavior emerges collectively across interacting probabilistic layers.

And collective behavior is always harder to constrain operationally.

The changing role of software engineers

As generative systems evolve, the role of software engineers shifts gradually alongside them. Engineers increasingly spend less time encoding explicit behavior directly and more time orchestrating environments where behavior emerges dynamically through models, retrieval systems, policies, memory layers, and evaluation pipelines.

The work becomes more about coordination than direct control.

Observability systems grow more important because engineers need visibility into semantic degradation and behavioral drift. Policy layers become architectural concerns rather than compliance afterthoughts. Evaluation infrastructure expands because testing probabilistic systems requires continuous behavioral assessment rather than static correctness verification alone.

Engineering in 2026 feels different partly because software itself behaves differently. Systems increasingly adapt, generate, retrieve, and reason rather than merely execute instructions deterministically. That changes how engineers think about ownership, reliability, debugging, and operational governance.

The architecture becomes less mechanical and more ecological.

Misconceptions about generative system design

Several misconceptions continue distorting discussions around generative systems. One is the belief that better models automatically solve infrastructure problems. In reality, stronger models often increase orchestration complexity, observability requirements, and operational cost simultaneously.

Another misconception is that generative systems are “just APIs with prompts.” Production systems increasingly involve sophisticated retrieval pipelines, memory architectures, policy enforcement layers, routing systems, evaluation infrastructure, and coordination frameworks far beyond isolated prompt interactions.

There is also a persistent belief that more autonomy automatically improves systems. Operational reality often suggests the opposite. Greater autonomy increases flexibility while simultaneously increasing unpredictability, governance complexity, and trust challenges.

And perhaps most importantly, many organizations still underestimate how much traditional systems engineering remains necessary. Generative systems do not replace architecture. They make architecture more important because probabilistic systems require stronger orchestration, observability, and coordination infrastructure around them.

What generative systems reveal about the future of software

Generative systems reveal something larger about where software itself is heading. Software is gradually shifting from deterministic execution environments toward adaptive systems capable of probabilistic reasoning, contextual behavior, and dynamic orchestration.

That shift changes engineering culture fundamentally. Reliability becomes less absolute. Ownership becomes more distributed across interacting systems. Observability expands beyond infrastructure health into behavioral monitoring. Human oversight becomes infrastructural rather than exceptional.

The future of software likely involves systems that remain partially unpredictable operationally even when functioning correctly. Engineers increasingly design architectures not around eliminating uncertainty entirely, but around constraining and managing uncertainty intelligently.

That is a very different philosophy from traditional software engineering.

And honestly, the industry still has not fully absorbed what that means yet.

Conclusion: architecture in a world of probabilistic systems

Generative systems represent a deeper architectural shift than most discussions currently acknowledge. They are not simply new interfaces layered on top of existing software patterns. They change foundational assumptions about predictability, orchestration, reliability, observability, and system behavior itself.

Engineers are increasingly designing systems whose outputs cannot be fully predicted ahead of time. The architecture revolves less around deterministic execution and more around shaping probabilistic behavior through orchestration, retrieval, context management, policy enforcement, and human oversight.

That does not make traditional software engineering obsolete. Distributed systems, scalability, infrastructure reliability, and operational discipline still matter enormously. But they now coexist with systems whose behavior emerges dynamically rather than mechanically.

The hardest part of generative systems isn’t making them intelligent. It’s building systems around intelligence that behaves unpredictably.

They waited for requests, waited for inputs, waited for explicit instructions written by engineers who already knew the sequence of actions the system was supposed to execute.

Even large-scale distributed systems largely operated under this assumption. Complexity came from scale, concurrency, and coordination—not from the system independently deciding what it should do next.

That assumption is starting to break.

Agentic system design represents something meaningfully different from earlier generations of software architecture because they shift systems away from rigid workflow execution and toward goal-oriented behavior. Instead of following fixed sequences of operations, these systems increasingly evaluate context, select actions dynamically, coordinate tools, maintain memory across interactions, and adapt behavior based on evolving objectives.

And once systems begin making decisions instead of simply executing instructions, many traditional assumptions in System Design become unstable.

Traditional systems execute workflows. Agentic systems pursue goals.

That sounds like a subtle distinction at first. In practice, it changes almost everything about how engineers reason about architecture, reliability, coordination, observability, and operational control.

Because workflows are predictable.

Goals are not.

The limits of traditional System Design assumptions

Most classic System Design principles emerged from environments where system behavior was largely deterministic. Services communicated through explicit contracts. APIs produced predictable outputs given known inputs. Orchestration layers followed predefined sequences. Failures were generally traceable to implementation bugs, infrastructure outages, or scaling bottlenecks.

Even when systems became highly distributed, the architecture itself remained intentionally constrained. Engineers designed workflows carefully because predictability was considered a feature, not a limitation. The more controlled the execution path, the easier the system became to reason about operationally.

This is why traditional distributed systems engineering placed enormous emphasis on consistency guarantees, reproducibility, observability, and deterministic debugging workflows. The assumption underneath all of it was that the system should behave in understandable ways if engineered correctly.

Agentic systems challenge that assumption directly.

Once systems begin dynamically selecting tools, generating plans, revising strategies, or coordinating actions autonomously, behavior becomes partially emergent rather than fully predefined. The architecture no longer represents a fixed execution graph. It becomes a decision-making environment.

That distinction matters because emergent systems are harder to predict, harder to debug, and harder to constrain operationally.

And yet modern AI infrastructure increasingly pushes directly in that direction.

What makes a system “agentic”

The term “agentic” gets used somewhat loosely now, often attached to any application involving large language models. But not every LLM-powered application is actually agentic in a meaningful architectural sense.

An agentic system is not simply generating responses. It is operating toward objectives.

The defining characteristic is not conversation—it is autonomous decision-making within bounded environments. These systems evaluate goals, reason about possible actions, select tools dynamically, maintain contextual memory, and adjust behavior based on evolving state.

Importantly, the sequence of operations is not always predetermined by the engineer ahead of time.

• The system decides what actions to take

• The sequence of actions is not always predefined

• Outcomes emerge dynamically from context and goals

This introduces a very different architectural model from traditional software systems. Instead of workflows being fully encoded into application logic, workflows become partially generated at runtime through reasoning processes.

That shift sounds abstract until you look at what modern agentic systems are actually doing operationally: retrieving context dynamically, invoking APIs autonomously, orchestrating multi-step workflows, coordinating between tools, maintaining long-term memory, and adapting behavior based on prior interactions.

The system stops behaving like a static application and starts behaving more like an adaptive operator moving through an environment.

And environments are inherently harder to control than workflows.

Why AI agents changed architectural thinking in 2026

The rise of large language models between 2023 and 2026 accelerated this shift dramatically because LLMs introduced a general-purpose reasoning interface flexible enough to coordinate actions dynamically across tools and systems.

Initially, most organizations used LLMs primarily for generation tasks—summarization, content creation, conversational interfaces, retrieval augmentation. But once models became capable of planning multi-step workflows and interacting reliably with external systems, architectural thinking started changing quickly.

The system was no longer simply responding.

It was acting.

That distinction triggered a major shift in infrastructure expectations. Suddenly engineers were not just building applications around inference APIs. They were designing environments where models could make operational decisions autonomously.

This introduced entirely new categories of architectural concerns:

• Tool orchestration

• Long-term memory management

• Multi-agent coordination

• Policy enforcement layers

• Behavioral observability

• Human intervention mechanisms

Traditional application infrastructure was never really designed for systems capable of adaptive planning behavior. It assumed deterministic orchestration controlled explicitly by developers.

Agentic architectures destabilize that assumption because orchestration itself becomes partially dynamic.

Coordination becomes the real problem

One of the most interesting things about agentic systems is that intelligence itself often turns out to be less operationally difficult than coordination.

Single-agent systems are already complicated enough. But once multiple autonomous agents begin interacting, familiar distributed systems problems start reappearing in new forms.

Shared state becomes complicated. Context synchronization becomes fragile. Decision conflicts emerge. Communication overhead increases. Cascading actions become harder to predict. Tool access requires coordination policies. Memory consistency starts mattering operationally.

In many ways, agentic systems quietly reintroduce distributed systems complexity through a completely different interface.

The hardest part of agentic systems isn’t intelligence. It’s coordination.

This becomes especially obvious in multi-agent orchestration environments where agents specialize around tasks but must still cooperate toward larger objectives. Suddenly engineers are reasoning not only about distributed services, but about distributed decision-making behavior.

And distributed decision-making systems are notoriously difficult to constrain.

Coordination complexity grows nonlinearly because agents are not simply executing static operations—they are adapting behavior dynamically while interacting with evolving context.

The architecture starts behaving less like software and more like an organizational structure. That creates fascinating possibilities operationally. It also creates extraordinary uncertainty.

Predictability vs autonomy

The central tension inside agentic system design is straightforward to describe and extremely difficult to resolve:

The more autonomy you give the system, the less predictable it becomes.

Traditional software engineering optimized heavily around predictability because predictability enables reliability, observability, debugging, and operational trust. Agentic systems intentionally loosen some of those constraints in exchange for adaptability and dynamic behavior.

That trade-off changes how engineers think about architecture fundamentally.

Autonomous systems can adapt to novel scenarios more flexibly than rigid workflows. They can reason through incomplete information, select tools dynamically, and coordinate actions without explicit programming for every edge case.

But flexibility introduces uncertainty.

The system may take actions engineers did not explicitly anticipate. Tool usage sequences may vary between similar requests. Behavior evolves over time as memory accumulates or environmental context changes.

Debugging becomes much harder because failures increasingly emerge behaviorally rather than deterministically.

The question stops being “Which line of code failed?”

And becomes:

“Why did the system decide this behavior made sense?”

That’s a fundamentally different category of operational reasoning.

Memory, context, and evolving system behavior

Persistent memory changes agentic systems in ways many engineers still underestimate.

Traditional applications generally treat interactions as relatively isolated events. Even stateful systems typically operate within predictable workflow boundaries. Agentic systems increasingly maintain contextual continuity across interactions, allowing prior experiences to influence future decisions dynamically.

That creates adaptive behavior over time.

The system no longer behaves identically under identical prompts because historical interactions shape reasoning context. Memory introduces evolving behavior patterns, long-term adaptation, and environmental learning effects.

Architecturally, this creates difficult questions:

These are not ordinary application design problems anymore. They resemble organizational knowledge management problems embedded inside software infrastructure.

And because memory influences future behavior probabilistically, operational debugging becomes significantly more complicated.

Human oversight in agentic architectures

Despite all the enthusiasm around autonomous systems, one thing becomes increasingly obvious operationally:

Humans remain critical infrastructure.

The more powerful agentic systems become, the more uncomfortable fully autonomous execution starts to feel. Engineers quickly realize that unrestricted autonomy creates unacceptable operational risk, especially once systems interact with real-world infrastructure, financial systems, customer data, or external services.

This is why modern agentic architectures increasingly include approval layers, intervention boundaries, escalation workflows, and policy enforcement systems.

• Autonomy without oversight creates risk

• Human review slows systems down but increases trust

• Full automation is often operationally uncomfortable

There’s a subtle but important shift happening here. Engineers are no longer merely designing systems that execute logic. They are designing systems that negotiate operational authority.

The architecture itself increasingly encodes trust boundaries between humans and autonomous systems. And trust boundaries are sociotechnical constructs as much as technical ones.

Traditional systems vs agentic systems

What this comparison really reveals is that agentic systems fundamentally alter the relationship between engineers and system behavior.

Traditional architectures are engineered primarily through explicit control. Agentic architectures increasingly rely on bounded autonomy, policy constraints, and behavioral supervision rather than strict workflow determinism.

That doesn’t make them “better” automatically.

It makes them operationally different in ways the industry is still learning how to manage.

Trade-offs at the center of agentic system design

Every meaningful discussion about agentic architectures eventually collapses into uncomfortable trade-offs.

• Autonomy vs control

• Adaptability vs predictability

• Intelligence vs operational reliability

• Flexibility vs safety

The difficult part is that these tensions do not resolve cleanly.

More autonomous systems adapt better but become harder to constrain. Stronger policy enforcement improves safety but reduces flexibility. Richer memory systems improve continuity while increasing behavioral unpredictability.

And unlike ordinary software systems, these trade-offs often evolve continuously after deployment because the systems themselves adapt over time.

That changes how architecture must be reasoned about operationally.

Failure modes in agentic systems

Failure in agentic systems feels different from traditional software failure because the system may technically function correctly while behavior still becomes undesirable.

Goal misalignment emerges. Tools are used unexpectedly. Agents reinforce incorrect assumptions. Autonomous loops cascade through systems unexpectedly. Silent reasoning errors accumulate gradually rather than triggering explicit failures.

In deterministic systems, bugs are usually implementation problems. In agentic systems, they increasingly become behavioral problems.

This distinction matters enormously because behavioral failures are much harder to diagnose operationally. Engineers cannot simply trace stack traces or infrastructure metrics anymore. They must reason about why the system arrived at particular decisions under probabilistic context conditions.

That’s a very different debugging paradigm from traditional software engineering.

And frankly, the industry still lacks mature operational frameworks for managing it consistently.

Misconceptions about agentic systems

Several misconceptions continue distorting conversations around agentic architecture.

“Agents are just chatbots.”

No. Conversational interfaces are only one surface area. Agentic systems are fundamentally about autonomous action and decision orchestration.

“More autonomy automatically means better systems.”

Often false. Excessive autonomy frequently introduces operational instability and trust problems.

“LLMs solve orchestration automatically.”

LLMs assist reasoning. They do not eliminate coordination complexity, observability challenges, or infrastructure constraints.

“Agentic systems eliminate engineering discipline.”

If anything, they increase the need for careful systems engineering because unpredictable behavior requires stronger operational boundaries, monitoring, and policy enforcement.

The hype cycle often frames autonomy as simplification. Operationally, autonomy frequently increases complexity instead.

What agentic systems reveal about the future of software

Agentic systems reveal something important about where software architecture is heading generally: Software is gradually shifting from static execution environments toward adaptive operational systems.

That shift changes the role of engineering itself. Infrastructure increasingly revolves around orchestration layers, observability systems, behavioral monitoring, policy enforcement, and coordination management rather than deterministic workflow implementation alone.

Engineers are becoming supervisors of adaptive systems as much as builders of deterministic logic.

This does not eliminate traditional engineering concerns. Distributed systems, scalability, consistency, and reliability still matter enormously. But now those concerns increasingly interact with probabilistic reasoning systems capable of partially autonomous behavior.

The result is software infrastructure that feels less mechanically predictable than previous generations of architecture. And that introduces both enormous capability and enormous operational discomfort simultaneously.

The future of agentic architecture

The next several years will probably involve increasing experimentation around multi-agent ecosystems, policy-driven orchestration frameworks, AI-native infrastructure layers, and collaborative human-agent operational environments.

But the systems that succeed operationally are unlikely to be the ones pursuing unrestricted autonomy blindly.

More likely, successful architectures will emphasize constrained autonomy inside carefully monitored environments with explicit intervention boundaries and strong observability systems.

In other words:

And trust is much harder to engineer than intelligence alone.

Final words: designing systems that increasingly make decisions on their own

Agentic systems represent a genuine architectural shift because they move software away from purely deterministic execution and toward adaptive decision-making behavior.

That transition changes everything about how engineers reason about infrastructure, coordination, reliability, debugging, and operational control. Systems stop behaving like static workflows and start behaving more like evolving participants inside dynamic environments.

And while that creates enormous new possibilities, it also introduces forms of uncertainty traditional software engineering was never really optimized to handle.

The deeper challenge is not simply building systems that can act autonomously. It is building systems whose autonomy remains understandable, observable, and operationally trustworthy over time.

Because once software begins making decisions on its own, engineering stops being purely about implementation.

It becomes increasingly about governance.

Not easy, necessarily, but bounded. You could disagree with the process, complain about algorithm puzzles, or question whether reversing binary trees reflected actual engineering work, but at least the evaluation itself felt relatively deterministic. Problems had constraints. Solutions had measurable correctness. Even system design interviews, for all their ambiguity, generally revolved around infrastructure that behaved predictably once designed correctly.

Machine learning system design interviews changed that dynamic completely.

Suddenly candidates weren’t just being asked to reason about distributed systems or scalability. They were being asked to reason about systems that evolve over time, systems whose outputs are probabilistic rather than deterministic, systems heavily shaped by data quality, operational feedback loops, and shifting real-world behavior. The interview stopped being about designing software in the traditional sense and became something closer to reasoning through uncertainty under pressure.

And that shift feels deeply connected to how software engineering itself changed between 2023 and 2026.

Machine learning interviews stopped being about writing correct answers a long time ago. Now they’re about reasoning through systems nobody fully controls.

That distinction matters because ML system design interviews are not merely harder versions of backend interviews. They represent a fundamentally different model of technical evaluation. They test whether engineers can think coherently in environments where correctness itself becomes slippery.

And increasingly, that’s what modern software systems actually look like.

Why ML systems feel fundamentally different

Traditional software systems are built around deterministic assumptions. Inputs lead to predictable outputs. Logic behaves consistently. Bugs can usually be reproduced. Failures often originate from explicit implementation errors or infrastructure breakdowns.

Machine learning systems violate those expectations constantly.

The model itself introduces probabilistic behavior into the architecture. Two identical inputs may produce slightly different outputs depending on context, model state, or inference conditions. Performance degrades gradually rather than catastrophically. Drift accumulates quietly. Feedback loops alter future behavior. Data distributions evolve independently of the application logic surrounding them.

That fundamentally changes how engineers reason about systems.

In ordinary backend development, the infrastructure primarily exists to preserve correctness. In ML systems, infrastructure exists partly to manage uncertainty itself. Monitoring systems track statistical behavior rather than explicit failures. Evaluation pipelines measure degradation probabilistically. Retraining workflows become operational necessities because the system’s environment changes continuously.

This is exactly why ML system design interviews feel cognitively heavier than traditional system design discussions. Candidates are not merely optimizing architectures. They are reasoning about systems whose behavior remains partially unstable even after deployment.

And unlike conventional distributed systems, many ML systems continue evolving after release.

The shift from coding problems to systems thinking

Part of what changed over the last several years is that implementation itself stopped being the primary bottleneck for many engineering organizations.

Code generation tooling improved. Infrastructure abstractions matured. Cloud orchestration became more standardized. AI-assisted development accelerated ordinary implementation work. As a result, interviews gradually shifted away from evaluating whether candidates could write and learn code quickly and toward evaluating whether they could reason through increasingly complex systems.

Machine learning accelerated that transition dramatically.

Modern ML system design interviews rarely focus deeply on implementation syntax because implementation is no longer the interesting part. The difficult part is understanding interactions between components, constraints, trade-offs, operational realities, and evolving behavior over time.

Candidates are expected to think in layers simultaneously:

• Infrastructure behavior

• Model performance

• Data quality

• Cost constraints

• Real-time latency requirements

• Long-term system evolution

That’s a very different cognitive exercise than solving isolated coding problems.

And in many ways, these interviews mirror what engineering roles themselves became after the AI boom. Software systems are no longer static products with deterministic workflows. Increasingly, they are adaptive systems driven by probabilistic models operating inside distributed infrastructure under changing environmental conditions.

The interviews simply inherited that complexity.

Subscribe now

The hidden complexity behind “design an ML system”

One of the stranger aspects of ML system design interviews is how casually enormous complexity gets compressed into tiny prompts.

These questions sound deceptively manageable because interview framing abstracts away the operational reality underneath. But every one of these systems contains enormous hidden complexity that real engineering organizations spend years managing.

The candidate is expected to reason through data ingestion pipelines, training infrastructure, feature computation, inference orchestration, monitoring, retraining workflows, evaluation systems, and scalability concerns, often within forty-five minutes.

Which is simultaneously unrealistic and revealing.

The interview environment simplifies reality aggressively while still exposing how candidates think about interconnected systems. The simplification is artificial, but the ambiguity is real.

• The model is only one piece

• Infrastructure decisions shape outcomes

• Data quality quietly controls everything

That last point especially tends to destabilize candidates because many engineers approach ML interviews assuming the “AI” portion is the center of the system. In practice, operational infrastructure often dominates the actual engineering complexity.

Production ML systems are mostly data systems wrapped around probabilistic inference layers.

And the interviews increasingly reflect that reality.

Leave a comment

Ambiguity becomes the actual interview

What makes these interviews psychologically difficult is not necessarily the scale of the systems themselves. It’s the absence of stable boundaries.

Traditional coding interviews feel constrained even when difficult. There is usually a defined problem space, a measurable goal, and some implicit understanding of correctness. ML system design interviews intentionally remove those stabilizing structures.

Constraints remain incomplete. Requirements stay underspecified. Trade-offs emerge dynamically during discussion. Sometimes the interviewer changes assumptions halfway through intentionally to see how candidates adapt.

At first, many engineers interpret this as poor interview structure.

Then eventually they realize the ambiguity is the structure.

Most candidates aren’t uncomfortable because the systems are large. They’re uncomfortable because nobody tells them what the “right” answer is.

That discomfort exposes something important about modern engineering work generally. Real ML systems rarely arrive with perfectly defined requirements. Teams operate under uncertainty constantly. Data behaves unpredictably. Metrics conflict. Latency requirements compete against model quality. Infrastructure budgets constrain experimentation.

The interview simply simulates that ambiguity in compressed form.

And many candidates struggle not because they lack technical knowledge, but because ambiguity itself creates cognitive pressure. They search for correctness when the real evaluation revolves around reasoning quality under uncertainty.

Why these interviews exploded after the AI boom

Between 2023 and 2026, hiring expectations changed dramatically.

Before the generative AI explosion, machine learning expertise remained relatively specialized. Most backend engineers could operate successfully without deep familiarity with model behavior, embeddings, inference infrastructure, or retrieval systems.

That separation collapsed quickly.

AI-native products emerged across nearly every category of software. Recommendation systems became more sophisticated. Search evolved into semantic retrieval. Copilots entered development workflows. LLM integrations spread into ordinary SaaS products. Suddenly ML awareness stopped being isolated to dedicated ML teams.

Companies responded predictably: interview processes evolved to evaluate broader systems thinking around machine learning infrastructure.

And importantly, these interviews expanded beyond pure ML engineering roles.

Backend engineers increasingly encountered ML-flavored infrastructure discussions. Platform engineers needed awareness of inference scaling and vector databases. Full-stack roles started including conversations about recommendation systems or retrieval pipelines.

Machine learning stopped being a specialization and became part of general systems literacy.

That shift fundamentally expanded the role of ML system design interviews inside technical hiring.

The widening expectations placed on engineers

One reason these interviews feel overwhelming is that they quietly combine multiple disciplines simultaneously.

Candidates are increasingly expected to reason across:

• Distributed systems

• Machine learning workflows

• Infrastructure scalability

• Data engineering

• Observability and monitoring

• Operational trade-offs

Ten years ago, these domains were often handled by relatively separate teams. Today they increasingly overlap inside production systems.

Modern engineers are expected to understand not only how software behaves, but how systems evolve statistically over time. They must think about infrastructure and models simultaneously. They must reason about operational cost alongside predictive quality.

That interdisciplinary expansion changed the emotional texture of technical interviews. The candidate is no longer evaluated solely as a programmer. They are evaluated as a systems thinker operating across unstable domains.

And honestly, that’s exhausting.

Interviews as simulations of operational thinking

The more ML system design interviews evolved, the less they resembled examinations of technical memorization and the more they resembled compressed simulations of operational reasoning.

The strongest candidates are rarely the ones who recite the most patterns mechanically. They are usually the ones who reason calmly through uncertainty, identify important trade-offs clearly, and prioritize constraints coherently.

This is partly why these interviews frustrate people coming from heavily preparation-driven environments. Memorization helps only up to a point because the actual evaluation revolves around dynamic reasoning.

Operational thinking matters more than architectural recall.

Can they prioritize imperfect decisions under incomplete constraints? Those are fundamentally operational questions.

And increasingly, that’s what engineering organizations actually care about.

The reason ML interviews feel more cognitively demanding is not merely that they are “harder.” It’s that they operate across multiple unstable dimensions simultaneously.

Traditional systems can often be evaluated based on correctness and scalability. ML systems require reasoning about behavior that remains partially uncertain even after deployment.

The candidate is effectively being asked to design infrastructure around evolving probabilistic behavior under operational constraints.

That’s a fundamentally different category of systems thinking.

Trade-offs sit at the center of everything

Nearly every meaningful ML system discussion eventually collapses into trade-offs.

• Accuracy vs latency

• Scalability vs operational cost

• Model quality vs infrastructure simplicity

• Automation vs human oversight

The difficult part is that none of these tensions resolve cleanly.

Higher-quality models often increase inference cost and latency. More aggressive automation introduces monitoring complexity. Better personalization increases privacy concerns. Real-time retraining improves adaptability but destabilizes infrastructure predictability.

And the interviews intentionally surface these tensions because the trade-offs themselves reveal how candidates think.

There is rarely a perfect architecture.

There are only architectures optimized around particular constraints.

Leave a comment

The uncomfortable reality of interview preparation

One reason candidates often feel perpetually unprepared for ML system design interviews is that preparation material and actual reasoning are not the same thing.

Courses, tutorials, and structured examples create familiarity. They expose patterns. They help candidates recognize components and workflows. But recognition creates a dangerous illusion because it feels very similar to understanding at first.

Then the actual interview starts.

The prompt shifts slightly outside rehearsed examples. Constraints remain unclear. The discussion evolves unpredictably. Suddenly the candidate realizes they understood the diagrams more than the systems themselves.

This is partly why so many engineers describe ML interview preparation as strangely unsatisfying. The more material they consume, the more they realize how much reasoning still feels unstable underneath.

The gap between recognition and operational intuition becomes painfully visible under pressure.

Misconceptions about ML system design interviews

Several misconceptions continue distorting how people approach these interviews.

“There’s a correct architecture.”

Usually there isn’t. Most systems involve competing trade-offs rather than objectively optimal designs.

“Better models solve everything.”

Infrastructure, data quality, observability, and operational constraints often dominate production outcomes far more than marginal model improvements.

“Memorizing patterns is enough.”

Patterns help establish vocabulary. They do not replace reasoning under uncertainty.

“These interviews only matter for ML engineers.”

Increasingly false. ML awareness now extends across backend, infrastructure, platform, and product engineering roles.

These misconceptions persist partly because people still approach ML interviews using mental models inherited from older software interview culture. But the underlying systems changed faster than the interview expectations surrounding them.

What these interviews reveal about the industry

In many ways, ML system design interviews reveal more about the software industry itself than about individual candidates.

They reflect an industry increasingly dependent on systems that behave probabilistically, evolve continuously, and operate under unstable real-world conditions. They reveal how software engineering itself shifted away from deterministic implementation toward adaptive infrastructure thinking.

And perhaps most importantly, they expose how uncertainty became normalized inside technical work.

The industry isn’t asking engineers to know everything. It’s asking whether they can think clearly while surrounded by uncertainty.

That distinction matters because many candidates interpret ambiguity as evidence of personal inadequacy when in reality the ambiguity reflects the systems themselves.

Modern engineering increasingly revolves around incomplete information, probabilistic behavior, operational trade-offs, and evolving infrastructure constraints.

The interviews merely mirror that environment.

The future of ML system design interviews

It’s difficult to predict exactly how these interviews evolve over the next several years, especially as AI tooling increasingly automates portions of implementation work.

But one trend seems relatively clear: reasoning will probably matter more, not less.

As copilots improve and infrastructure abstractions mature further, implementation details become easier to outsource to tooling. The difficult part remains deciding what systems should optimize for, how constraints interact, and how trade-offs should be prioritized.

In other words, operational reasoning becomes the durable skill.

Ironically, the more AI assists ordinary development, the more interviews may shift toward evaluating the uniquely human aspects of systems thinking: judgment, prioritization, ambiguity management, and trade-off reasoning.

Not because humans are irreplaceable abstractly, but because modern systems themselves remain operationally messy.

And messy systems still require coherent reasoning.

Conclusion: interviews as reflections of modern engineering complexity

Machine learning system design interviews feel difficult because they are evaluating something much larger than technical recall.

They are evaluating whether engineers can reason through evolving systems operating under uncertainty, conflicting constraints, imperfect data, and long-term operational pressure. They reflect an industry increasingly shaped by probabilistic infrastructure rather than deterministic software alone.

And perhaps that’s why these interviews feel psychologically different from older technical evaluations.

They don’t just test knowledge.

They test whether candidates can remain intellectually coherent inside environments where certainty itself becomes unstable.

That, more than anything else, may define modern software engineering now.

Machine learning used to occupy a fairly specialized corner of software engineering. There were researchers, data scientists, infrastructure teams supporting models at scale, and a relatively small number of engineers working directly on ML systems. Most developers could safely ignore the field without feeling professionally vulnerable. That boundary disappeared faster than most people expected.

By 2026, machine learning stopped feeling like an optional specialization and started feeling like ambient career pressure. Backend engineers suddenly needed to understand embeddings. Frontend developers were integrating copilots into workflows they barely understood themselves. Infrastructure engineers were being asked about vector databases and inference latency in meetings that had nothing to do with traditional ML. Product roadmaps became saturated with “AI-first” language almost overnight.

And because the transition happened so quickly, many engineers developed the same quiet feeling simultaneously: the feeling that everyone else somehow already understands this.

The hardest part about learning machine learning today isn’t the complexity. It’s the speed at which everyone expects you to understand it.

Over the last few years at Educative, I’ve watched this pressure show up repeatedly in how engineers approach learning. Not calmly. Not methodically. Usually in bursts of panic-driven acceleration. People trying to compress years of foundational understanding into weekends, late-night study sessions, or hurried learning sprints between meetings and production incidents.

What makes this difficult is not simply the amount of information. Software engineers are already accustomed to large technical domains. The harder part is that machine learning arrived wrapped in a cultural expectation of urgency. The industry stopped treating ML as something you gradually grow into and started treating it as something you should already know.

That changes how people learn. Usually not for the better.

The industry’s obsession with acceleration

One of the more exhausting aspects of the AI era is how aggressively it compresses expectations around learning.

Every new framework, model release, orchestration layer, or infrastructure abstraction arrives with the implication that immediate familiarity is necessary for professional relevance. Engineers don’t just encounter new ideas anymore. They encounter new ideas accompanied by collective anxiety.

The ecosystem moves fast enough that people rarely feel finished understanding one concept before the industry pivots toward another. One month everyone is discussing prompt engineering. Then retrieval-augmented generation dominates conversations. Then agents. Then multimodal systems. Then infrastructure tooling around agents. Then model routing. Then evaluation pipelines. The abstractions accumulate faster than intuition can stabilize.

This creates a strange kind of cognitive fatigue that I think many engineers struggle to articulate properly. It’s not ordinary burnout exactly. It’s closer to the feeling of perpetually running behind a moving train while the people already on board insist the train is easy to catch.

And because modern technical culture rewards visible participation, people often respond by increasing consumption rather than understanding. More newsletters. More videos. More podcasts. More tutorials. More “10 things every AI engineer must know” threads written with absolute certainty about technologies that barely existed six months earlier.

The result is that many developers end up technically saturated but conceptually fragmented.

Subscribe now

The illusion of “catching up”

One thing I’ve observed repeatedly is that machine learning creates an unusually strong illusion of progress.

You can consume a tremendous amount of ML content very quickly. You can recognize terminology within days. You can follow architectural diagrams after a few weeks. You can become conversationally functional surprisingly fast. And because the internet rewards familiarity signals, that surface-level understanding often feels deeper than it really is.

But machine learning resists compression in ways many people underestimate.

There’s a difference between understanding what a transformer architecture is and understanding why large-scale ML systems behave unpredictably under real-world constraints. There’s a difference between recognizing terms like embeddings, fine-tuning, and inference pipelines and developing intuition about how these systems fail, drift, degrade, or interact operationally.

Rapid content consumption creates confidence before it creates mental models.

That distinction matters.

The engineers who struggle most are often not the ones who know the least. They’re the ones who mistake familiarity for understanding and then become frustrated when deeper reasoning still feels inaccessible. They can follow explanations while consuming them, but cannot reconstruct the ideas independently afterward.

And honestly, that’s not a personal failure. It’s a predictable consequence of how modern technical learning environments operate.

What developers are actually struggling with

After watching thousands of engineers move through ML learning material, certain patterns appear repeatedly.

The confusion is often less about mathematics than people assume. More commonly, it comes from systems thinking gaps. Engineers understand isolated concepts but struggle to connect them into coherent behavior.

• People understand the words but not the systems

• They recognize architectures without understanding trade-offs

• They consume information faster than they process it

This shows up constantly around probabilistic behavior. Traditional software engineering trains people to expect deterministic systems. Inputs produce predictable outputs. Logic behaves consistently. Failures can usually be reproduced.

Machine learning systems violate those expectations constantly.

Models behave statistically rather than deterministically. Performance degrades gradually. Data quality influences outcomes in nonlinear ways. Systems drift over time. Small distribution shifts create unexpected behavior. Suddenly engineers aren’t just debugging code anymore—they’re debugging uncertainty.

That changes the learning experience fundamentally because it requires a different kind of intuition than traditional application development.

Why structured learning became necessary

Part of the reason we built resources like Fundamentals of Machine Learning for Software Engineers was because we kept seeing the same pattern repeatedly: engineers drowning in disconnected information while lacking conceptual continuity.

One thing we realized early is that most developers do not need more hype-driven summaries. They need structure. They need explanations that connect ideas gradually instead of treating machine learning as a collection of isolated buzzwords.

The same pattern appeared with Machine Learning System Design. A lot of developers were learning models abstractly without understanding how ML systems actually behave in production environments. But production ML is not just about algorithms. It’s about infrastructure, latency, data pipelines, observability, drift, scaling constraints, and operational trade-offs.

And interview pressure amplified this further, which is partly why courses like Grokking the Machine Learning Interview emerged. Not because interviews represent ideal learning environments—they absolutely do not—but because engineers increasingly felt pressure to perform ML fluency publicly before their understanding fully matured.

Even the Key Principles of Machine Learning Every Developer Should Know blog came from observing the same recurring confusion: people trying to absorb advanced abstractions without stable mental models underneath.

The goal was never to simplify machine learning into something artificial. It was to reduce unnecessary chaos around the learning process itself.

Share Grokking the Tech Career

Machine learning feels different because it behaves differently

I think one reason machine learning creates so much discomfort for traditional engineers is that the systems themselves violate deeply ingrained expectations about software behavior.

Conventional systems engineering revolves around precision. You design explicit logic paths. You reason deterministically. Failures are usually traceable through code and infrastructure. Machine learning introduces systems that behave probabilistically, adapt dynamically, and sometimes produce outputs that are difficult to explain fully even when the implementation is technically correct.

That ambiguity changes how people learn.

Engineers accustomed to deterministic reasoning often experience ML as conceptually slippery. You can understand the architecture and still feel uncertain about the behavior. You can understand the pipeline and still feel unclear about the outputs. The systems resist tidy mental compression because the behavior itself emerges statistically rather than logically.

And that uncertainty becomes psychologically difficult in an industry obsessed with confidence signaling.

The pressure to become AI-native

Somewhere between 2023 and 2026, machine learning quietly stopped being optional knowledge.

Somewhere between 2023 and 2026, machine learning quietly stopped being optional knowledge.

That transition happened unevenly, but the cultural effect was enormous.

Suddenly ML awareness became expected even for engineers who never intended to specialize in machine learning directly. Backend engineers felt pressure to understand retrieval systems. Frontend developers were integrating AI features into products. Infrastructure teams were expected to reason about inference costs and vector search latency.

The result is that many engineers now experience machine learning less as curiosity and more as professional obligation.

And obligation changes the emotional texture of learning. People stop learning because they’re fascinated and start learning because they’re afraid of becoming obsolete.

That fear rarely gets discussed honestly, but it’s present everywhere.

Interview culture and performance anxiety

Interview culture intensified all of this considerably.

The rise of ML-focused interview preparation created another layer of pressure because engineers increasingly felt expected to demonstrate machine learning fluency performatively, often before developing durable intuition.

That’s partly why resources like Grokking the Machine Learning Interview resonated with people. Structured interview preparation provides emotional reassurance in uncertain environments. It gives the impression that machine learning can be organized into recognizable patterns and predictable expectations.

But there’s also a danger there.

Memorization creates temporary fluency. Real understanding develops much more slowly.

And machine learning interviews sometimes reward the former while actual engineering work depends heavily on the latter.

This creates an uncomfortable mismatch where people optimize for sounding informed before they feel genuinely grounded conceptually.

The difference between consuming information and building understanding

What this table really highlights is that genuine understanding often feels slower and emotionally less satisfying initially.

Fast content consumption produces immediate familiarity and conversational fluency. Structured learning feels slower because it prioritizes conceptual continuity over novelty. Long-term understanding feels slowest of all because it requires ambiguity, repetition, confusion, and gradual mental model formation.

Unfortunately, modern internet culture disproportionately rewards the first category while actual engineering capability depends much more heavily on the last one.

Trade-offs in learning ML quickly

Every engineer learning machine learning today is navigating the same difficult tensions.

• Speed vs depth

• Breadth vs intuition

• Trend awareness vs fundamentals

There’s no perfect balance here.

Move too quickly and understanding fragments into disconnected abstractions. Move too slowly, and the ecosystem evolves around you faster than your confidence stabilizes. Focus only on trends, and fundamentals remain weak. Focus only on fundamentals, and you risk feeling disconnected from industry conversations.

The pressure comes partly from trying to optimize all of these simultaneously, which is usually impossible.

The emotional side of modern technical learning

I think many engineers quietly underestimate how emotionally exhausting technical acceleration has become.

Most people are learning machine learning while already working demanding jobs, dealing with deadlines, navigating layoffs, handling burnout, or trying to maintain ordinary life responsibilities. Learning doesn’t happen in calm academic environments. It happens late at night after meetings. On weekends. During fragmented periods of exhaustion.

And because the field moves so quickly, people internalize this persistent sense that slowing down equals falling behind.

What rarely gets admitted publicly is how much comparison drives modern technical anxiety. Everyone online appears to understand more than they actually do. Social media disproportionately amplifies signals of certainty, speed, and productivity while almost entirely hiding confusion and slow learning.

So engineers end up privately overwhelmed while publicly pretending to keep pace.

What actually helps over time

The developers who eventually build strong intuition usually do something counterintuitive at some point:

They slow down.

Not permanently. Not by disengaging from the field entirely. But by shifting focus away from pure consumption velocity and toward mental model formation.

Understanding starts improving once engineers stop trying to absorb every new abstraction immediately and begin organizing relationships between ideas instead. Why systems behave probabilistically. Why infrastructure constraints shape model behavior. Why data quality matters more than most people initially realize. Why trade-offs dominate production ML.

This is where structure becomes genuinely valuable—not because it accelerates expertise artificially, but because it reduces cognitive chaos enough for understanding to develop gradually.

That process is slower than hype culture encourages. But it’s also much more sustainable.

Misconceptions about learning ML in 2026

There are a few assumptions I wish more engineers would question openly.

“Everyone else already understands this.”

Most people are far more uncertain than they appear publicly.

“You need to know everything.”

The field is now too broad for comprehensive mastery to be realistic even for specialists.

“Watching enough content equals mastery.”

Consumption creates recognition. Understanding requires reflection, struggle, and time.

These distinctions matter because unrealistic expectations distort the entire learning process.

Slowing down in a field that keeps accelerating

Machine learning is genuinely difficult. Not because engineers lack intelligence, but because the systems themselves are probabilistic, fast-moving, infrastructure-heavy, and conceptually layered in ways that resist rapid compression.

And maybe the healthier realization is that confusion is not evidence of failure. It’s evidence that you are engaging with something legitimately complex.

The industry often behaves as though expertise should arrive immediately after exposure. But real understanding still develops the same way it always has: slowly, unevenly, through repetition, uncertainty, and gradual intuition-building.

That process hasn’t changed, even if the hype cycles surrounding it have accelerated dramatically.

Most engineers aren’t behind. They’re just trying to learn a field that the industry itself barely understands completely.

Everyone shows up at once. Everyone believes timing matters. Everyone thinks they still have a chance. And then the system starts bending under pressure.

Subscribe now

Pages stall. Queues freeze. Sessions expire. Inventory appears and disappears unpredictably. Users get kicked out after waiting an hour. Social media fills with screenshots of errors within minutes. Somewhere deep inside the infrastructure, distributed coordination mechanisms are fighting to maintain consistency while traffic spikes harder than almost any normal web application ever experiences.

Most distributed systems fail quietly. Ticketing systems fail in public.

That’s what makes systems like Ticketmaster so interesting from an engineering perspective. They expose the limits of modern infrastructure in a way most applications never do. This isn’t “just another e-commerce system,” even though people often frame it that way. Buying a concert ticket looks superficially similar to buying shoes online, but the system behavior underneath is fundamentally different. Scarcity changes everything. Human behavior changes everything. The synchronization of demand changes everything.

And unlike many system design operations, ticketing platforms operate under emotional pressure, not just technical pressure. Nobody calmly retries later when tickets disappear in seconds. The infrastructure isn’t just serving requests—it’s managing collective frustration at internet scale.

Share Grokking the Tech Career

The illusion that ticketing is simple

From the outside, ticketing systems appear deceptively straightforward. A venue has seats. Users pick seats. Users pay. Inventory decreases. End of story. Compared to something like distributed databases or global messaging systems, the interface feels almost embarrassingly simple.

That illusion disappears the moment demand becomes synchronized.

The complexity of Ticketmaster-like systems doesn’t come from complicated business logic. It comes from extreme coordination under contention. The system has to maintain a globally consistent understanding of inventory while millions of users aggressively compete for the same finite resources in real time. That’s not normal web traffic. That’s closer to a distributed resource allocation problem happening in public under intense scrutiny.

Most applications are optimized around relatively predictable patterns. Traffic fluctuates gradually. Demand distributes naturally across products and time zones. Users tolerate minor delays because alternatives exist. Ticketing systems don’t get those luxuries. Demand concentrates around exact timestamps, specific artists, specific venues, and extremely small inventory windows.

The system isn’t serving “customers browsing products.” It’s handling synchronized digital panic.

And that changes almost every architectural assumption.

Scarcity changes everything

Scarcity is what transforms ticketing systems from ordinary transactional infrastructure into something much more volatile. The moment inventory becomes severely limited relative to demand, user behavior changes dramatically. Rational browsing disappears. Refresh behavior intensifies. Retry storms begin. Every millisecond suddenly feels meaningful to users, even if the backend reality is far messier than that perception.

The system stops behaving like a store and starts behaving like a contested resource market.

• Everyone arrives at once

• Everyone wants the same resource

• Nobody tolerates delays or inconsistency

That combination is catastrophic for distributed systems.

Most internet infrastructure benefits from probabilistic spreading of activity. Ticket drops create the opposite effect: perfect synchronization. Millions of users generate near-identical traffic patterns simultaneously, often targeting the same inventory objects. That produces concentrated hotspots at every layer of the stack—databases, caches, queues, session systems, payment workflows, inventory coordinators.

And scarcity amplifies emotional behavior. Users don’t passively consume the system. They attack it unintentionally through desperation. Multiple tabs. Repeated refreshes. Rapid retries. Session hopping. Every user behaves like a mild denial-of-service actor without realizing it.

The infrastructure isn’t just fighting scale. It’s fighting synchronized human anxiety.

Real-time contention and distributed coordination

The hardest technical problem inside systems like Ticketmaster is deceptively simple to describe:

How do you guarantee that one seat gets sold to exactly one person while millions of users are trying to claim it simultaneously?

That sounds trivial until you actually try to enforce it globally at scale.

The moment inventory becomes limited, every seat effectively becomes a contested distributed lock. Multiple users can see it simultaneously. Multiple sessions can attempt to reserve it simultaneously. Multiple geographically distributed services may temporarily disagree about its state. And all of this happens under extreme latency sensitivity because users interpret delay as unfairness.

Preventing double-booking requires coordination, but coordination introduces latency. Latency reduces perceived fairness. Reduced fairness increases retries and refresh storms. Those retries create more contention.

This is where clean interview diagrams completely collapse under real-world pressure.

In theory, you can serialize inventory updates and enforce strict transactional consistency. In practice, that coordination overhead becomes extremely expensive when millions of requests converge simultaneously. So systems start making compromises. Temporary reservations. Distributed locking. Soft holds. Expiring inventory claims. Queue gating. Eventual synchronization between replicas.

None of these mechanisms are perfect. They’re all attempts to balance speed, consistency, and survivability under pathological load.

And the uglier truth is that even small inconsistencies become highly visible because users are emotionally invested in outcomes.

Leave a comment

Queues as both infrastructure and psychology

Virtual queues are one of the most fascinating aspects of ticketing systems because they serve two completely different purposes simultaneously.

Technically, queues are load regulation systems. They smooth traffic spikes, protect downstream services, and prevent infrastructure collapse. But psychologically, they’re also fairness theater.

In systems like Ticketmaster, the queue is part infrastructure, part crowd control.

Users don’t just want access. They want perceived fairness. They need to believe the process is orderly, even if the underlying infrastructure is highly probabilistic and imperfect. The queue becomes a social contract between the platform and its users.

That’s why queue behavior matters so much beyond pure throughput optimization. Randomness feels unfair. Sudden jumps feel unfair. Freezes feel unfair. Even if the backend is technically functioning correctly, user trust erodes the moment the queue appears inconsistent.

In reality, many queue systems are doing far more than simply placing users in line. They’re segmenting traffic, prioritizing verified users, throttling suspicious behavior, coordinating admission rates with backend health, and dynamically adjusting based on inventory conditions.

The queue isn’t just protecting the infrastructure.

It’s protecting the perception of legitimacy.

The bot problem and adversarial traffic

Most web applications operate under relatively cooperative assumptions. Ticketing systems absolutely do not.

The moment high-demand events become financially valuable, the platform enters adversarial territory. Scalpers, automated buyers, browser farms, distributed bot networks, and scripted purchasing workflows all begin targeting the system aggressively. Suddenly, the infrastructure isn’t merely serving users—it’s defending itself against economically motivated automation.

This changes architectural priorities dramatically.

Anti-bot systems introduce friction intentionally. CAPTCHAs, behavioral analysis, device fingerprinting, rate limiting, identity verification, session validation—these mechanisms all exist to slow down adversarial actors. But they also degrade the experience for legitimate users.

That’s the uncomfortable trade-off ticketing systems constantly navigate:

The more aggressively you defend against bots, the more legitimate humans get caught in the blast radius.

And sophisticated bots increasingly behave like humans anyway. They simulate mouse movement, distribute requests geographically, rotate sessions, mimic browsing patterns, and adapt quickly to new defenses. It becomes an arms race where infrastructure must continuously evolve against automated adversaries with direct financial incentives.

This is why simplistic critiques of ticketing platforms often miss the reality of the environment they operate in. The system isn’t just overloaded. It’s actively under attack during major events.

Caching, replication, and stale reality

In most large-scale systems, aggressive caching is one of the easiest ways to improve performance. Ticketing systems complicate that assumption because stale data becomes emotionally explosive.

If inventory data propagates slowly across replicas or caches, users see seats that no longer exist. They click available tickets that disappear during checkout. They believe the platform is broken or manipulative. Even tiny synchronization delays create visible frustration because users interpret stale inventory as deception.

Caching suddenly becomes dangerous.

The problem is that you still need caching because origin systems cannot survive direct traffic at this scale. So platforms end up operating in an uncomfortable middle ground where they aggressively cache some data while carefully coordinating highly volatile inventory states closer to real time.

But “real time” in distributed systems is always messier than users imagine.

Inventory state propagates through layers of caches, replicated databases, reservation services, and geographically distributed infrastructure. During peak load, tiny timing inconsistencies become amplified into visible user-facing failures.

And users don’t care about eventual consistency models when they lose seats during checkout.

The infrastructure cost of rare events

One of the least appreciated aspects of Ticketmaster-like systems is how economically inefficient they are by nature.

Most systems optimize around relatively stable utilization. Ticketing systems optimize around catastrophic spikes that happen briefly but unpredictably. Infrastructure must survive massive synchronized surges that may last minutes or hours, even if average daily traffic is far lower.

That creates an ugly operational reality: enormous infrastructure capacity exists primarily for rare moments of collective urgency.

Cloud elasticity helps, but only partially. Scaling infrastructure reactively during explosive demand spikes is harder than people assume. Database coordination, queue systems, session management, inventory locking, and replication delays don’t magically disappear because compute instances scale horizontally.

The hardest bottlenecks are often stateful systems, not stateless application servers.

And provisioning for worst-case load is expensive. Very expensive.

Traditional e-commerce vs ticketing systems

This is why comparing Ticketmaster to ordinary shopping platforms misses the point entirely.

Traditional e-commerce systems benefit from diversification. Users browse different products, arrive at different times, and tolerate retries reasonably well. Ticketing systems experience concentrated warfare around identical inventory objects with emotionally charged users operating under severe scarcity.

The architecture isn’t just handling transactions. It’s handling synchronized contention at internet scale.

And synchronized contention is one of the hardest problems in distributed systems.

Trade-offs at the center of ticketing platforms

Every major design decision inside systems like Ticketmaster revolves around uncomfortable trade-offs.

• Fairness vs speed

• Availability vs consistency

• Security vs user experience

• Scalability vs operational cost

Strict consistency improves inventory correctness but increases latency. Aggressive anti-bot protections improve fairness but frustrate legitimate users. Large virtual queues protect infrastructure but create public anger. Relaxed synchronization improves throughput but risks overselling inventory.

There are no clean solutions here. Only compromises.

And those compromises become highly visible because users interact with the system during emotionally significant moments. Nobody cares about distributed systems theory when they fail to buy tickets after waiting three hours.

Failure in public

Most infrastructure failures remain invisible outside engineering teams. Ticketing failures become national conversations.

Nobody tweets when a background service crashes. They absolutely tweet when they can’t buy concert tickets.

That visibility changes operational pressure enormously.

Every outage becomes public theater. Every queue freeze generates screenshots. Every error message spreads instantly across social platforms. Infrastructure failures become cultural events because the affected users are emotionally invested and simultaneously online.

This creates extraordinary pressure on incident response teams. The system isn’t merely failing technically—it’s failing socially in real time under public scrutiny.

And unlike internal enterprise systems, there’s no graceful degradation path that users quietly accept. Ticketing failures directly collide with anticipation, money, scarcity, and disappointment all at once.

That’s a uniquely brutal operational environment.

Misconceptions about systems like Ticketmaster

The internet loves simplistic explanations for ticketing failures.

Those critiques sound satisfying, but they dramatically underestimate the complexity of coordinated contention systems.

Adding more stateless servers doesn’t solve distributed locking bottlenecks. Cloud scaling doesn’t eliminate consistency trade-offs. Perfect fairness isn’t technically achievable under massive synchronized demand. And anti-bot protections inherently create friction for real users too.

The harder truth is that these systems operate near the boundaries of what distributed coordination can realistically handle under adversarial conditions.

That doesn’t mean platforms are beyond criticism. It means the problem itself is fundamentally ugly.

What Ticketmaster reveals about modern distributed systems

Ticketing platforms expose something important about large-scale systems generally:

The hardest problems usually aren’t about serving content. They’re about coordinating scarce state under unpredictable human behavior.

Interview diagrams simplify distributed systems into neat boxes and arrows. Real systems are full of retries, race conditions, overloaded dependencies, stale replicas, partial failures, adversarial traffic, and emotional users behaving irrationally under pressure.

Ticketmaster-like systems make those realities impossible to ignore because the failure modes are public and immediate.

They remind us that distributed systems are ultimately sociotechnical systems. Infrastructure behavior cannot be separated from human behavior because humans adapt to the system continuously, often in destructive ways.

The future of large-scale ticketing systems

The next generation of ticketing infrastructure will probably become increasingly identity-driven and behavior-aware. AI-assisted traffic analysis, adaptive queue admission, stronger identity verification, device reputation systems, and more aggressive bot detection will continue evolving because the economic incentives are too strong not to.

But none of that fully solves the deeper issue.

The underlying problem is synchronized scarcity itself. When millions of people want the same limited thing at the same moment, no architecture can make the experience feel completely fair to everyone. Infrastructure can mitigate contention, regulate flow, and improve resilience, but it cannot eliminate disappointment.

And that’s the uncomfortable reality beneath these systems.

They aren’t just selling tickets.

They’re managing collective urgency at internet scale.

Conclusion: systems built around moments of collective urgency

Ticketmaster is fascinating because it reveals how fragile distributed systems become when scarcity, synchronization, and human emotion collide simultaneously.

The hardest part isn’t serving pages or processing payments. Modern infrastructure is already very good at that. The hard part is maintaining fairness, consistency, trust, and survivability while millions of users aggressively compete for the same finite resources in real time.

That’s what makes ticketing systems fundamentally different from ordinary web applications.

They are systems designed around moments of collective urgency.

And moments of collective urgency tend to break things.

That sense of predictability is fading.

Today, engineers are working with systems that don’t just execute logic—they evolve, react, and sometimes behave in ways that are difficult to fully anticipate. The boundaries that once made architecture feel clean and understandable are blurring. You deploy something, but what you observe in production is often a moving target.

This is why system design no longer feels like a static discipline. It feels more like navigating a system than constructing one. And that shift is not accidental—it’s the result of deeper changes in how modern systems are built and how they behave.

Subscribe now

What system design used to optimize for

For a long time, the priorities were not just clear—they were stable. Engineers designed systems around a small set of core concerns that felt universal and sufficient for most problems. Scalability, reliability, and availability formed the backbone of almost every architectural discussion, and importantly, these goals aligned well with how systems actually behaved.

Scalability was about growth. You expected more users, more data, more requests—and you designed your system to handle that increase without breaking. The key assumption here was that growth was mostly linear and predictable. If you could handle ten times the traffic, you could scale your system accordingly using techniques like sharding, caching, and load balancing.

Reliability was about correctness. Given the same inputs, the system should behave the same way every time. If something failed, it was usually because of a known issue—a bug, a network failure, or resource exhaustion. These were problems you could reason about, simulate, and fix with confidence.

Availability ensured that even when parts of the system failed, the overall system remained accessible. Redundancy, replication, and failover strategies were well-understood patterns that gave engineers a sense of control over failure scenarios.

What made all of this work was a deeper assumption: systems were deterministic. They followed rules. If you understood the rules, you understood the system.

That assumption is what made the discipline feel structured.

A shift driven by modern systems

Modern systems have quietly broken that assumption.

Today’s architectures are shaped by forces that don’t fit neatly into deterministic models. Systems now interact with constantly changing data, external services, and components that don’t always behave predictably. As a result, the system you deploy is not the same system you observe a week later—even if the code hasn’t changed.

This shift is driven by several patterns that have become increasingly common:

• Systems that learn from data rather than follow fixed logic

• Systems that change behavior over time based on feedback

• Systems that depend on external models and services outside your control

These aren’t edge cases anymore—they’re the norm.

When a system incorporates machine learning, its behavior becomes probabilistic. When it depends on third-party APIs or distributed systems and pipelines, its reliability depends on factors outside your immediate control. When it processes real-time data streams, its state is constantly evolving.

This introduces a new kind of complexity—not just in scale, but in behavior.

You’re no longer designing a system that executes a defined set of rules. You’re designing a system that reacts, adapts, and sometimes behaves in ways that you didn’t explicitly encode.

And that makes reasoning about it fundamentally harder.

AI-native systems and new complexity

The integration of AI into modern architectures has amplified this shift.

AI systems—especially those built around large language models—don’t behave like traditional software components. They don’t offer strict guarantees. They don’t always produce consistent outputs. And they often operate as black boxes, where the internal logic is not fully transparent.

Modern systems don’t just execute logic anymore—they generate behavior, and that changes how we reason about them.

This introduces entirely new dimensions to architectural thinking.

You now have to consider:

• Variability in outputs, even for identical inputs

• Latency that depends on model inference time

• Costs that scale with usage in unpredictable ways

The system becomes a mix of deterministic and probabilistic components. Some parts behave exactly as expected; others behave based on learned patterns that can shift over time.

This hybrid nature creates tension. Do you trust the model’s output, or do you build safeguards around it? Do you optimize for speed, or for cost? Do you treat the model as a core component or an external dependency? These are not questions that traditional architectures were designed to answer.

Share

Real-time and event-driven architectures

At the same time, the way systems process data has changed dramatically.

Modern architectures are increasingly event-driven, where systems react to streams of events rather than discrete requests. Instead of handling isolated interactions, they operate continuously, processing data as it flows through the system.

This introduces a different kind of complexity—one that is less about individual operations and more about interactions over time.

In traditional systems, you could trace a request from start to finish. You knew where it began, how it was processed, and where it ended. In event-driven systems, that clarity disappears. An event may trigger multiple downstream processes, each of which may generate further events.

The system becomes a network of flows.

This has several implications:

• There is no single “path” through the system

• Components are loosely coupled but highly interdependent

• State is distributed and constantly changing

Debugging becomes harder because you’re not tracing a single execution path—you’re reconstructing behavior across multiple asynchronous interactions. Small delays or inconsistencies can propagate in unexpected ways, making issues difficult to isolate.

What used to be a clean, layered architecture now feels more like a dynamic graph of interactions.

Observability over control

One of the most important shifts is how engineers understand systems. You no longer fully control them, and you observe them.

You don’t understand a modern system by reading its code. You understand it by watching it behave.

In traditional systems, you could reason about behavior by looking at the code and the architecture. In modern systems, that’s not enough. The behavior emerges from interactions between components, data, and external dependencies.

This is why observability has become central.

Logs, metrics, and traces are not just debugging tools—they are your primary way of understanding what the system is doing. They provide visibility into a system that is too complex to fully model mentally.

You look for patterns and track anomalies, and even interpret signals over time. The system becomes something you study. This changes the nature of engineering work. It’s less about defining behavior upfront and more about continuously interpreting what the system is doing in production.

Adaptability as a core requirement

If there’s one idea that defines modern architecture, it’s adaptability. Systems are no longer expected to remain stable—they are expected to evolve. Features are rolled out incrementally. Experiments are run in production. Feedback loops drive continuous change.

You’re not building a finished system. You’re building something that will keep changing. This requires a shift in thinking.

Instead of asking whether a design is correct, you ask whether it can adapt. Can it handle new requirements without breaking? Can it evolve as data and usage patterns change? Can it support experimentation without introducing instability? Adaptability becomes the new baseline.

This doesn’t mean abandoning structure—it means designing systems that can absorb change without collapsing under it. It means embracing flexibility while still maintaining enough discipline to keep the system understandable. The goal is no longer perfection but resilience in the face of change.

Comparison table

This shift reflects a deeper change in how systems are understood.

Traditional systems were about controlling complexity. Modern systems are about navigating it. The complexity hasn’t just increased—it has changed form, becoming more dynamic and less visible.

Trade-offs that define modern systems

Modern architectures are shaped by a set of tensions that don’t have clean resolutions:

• Control vs adaptability

• Simplicity vs capability

• Performance vs cost

Control versus adaptability reflects the trade-off between predictability and flexibility. Systems that are tightly controlled are easier to reason about, but harder to evolve. Adaptable systems can respond to change, but they introduce uncertainty.

Simplicity versus capability highlights how adding features often increases hidden complexity. The system may appear simple on the surface, but internally it becomes harder to understand and maintain.

Performance versus cost has become especially critical with AI-driven systems. Faster responses often come at a higher financial cost, forcing engineers to consider trade-offs that go beyond technical metrics.

These tensions define modern architecture in ways that traditional frameworks did not fully capture.

Failure in modern systems

Failure has changed because it’s no longer always obvious when something is wrong. Systems can continue to function while gradually degrading in performance or correctness. Model drift is a clear example. A system may produce slightly worse outputs over time as underlying data changes. There’s no crash, no alert—just a slow decline in quality.

Distributed inconsistencies introduce another layer of difficulty. In event-driven systems, different components may operate on slightly different versions of state, leading to subtle bugs that are hard to reproduce.

Cascading failures are also more complex. A delay in one part of the system can propagate through others, creating issues that don’t have a single root cause. Debugging becomes less about fixing a broken component and more about understanding patterns across the system.

Misconceptions about system design today

Despite these changes, some old assumptions still persist.

One is that scaling remains the hardest problem. While scaling is still important, it’s no longer the defining challenge. Managing complexity, unpredictability, and continuous change has become equally critical.

Another misconception is that more tools simplify system design. In reality, tools often add layers of abstraction that make systems harder to understand. They solve problems, but they also introduce new ones.

There’s also the belief that systems can be fully planned up front. Modern systems evolve in ways that cannot be fully anticipated, making rigid planning less effective. These misconceptions can lead to architectures that look good on paper but struggle in real-world conditions.

The evolving role of engineers

As systems evolve, so does the role of engineers.

The job is no longer to design perfect systems. It’s to design systems that can survive change.

You’re not just building components—you’re managing behavior over time. You’re working across layers, interpreting signals, and guiding systems as they evolve.

This requires a broader perspective. You need to understand not just how systems are structured, but how they behave under real-world conditions. Engineering becomes less about control and more about stewardship.

Final words: designing for uncertainty

The discipline has changed in ways that are easy to miss until you step back and look at the bigger picture.

System design is no longer about creating stable, predictable systems that behave exactly as intended. It’s about building systems that can operate effectively in environments that are constantly changing.

You’re not designing something that will remain fixed. but rather for something that will evolve. And that means embracing uncertainty—not as a problem to eliminate, but as a reality to design for.

This kind of experience is rare in traditional systems, where behavior is a direct consequence of explicit instructions. In those systems, even complex ones, there is a comforting assumption that every output can be traced back to a specific branch, condition, or function. If something goes wrong, it is because something was written incorrectly or an edge case was overlooked. The relationship between cause and effect remains intact, even when debugging becomes tedious or time-consuming. The system may be large, but it is still fundamentally understandable.

Machine learning introduces a different kind of discomfort, one that challenges this deeply ingrained mental model. Instead of executing logic that you have written, the system begins to reflect patterns it has inferred, often in ways that are difficult to articulate or predict. The behavior becomes less about following instructions and more about interpreting data, and that shift subtly erodes the sense of control engineers are used to. The system is no longer just a deterministic machine; it becomes something closer to a statistical instrument.

Subscribe now

This is where the core question begins to emerge, not as a technical comparison but as a philosophical one: how is machine learning different from traditional programming? It is not simply a matter of tools or techniques, but a fundamental shift in how we define, build, and trust software systems. The difference lies not only in how code is written, but in how outcomes are produced, validated, and understood. Once you cross that boundary, you begin to realize that building software is no longer just about encoding logic, but about shaping behavior through data.

The deterministic world of traditional programming

Traditional programming rests on a foundation of determinism, where every possible outcome is the result of explicitly defined logic. When you write a program, you are essentially constructing a map of decisions, each one carefully designed to guide the system from input to output. Even in highly complex systems, the principle remains the same: the program does exactly what it has been told to do, nothing more and nothing less. This creates a sense of predictability that is both powerful and reassuring.

In this paradigm, developers act as architects of behavior, defining rules that govern every aspect of the system’s operation. Conditionals, loops, and functions are not just constructs; they are expressions of intent, translated directly into machine-executable form. When a system behaves unexpectedly, the assumption is that somewhere within this structure, a rule has been misapplied or omitted. The debugging process becomes an exercise in tracing logic, following the path of execution until the discrepancy is found.

This level of control allows for a high degree of confidence in system behavior, especially in domains where precision is critical. Financial systems, operating systems, and embedded programming rely heavily on this determinism, where even minor deviations can have significant consequences. The predictability of these systems is not accidental; it is a direct result of the explicitness with which they are constructed. Every edge case, every exception, is accounted for through deliberate design.

However, this approach also imposes limitations, particularly when dealing with problems that do not lend themselves to clear rules. Tasks like image recognition, natural language understanding, or recommendation systems require handling ambiguity, variation, and scale in ways that traditional logic struggles to capture. In such cases, the deterministic model begins to feel rigid, unable to adapt to the complexity of real-world data. It is here that the boundaries of traditional programming become most apparent.

The probabilistic nature of machine learning systems

Machine learning systems operate in a fundamentally different space, one defined not by explicit rules but by inferred patterns. Instead of telling the system exactly how to behave, developers provide it with data and allow it to learn relationships within that data. The resulting model does not execute a predefined set of instructions; it generates predictions based on probabilities. This introduces a layer of uncertainty that is intrinsic to the system’s behavior.

In this context, outputs are no longer guaranteed to be correct in the traditional sense. They are instead evaluated in terms of likelihood, confidence, and statistical accuracy. A model might classify an image with 92% confidence, but that remaining uncertainty is not an error in the conventional sense. It is a reflection of the model’s understanding, shaped by the data it has seen and the patterns it has internalized. This probabilistic nature changes how we interpret results, shifting the focus from correctness to performance metrics.

The implications of this shift are profound, particularly when it comes to reasoning about system behavior. In a deterministic system, you can ask why a specific output was produced and expect a clear answer. In a machine learning system, that question often leads to a more complex exploration of feature interactions, training data distributions, and model architecture. The answer is not a single line of code, but a combination of influences that are difficult to disentangle.

Shifting the burden: from logic to data

One of the most significant changes introduced by machine learning is the shift in responsibility from writing logic to curating data. In traditional programming, the quality of the system is largely determined by the correctness and completeness of the code. In machine learning, that responsibility moves upstream, into the data itself. The model can only learn what the data allows it to learn, making data quality a central concern.

This shift transforms the development process in subtle but important ways. Engineers spend less time defining rules and more time understanding datasets, cleaning inputs, and ensuring that the data reflects the problem space accurately. The focus moves from crafting algorithms to shaping the environment in which those algorithms operate. It becomes less about telling the system what to do, and more about showing it examples of what it should learn.

The implications for system behavior are equally significant. A model trained on biased or incomplete data will reflect those limitations, often in ways that are difficult to detect. Unlike code, where errors are usually explicit, data issues can manifest as subtle patterns that only emerge over time. This makes the process of building reliable systems more complex, requiring a deeper understanding of both the data and the domain.

As a result, the role of the engineer begins to evolve, blending aspects of software development with elements of data science. The boundaries between these disciplines become less distinct, as building effective systems requires both technical precision and contextual awareness. This shift challenges traditional notions of expertise, pushing engineers to think beyond code and consider the broader ecosystem in which their systems operate.

Debugging: from fixing code to interpreting behavior

Debugging in traditional systems is a relatively structured process, grounded in the assumption that errors originate from incorrect logic. When something goes wrong, the task is to identify the point of failure and correct the underlying code. Tools like debuggers, logs, and stack traces provide a clear path for investigation, allowing developers to trace the execution flow and isolate the issue. The process may be complex, but it is fundamentally linear.

In machine learning systems, debugging takes on a different character, one that is less about fixing code and more about interpreting behavior. When a model produces unexpected results, the question is not necessarily what is broken, but why the model has learned a particular pattern. This requires examining training data, feature representations, and model parameters, often without a clear starting point. The process becomes exploratory rather than procedural.

This distinction introduces a new kind of ambiguity, where it is not always clear whether an issue is a bug or a limitation of the model. A misclassification might be the result of insufficient data, poor feature engineering, or inherent ambiguity in the problem itself. Unlike traditional bugs, which can be fixed by changing code, these issues often require iterative experimentation and refinement. The concept of “fixing” becomes less precise, replaced by the idea of improving performance.

As a result, debugging in machine learning systems demands a different mindset, one that is comfortable with uncertainty and iteration. It requires thinking in terms of distributions and patterns, rather than discrete conditions and outcomes. This shift can be challenging for engineers accustomed to deterministic systems, where every problem has a clear solution. In the world of machine learning, solutions are often incremental, and understanding evolves alongside the system itself.

Control vs adaptation

Traditional programming is fundamentally about control, where developers define exactly how a system should behave under all conditions. This control is achieved through explicit logic, ensuring that every possible scenario is accounted for. The result is a system that behaves predictably, with outcomes that can be traced back to specific decisions. This level of control is essential in domains where reliability and precision are paramount.

Machine learning, on the other hand, prioritizes adaptation over control. Instead of defining behavior explicitly, developers create systems that can adjust to new data and changing conditions. This adaptability allows machine learning systems to handle complexity and variability in ways that traditional systems cannot. However, it also introduces a trade-off, as increased flexibility often comes at the cost of reduced predictability.

This trade-off is not merely technical; it reflects a deeper philosophical difference in how systems are designed. Traditional programming seeks to eliminate uncertainty by defining rules, while machine learning embraces uncertainty as a means of capturing real-world complexity. The choice between these approaches depends on the nature of the problem, as well as the acceptable level of risk and variability.

Understanding this balance is key to effectively using both paradigms. It is not a matter of choosing one over the other, but of recognizing their respective strengths and limitations. In many cases, the most effective systems combine elements of both, leveraging the control of traditional programming with the adaptability of machine learning. This hybrid approach reflects the evolving nature of software development, where flexibility and reliability must coexist.

System evolution over time

Traditional systems are inherently stable, changing only when developers modify the code. This stability is a direct consequence of their deterministic nature, where behavior remains consistent as long as the underlying logic is unchanged. While this makes systems easier to reason about, it also limits their ability to adapt to new conditions. Updates require deliberate intervention, often involving careful testing and deployment.

Machine learning systems, by contrast, are designed to evolve over time, particularly as new data becomes available. This evolution can be both a strength and a challenge, as it allows systems to improve but also introduces the possibility of drift. Model drift occurs when the data distribution changes, causing the model’s performance to degrade. Unlike traditional systems, where behavior remains static, machine learning systems must be continuously monitored and updated.

Share Grokking the Tech Career

This dynamic nature requires a different approach to system maintenance, one that emphasizes ongoing evaluation and retraining. Engineers must consider not only the current performance of the model, but also how it might change in the future. This adds a temporal dimension to system design, where behavior is not fixed but evolves alongside the data.

The concept of evolution also raises questions about ownership and responsibility, as systems become less tied to specific code changes and more influenced by external factors. Managing this complexity requires new tools and practices, as well as a shift in how engineers think about system lifecycle. It is no longer enough to build and deploy; systems must be continuously guided and refined.

The role of uncertainty in system behavior

Uncertainty is a fundamental aspect of machine learning systems, shaping how they operate and how their outputs are interpreted. Unlike traditional systems, where uncertainty is typically seen as a problem to be eliminated, machine learning treats it as an inherent feature. Predictions are expressed in terms of probabilities, reflecting the model’s confidence rather than absolute certainty.

This perspective changes how systems are evaluated and used, particularly in real-world applications. Instead of expecting perfect accuracy, stakeholders must consider trade-offs between precision, recall, and other performance metrics. Decisions are made based on likelihoods rather than guarantees, requiring a more nuanced understanding of risk and uncertainty. This shift has implications not only for engineering, but also for how systems are integrated into broader processes.

Handling uncertainty effectively requires careful design, particularly in how outputs are communicated and acted upon. Systems must be able to convey confidence levels, allowing users to make informed decisions. This adds an additional layer of complexity, as engineers must consider not only the accuracy of predictions, but also how those predictions are interpreted.

In many ways, embracing uncertainty represents a departure from traditional notions of software reliability. It challenges the idea that systems should always produce correct answers, instead focusing on producing useful ones. This shift reflects the growing importance of machine learning in domains where ambiguity is unavoidable, and where rigid logic is insufficient.

Comparison table

Traditional programming treats problems as sets of conditions that can be explicitly defined and solved through logic. Machine learning, by contrast, treats problems as patterns that can be approximated through data. This difference influences not only how systems are built, but also how they are understood and trusted.

In traditional systems, correctness is a binary property, where outputs are either right or wrong based on the defined logic. In machine learning systems, correctness becomes a spectrum, evaluated through metrics and probabilities. This shift requires a different mindset, one that is comfortable with approximation and uncertainty. It also changes how success is measured, moving from absolute correctness to statistical performance.

The philosophical divide between these paradigms reflects a broader evolution in software development, where the boundaries of what can be automated are expanding. Machine learning does not replace traditional programming, but it redefines its role, introducing new ways of thinking about systems and their capabilities.

Trade-offs between the two paradigms

The choice between traditional programming and machine learning is often framed as a trade-off between precision and flexibility. Traditional systems offer precise control over behavior, ensuring that outputs are consistent and predictable. Machine learning systems, on the other hand, provide flexibility, allowing them to adapt to complex and changing environments. This trade-off is not absolute, but it shapes how systems are designed and deployed.

Control versus learning is another key dimension, reflecting the tension between explicit rules and inferred patterns. Traditional programming excels in scenarios where rules can be clearly defined, while machine learning thrives in situations where patterns are too complex to articulate. The decision to use one approach over the other depends on the nature of the problem, as well as the desired level of control.

These trade-offs highlight the importance of choosing the right tool for the job, rather than viewing one paradigm as inherently superior. Both approaches have their place, and the most effective systems often combine elements of both. The challenge lies in understanding their respective strengths and limitations, and in applying them thoughtfully.

Subscribe now

Where traditional programming still dominates

Traditional programming remains the preferred approach in domains where reliability and predictability are critical. Systems that control physical processes, such as aviation or medical devices, require precise and deterministic behavior. In these contexts, the uncertainty inherent in machine learning is unacceptable, as even small deviations can have significant consequences. The ability to reason about every possible outcome is essential.

In addition, problems that can be clearly defined through rules are often better suited to traditional programming. Tasks involving calculations, data transformations, or well-understood workflows do not benefit from the flexibility of machine learning. In these cases, the overhead of training and maintaining models outweighs any potential advantages. Simplicity and clarity become more valuable than adaptability.

Traditional systems also offer advantages in terms of transparency and maintainability. Code can be reviewed, tested, and understood in a way that machine learning models often cannot. This makes it easier to ensure compliance with regulations and standards, particularly in industries where accountability is critical. The explicit nature of traditional programming provides a level of assurance that is difficult to replicate with machine learning.

Where machine learning becomes essential

Machine learning becomes indispensable in scenarios where patterns are too complex or too numerous to be captured through explicit rules. Tasks like image recognition, speech processing, and recommendation systems involve high-dimensional data and subtle relationships that are difficult to encode manually. In these cases, machine learning provides a way to approximate solutions that would otherwise be infeasible.

The ability to scale with data is another key advantage, allowing machine learning systems to improve as more information becomes available. Unlike traditional systems, which require manual updates to incorporate new knowledge, machine learning models can adapt through retraining. This makes them particularly well-suited for dynamic environments where conditions change over time.

Machine learning also enables new kinds of applications, particularly those involving personalization and prediction. By analyzing user behavior and preferences, systems can tailor their outputs to individual needs. This level of customization is difficult to achieve with traditional programming, where rules must be defined in advance. The flexibility of machine learning allows for more responsive and adaptive systems.

These capabilities have driven the widespread adoption of machine learning across industries, from healthcare to finance to entertainment. While not a universal solution, it has expanded the range of problems that software can address. Understanding where it is most effective is key to leveraging its potential.

Leave a comment

Common misconceptions

There is a common belief that machine learning will eventually replace traditional programming entirely, rendering explicit logic obsolete. This perspective overlooks the complementary nature of the two paradigms, as well as the continued importance of deterministic systems. Machine learning extends the capabilities of software, but it does not eliminate the need for clear and precise logic.

Another misconception is that more data always leads to better models, regardless of its quality or relevance. In reality, the effectiveness of a machine learning system depends heavily on the quality of the data it is trained on. Poor data can lead to misleading patterns, resulting in systems that perform poorly or behave unpredictably. Quantity alone is not sufficient; context and accuracy are equally important.

There is also a tendency to view machine learning systems as inherently smarter or more advanced than traditional systems. While they can handle complex tasks, they are still limited by their training data and design. They do not possess understanding in the human sense, but rather operate within the constraints of their learned representations. This distinction is often overlooked, leading to unrealistic expectations.

Clarifying these misconceptions is important for developing a balanced understanding of machine learning. It is a powerful tool, but it is not a panacea. Recognizing its limitations is as important as appreciating its strengths, particularly as it becomes more integrated into software systems.

The future of software development

The rise of machine learning is reshaping the role of the software engineer, blurring the lines between programming and data science. Engineers are increasingly required to think not only about code, but also about data, models, and system behavior over time. This shift is changing how systems are designed, tested, and maintained, introducing new challenges and opportunities.

Hybrid systems that combine traditional programming with machine learning are becoming more common, leveraging the strengths of both approaches. These systems use deterministic logic to handle well-defined tasks, while relying on machine learning for more complex or ambiguous components. This integration reflects a more nuanced understanding of software development, where different paradigms are applied as needed.

Looking ahead, the distinction between these paradigms may become less pronounced, as tools and frameworks evolve to bridge the gap. However, the underlying philosophical differences will remain, shaping how engineers approach problem-solving. Understanding these differences is essential for navigating the future of software development.

Redefining what it means to build software

At its core, the question of how machine learning differs from traditional programming is not just about techniques, but about perspective. It challenges the idea that software is simply a collection of rules, replacing it with a more fluid and adaptive model of behavior. This shift requires rethinking not only how systems are built, but also how they are understood and trusted.

Machine learning introduces a new way of thinking about software, one that embraces uncertainty and complexity rather than trying to eliminate them. It expands the boundaries of what can be automated, while also redefining the role of the engineer. Building software becomes less about writing instructions and more about shaping systems that learn and evolve.

In this sense, machine learning does not replace traditional programming, but transforms it, adding a new dimension to the craft. It invites engineers to think differently, to question assumptions, and to explore new possibilities. And in doing so, it changes not just how we build software, but how we think about it altogether.

These systems do not present themselves as “machine learning features,” but rather as seamless experiences that feel intuitive and responsive. The intelligence behind them is abstracted away, integrated deeply into the infrastructure that powers user interactions. This invisibility is not accidental but a result of deliberate system design, where machine learning is treated as a background capability rather than a standalone component. Engineers must think beyond models and consider how these systems integrate into broader architectures.

Share

Machine learning is not simply applied to isolated problems; it is woven into the fabric of systems that manage data, drive decisions, and shape user experiences. Each application is part of a larger pipeline that includes data ingestion, feature processing, model inference, and feedback loops. These pipelines operate continuously, adapting to new data and evolving conditions. The real-world impact of machine learning lies not in individual predictions but in how these predictions influence system behavior over time. Understanding this requires a shift from thinking about models to thinking about systems.

Machine learning as an invisible system layer

As machine learning becomes more pervasive, it also becomes more complex, introducing challenges that extend beyond algorithmic performance. Systems must handle large-scale data processing, ensure low-latency responses, and maintain reliability under varying conditions. Engineers must design architectures that can support these requirements, balancing performance with cost and maintainability. The integration of machine learning into production systems requires careful orchestration of multiple components, each with its own constraints and dependencies. This orchestration defines how machine learning operates in practice, shaping its effectiveness and impact.

Ultimately, machine learning serves as an invisible layer that enhances system capabilities, enabling applications to adapt and respond dynamically. Its presence is felt through improved personalization, smarter decision-making, and more efficient operations. However, this integration also demands a deeper understanding of how systems behave under real-world conditions. Engineers must design systems that can harness the power of machine learning while managing its complexity, ensuring that it contributes to overall system performance.

The shift from models to systems

The evolution of machine learning from isolated models to integrated systems represents one of the most significant shifts in modern software engineering. Early applications of machine learning focused primarily on model development, where the primary goal was to achieve high accuracy on a specific task. However, as these models moved into production, it became clear that their performance depended not only on their predictive capabilities but also on how they were integrated into larger systems. Models must now operate within pipelines that handle data ingestion, preprocessing, inference, and feedback, creating a complex ecosystem of interdependent components. Engineers must design these systems to ensure that models can function effectively in real-world environments.

This shift introduces new challenges that extend beyond traditional machine learning concerns, particularly in terms of scalability and reliability. Systems must handle large volumes of data, process it efficiently, and deliver predictions with minimal latency. This requires robust infrastructure that can support distributed systems processing and real-time decision-making. Engineers must also consider how models interact with other system components, ensuring that data flows smoothly and that dependencies are managed effectively. The complexity of these interactions often determines the success or failure of machine learning systems in production.

Another critical aspect of this transition is the role of feedback loops, which allow systems to learn and adapt over time. Unlike static models, production systems must continuously update their understanding based on new data, requiring mechanisms for monitoring and retraining. These feedback loops introduce additional complexity, as they must be carefully designed to avoid instability or unintended behavior. Engineers must ensure that updates improve system performance without introducing new issues.

The shift from models to systems highlights the importance of holistic design, where machine learning is treated as part of a broader architecture. Engineers must consider how models fit into the system, how they interact with data and infrastructure, and how they evolve over time. This perspective is essential for building systems that can leverage machine learning effectively.

What Are Real-World Applications of Machine Learning Today?

Understanding real-world applications of machine learning requires moving beyond individual use cases and examining how these systems operate across industries. Machine learning is not applied in isolation but as part of larger systems that manage data, automate decisions, and enhance user experiences. Each application reflects the specific needs and constraints of its domain, shaping how machine learning is integrated and deployed. Engineers must consider how these systems function as a whole, rather than focusing on individual components.

Different industries leverage machine learning in distinct ways, reflecting the diversity of challenges they face. In e-commerce, machine learning drives personalization and recommendation systems, while in finance, it supports fraud detection and risk analysis. Healthcare applications focus on diagnostic support and predictive analytics, while autonomous systems rely on machine learning for perception and decision-making. Despite these differences, all applications share a common foundation in data-driven decision-making. Engineers must design systems that can process and interpret data effectively, enabling machine learning to deliver meaningful insights.

The integration of machine learning into these systems requires careful consideration of infrastructure, data pipelines, and user interactions. Models must be deployed in environments that can support their computational requirements, while data pipelines must ensure that inputs are accurate and up-to-date. User-facing applications must present predictions in a way that is intuitive and actionable, bridging the gap between machine learning and human decision-making. Engineers must design systems that can handle these complexities while maintaining performance and reliability.

Ultimately, real-world applications of machine learning are defined by their ability to integrate into existing systems and enhance their capabilities. This integration requires a deep understanding of both machine learning and system design, ensuring that applications can operate effectively in production environments.

Personalization and recommendation systems

Personalization systems represent one of the most visible applications of machine learning, yet their complexity is often underestimated because of how seamlessly they integrate into user experiences. These systems operate by continuously analyzing user behavior, preferences, and interactions to generate tailored recommendations. Unlike static systems, personalization engines must adapt in real time, updating their predictions as new data becomes available. This requires a combination of data pipelines, feature engineering, and model inference, all operating within a distributed environment. Engineers must design systems that can handle this continuous flow of information while maintaining low latency.

The core challenge in recommendation systems lies in balancing relevance with diversity, ensuring that users are exposed to content that is both interesting and varied. Overemphasis on past behavior can lead to echo chambers, where users are repeatedly shown similar content, reducing engagement over time. Engineers must design algorithms that can introduce novelty while maintaining accuracy, requiring careful tuning of model parameters and feedback mechanisms. This balance is critical for maintaining user engagement and satisfaction.

Scalability is another significant challenge, as recommendation systems must handle millions or even billions of users and items. The system must process large volumes of data efficiently, ensuring that recommendations are generated quickly and accurately. This requires distributed computing frameworks and optimized data storage solutions, enabling the system to scale horizontally. Engineers must design systems that can handle these demands while maintaining performance.

Bias and fairness also play a crucial role in personalization systems, as recommendations can inadvertently reinforce existing patterns or exclude certain groups. Engineers must consider these factors when designing systems, ensuring that recommendations are equitable and inclusive. This requires ongoing monitoring and adjustment, as biases can emerge over time.

Share Grokking the Tech Career

Fraud detection and risk analysis systems

Fraud detection systems represent a critical application of machine learning, where the goal is to identify anomalous behavior that may indicate fraudulent activity. These systems operate in real time, analyzing transactions and user behavior to detect patterns that deviate from normal activity. Unlike traditional rule-based systems, machine learning models can adapt to new patterns, improving their ability to detect fraud over time. Engineers must design systems that can process data quickly and accurately, ensuring that fraudulent activity is identified before it causes significant damage.

The challenge in fraud detection lies in balancing sensitivity and specificity, as overly aggressive models can generate false positives, while conservative models may miss fraudulent activity. Engineers must carefully tune models to achieve the desired balance, considering the impact of errors on both users and the system. False positives can lead to user frustration, while false negatives can result in financial losses. This trade-off is a central consideration in system design.

Real-time processing is essential for fraud detection systems, as delays can reduce their effectiveness. The system must analyze transactions as they occur, requiring low-latency data processing and model inference. Engineers must design infrastructure that can support these requirements, ensuring that decisions are made quickly and accurately. This often involves distributed systems and optimized data pipelines.

Fraud detection systems must also adapt to evolving threats, as fraudsters continuously develop new techniques to bypass detection. Engineers must design systems that can learn from new data, updating models and rules to address emerging patterns. This requires robust feedback mechanisms and continuous monitoring, ensuring that the system remains effective over time.

Comparison of system approaches

The comparison illustrates how machine learning fundamentally changes system behavior, particularly in terms of adaptability and scalability. Rule-based systems rely on predefined logic, making them easy to implement but difficult to adapt to changing conditions. Traditional statistical systems offer some flexibility but are limited in their ability to handle complex patterns. Machine learning systems, by contrast, can learn from data and adapt over time, enabling them to handle a wide range of scenarios.

However, this adaptability comes at the cost of increased complexity, as machine learning systems require sophisticated infrastructure and ongoing maintenance. Engineers must design systems that can manage this complexity, ensuring that models remain accurate and reliable. The trade-offs between simplicity and adaptability are central to system design, influencing how machine learning is integrated into production systems.

The comparison underscores the importance of understanding how different approaches impact system behavior, guiding engineers in choosing the appropriate solution for their needs.

But what’s the best way to actually get hands-on with AI skills?

Building projects.

Here’s why:

• Developers learn by building. Passive learning isn’t fruitful, active learning is.

• Employers see your portfolio as proof. A project on your portfolio beats a keyword on your resume any day. It shows you can build with a technology before you even interview.

That’s why today I’ll be sharing my top 6 Educative Projects for AI upskilling. These selections are based on the technologies my colleagues and I are prioritizing as we hire in 2026.

If you’re unfamiliar with our Projects, they’re our solution to help developers build production-grade projects while they learn.

They include:

• Walkthroughs for portfolio-ready projects

• Code and build as you go, all inside your browser

• No production set up or third party access needed

And for one more day, you can access these projects with a discount of at least 50% off.

Let’s jump in.

1. Build a Chatbot with OpenAI GPT-5 Using Flask

Technologies: Flask, Python, OpenAI

Build practical skills in backend development and AI integration using GPT-5 — one of the most advanced models available today.

You’ll build a chatbot with Flask, connect it to the GPT-5 API, and generate intelligent, context-aware responses. You’ll also implement chat history tracking so the conversation stays coherent across messages (just like real production apps do).

👷 Who it’s for: Developers who want to build smarter AI-powered apps with real conversational memory.

👉 Build a Chatbot with OpenAI GPT-5 Using Flask

Want a Flask project that leverages DSPy too? Check out: Build an AI Financial Analyst Assistant with DSPy and Flask.

Screenshot from Build an AI Financial Analyst Assistant with DSPy and Flask

2. FREE: Build a Multi-Agent Job Search System with CrewAI and Python

Technologies: CrewAI, LangChain, Python, Streamlit, Gemini

If you’re on the job hunt, this project is a 2-for-1.

You’ll come out with:

• An AI project for your portfolio

• And a job search system that saves you time (and sanity)

You’ll define specialized agents to:

• Search job postings

• Generate tailored resumes and cover letters

• Create outreach messages

…And connect them into a working Crew to streamline your own job hunt.

This project is your hands-on introduction to agent orchestration and workflow design. Best of all: it’s free.

👷 Who it’s for: Developers who want to build AI chops + save time during the job hunt.

👉 Build a Multi-Agent Job Search System with CrewAI and Python

Interested in another CrewAI + LangChain project?

Check out: Build an Agentic Workflow with CrewAI and GitHub MCP Server Tools.

3. Explore DeepSeek Models via OpenRouter

Technologies: OpenRouter API, Jupyter Notebook, DeepSeek, Python

This project teaches you how to integrate multiple DeepSeek models into a Python workflow using the OpenRouter API, so you can build smarter tools that go beyond a single prompt.

Inside a Jupyter Notebook, you’ll tackle tasks like summarizing research, writing support emails, generating social content, and solving code problems.

Along the way, you’ll explore essential skills for any modern AI developer, including:

• Prompt engineering

• Token management

• Model chaining

👷 Who it’s for: Developers who want to move from prompting to building with large language models.

👉 Explore DeepSeek Models via OpenRouter

4. Develop a Multi-Agent Research Assistant Using AutoGen

Technologies: AutoGen, Python, OpenAI API

Multi-agent systems like AutoGen are redefining what AI can do — not just answering questions, but coordinating complex tasks across specialized roles.

In this project, you’ll use AutoGen to build a research assistant that automates the full pipeline of academic exploration: refining topics, finding papers, extracting insights, and compiling structured reports.

You’ll get experience with skills like:

• Agent collaboration

• Sequential task execution

• Custom workflow design

… All while building a portfolio-ready tool for research automation.

👷 Who it’s for: Developers interested in automating complex workflows and building AI tools for knowledge work.

👉 Develop a Multi-Agent Research Assistant Using AutoGen

5. Build a RAG Chatbot Using DeepSeek and LlamaIndex

Technologies: DeepSeek, LlamaIndex, ChainLit, React

RAG is one of the most effective ways to reduce hallucinations in LLMs by grounding answers in real sources instead of just guessing. In this project, you’ll build a chatbot that does exactly that: fetching up-to-date content from Wikipedia to generate accurate, context-aware responses.

In this project, you’ll:

• Create an interactive chat interface using ChainLit

• Connect to Groq-hosted LLMs like DeepSeek or LLaMA 3 for ultra-fast inference

• Rely on LlamaIndex for indexing and retrieval

By the end, you’ll have a fully functional ReAct-based agent that can research, reason, and respond using live data.

👷 Who it’s for: Developers who want to build faster, smarter AI systems that actually know what they’re talking about.

👉 Build a RAG Chatbot Using DeepSeek and LlamaIndex

Want a RAG project that gets you hands-on with FastAPI? Check out: Building a Retrieval-Augmented Generation System Using FastAPI

6. Build an AI Travel Planner with Multi-Agent A2A Protocol

Technologies: Gemini, Python

Screenshot from Build an AI Travel Planner with Multi-Agent A2A Protocol

Agent-to-Agent (A2A) collaboration is quickly becoming one of the hottest trends in AI — powering more dynamic, modular, and scalable systems that go far beyond single-model solutions.

In this project, you’ll explore that frontier by building an AI travel planner where specialized agents (flights, hotels, attractions, weather) coordinate through a central root agent to plan trips intelligently.

You’ll learn how to:

• Use Google’s Agent Development Kit (ADK) to create role-based agents

• Register and delegate tasks through root agents

• Run asynchronous, CLI-based agent systems in Python

👷 Who it’s for: Developers exploring the cutting edge of multi-agent AI — and looking to apply it to real, high-value use cases.

👉 Build an AI Travel Planner with Multi-Agent A2A Protocol

Build your future one project at a time

With new models and tools emerging every week, there’s a lot of noise in the AI space right now.

But what matters most is what you can build with it.

That’s the throughline across every project I shared today: moving from knowing to doing. From “I’ve read about that” to “I built this.”

You won’t be the only one. Last year alone, developers spent over 5,000 hours building with Educative Projects — leveling up, shipping portfolio-ready work, and learning by doing.

If you’re serious about futureproofing your skills in 2026, this is the way forward. Pick a project, get your hands dirty, and let the work teach you.

We built Projects for exactly this kind of learning — and I hope one of these gives you the spark to start. You can explore our entire catalog of Projects here.

Last call: Access any of these Projects with 50% off an Educative subscription during our Resolutions Sale. Offer ends Thursday.

Get 50% off (or more)

What’s your resolution this year — and how can I help you get there? Hit reply and let me know.

Until then, happy learning!

— Fahim

That question is forcing a reset across the industry. AI allows us to produce functional code in a matter of seconds, making it easier than ever to “look good” on traditional interview formats. At the same time, companies still need engineers who can ship reliable systems, work well with others, and make good decisions under uncertainty.

This week, we’re predicting how developer hiring will continue to shift in 2026 and what that signals about hiring in the AI era.

Before we dive in: our New Year’s Resolution Sale is live for just a few more days. You can lock in at least 50% off an Educative Unlimited subscription, which includes everything from System Design to Generative AI, plus hands-on practice tools like Cloud Labs and Projects.

👉 Don’t be late to the party — use this discount while you still can.

Now, let’s get into those predictions.

1) Interview formats will adapt for AI

Traditional coding interviews weren’t built for a world where ChatGPT can ace the test.

LeetCode-style questions reward pattern matching, which AI is particularly good at. This forces companies to change what they measure — or accept that the interview is now mostly testing who knows how to use a tool.

So interview formats will shift to test engineers for strong reasoning, so they can use AI responsibly:

• Reviewing and debugging AI code

• Designing around AI’s unpredictability in System Design

• Knowing when to use AI (and when not to)
  

A lot of interviews will also start allowing AI tools explicitly. Some teams will even expect it. While anti-AI proctoring will continue to show up, it tends to backfire. Lockdowns and surveillance erode trust, and they select for candidates willing to tolerate an adversarial process. The better approach is open-book: allow tools, then test judgment.

What this means: interview success shifts from “fastest solution” to “best reasoning.” Strong candidates will stand out by how well they review AI output, spot mistakes, and explain what they’d ship and why.

👉 If your interviews start looking like debugging, review, and System Design, your prep should match that. Educative’s AI Mock Interviews and interactive lessons help you practice talking through decisions, not just writing code under pressure. That’s closer to the direction interviews are headed.

2) AI fluency is the new baseline

This year, AI skills will spread through job descriptions the way “Git” and “cloud” did a decade ago — because it’s no longer a specialty.

But AI fluency doesn’t mean you need to learn niche skills like fine-tuning or model training. More often, we’ll see companies looking for engineers who can:

• Build with AI tools responsibly

• Understand model failure modes

• And integrate AI into workflows with guardrails
  

Expect more “AI-first” job descriptions, especially from companies building internal copilots, agent workflows, and AI features into mature products.

3) “Harder languages” will grow in demand

When AI increases code surface area, companies care more about correctness, safety, and performance. Rust, Go, Kotlin, Zig — these become signals of discipline and reliability.

So we saw languages that prioritized safety and clarity surge:

• Rust became a default for performance-critical tooling.

• Kotlin overtook Java in enterprise backends.

• Python, once written off as glue code, earned more respect — not for scale, but for speed, readability, and LLM prototyping.

• Zig gained traction among devs tired of “magic.”

These languages are signals of discipline and reliability, not just taste.

4) Communication becomes a much bigger part of the hiring signal

In a saturated market where candidates share tech skills, communication sets you apart. Besides, AI can already handle syntax. What makes you a unique contributor is how you think and how you explain.

This means hiring teams will place greater weight on:

• How you collaborate,

• How you justify tradeoffs,

• How you talk through uncertainty,

• How clearly you can explain a design decision.
  

Everyone knows that behavioral interviews assess these.

But where most people underestimate communication is where it matters most: System Design interviews. This interview doesn’t have a single right answer, but how you get there — and communicate along the way — is paramount to success.

👉 What this means: Mastering System Design is one of the best investments you can make right now. Courses like Grokking the Modern System Design Interview help you build a repeatable way to reason about systems, tradeoffs, and failures — skills that show up in interviews and in real work.

5) Hiring competition extends beyond borders

AI accelerates work, but it also broadens the talent pool. Remote teams keep expanding, and companies are sourcing more globally.

This increases competition, which means you need to think more critically about your “leverage.” Companies will favor candidates who can demonstrate scarce skills, so your best bet is to highlight AI, System Design, and production readiness.

Visa dynamics also matter. H-1B and other talent flows make AI fluency a serious edge for candidates who want stability in uncertain (and uber competitive) hiring environments.

One last note: The “bubble vs. boom” debate misses the point

People love arguing whether AI is a bubble. But that debate doesn’t help you prepare.

Even if parts of the market cool, AI investment still drives hiring. Even if some products collapse, the infrastructure, tooling, and workflows keep moving forward. A “good bubble” can still reshape how engineering work is done for years.

So here’s the pragmatic view: AI hype creates opportunities, and stability comes from fundamentals.

The engineers who win in 2026 won’t be the ones chasing every tool. They’ll be the ones building durable skills: System Design thinking, debugging instincts, communication and collaboration, and AI fluency that translates into shipping.

So if you want to stay competitive in 2026, do this:

1. Practice interviews that look like real work (debugging, review, reasoning).

2. Get fluent with AI tools without outsourcing your judgment.

3. Build System Design instincts so tradeoffs feel natural.

4. Strengthen communication — it’s increasingly part of the signal.
   

And if you want a clean way to start the year: our New Year’s Resolution Sale is live for just a few more days. You can lock in 50% off Educative Unlimited and get access to our full catalog plus hands-on practice tools that match what hiring is evolving toward.

Get 50% Off (Or More)

Disagree with any of these shifts? Hit reply and tell me why.

Until then, happy learning!

— Fahim

It was an eventful and turbulent year: open-source went rogue, IDEs got eaten, the JS ecosystem hit a wall, and infra cracked under the weight of real-time inference.

But through it all, developers kept building.

This is the year in review: 7 takeaways that defined 2025 (and what they say about the year ahead).

1. Open Source Went Rogue

👉 TL;DR: Open-source AI stopped asking for permission. It reverse-engineered, optimized, and out-shipped the incumbents.

In January 2025, DeepSeek stole the show as the Chinese startup dropped R1 — an open-source reasoning model that rivaled GPT-4… without access to NVIDIA’s top chips, and allegedly, without OpenAI’s permission either.

Built in weeks, trained under U.S. chip sanctions, and shipped to millions overnight, R1 shattered assumptions:

• You don’t need access to frontier hardware

• You don’t need a billion-dollar budget

• You might not even need your own weights

Speculation swirled about whether DeepSeek leaked, laundered, or reverse engineered its way to the top… But it didn’t matter. Within days of its launch, DeepSeek became the #1 downloaded app in the US.

And the rest of the open-source AI community made major moves in 2025:

• Mistral’s 7B model went toe-to-toe with GPT-3.5 — and became the go-to small model stack.

• WHAMM let devs run GPT-4-class models in a browser tab (until it melted GPUs).

• ​​Cloud GPU access got more democratized — with services like Modal, Replicate, and Brev.dev removing the “Azure” tax from building with AI.

• Self-hosting, quantization, and flash-distillation became default workflows.

• OpenDevin, CrewAI, AutoGen, and LangGraph showed up with open-source agent frameworks.

The mood shifted. Open source wasn’t imitating anymore. It was infiltrating (and sometimes, outpacing).

2. Tools Either Grew Up or Got Rewritten

👉 TL;DR: JavaScript tools hit their ceiling. Rust and Go became the new baseline for anything that needed to move and scale fast.

In 2025, Microsoft announced it was rewriting the TypeScript compiler in Go because the old version, built in TS and running on Node.js, couldn’t keep up with modern codebases. The limitations were obvious: single-threaded execution, memory bloat, and sluggish performance in monorepos and CI pipelines.

But this was about more than just TypeScript.

JavaScript tooling — once good enough by default — had finally hit its ceiling.

Developers now expected sub-second feedback from AI copilots, blazing-fast local builds, and toolchains that could scale with them — not slow them down.

So 2025 became a purge of the old stack:

• Create React App was officially deprecated.

• Turbopack (Rust) replaced Webpack in production-grade Next.js apps.

• RomeJS shut down after missing performance targets.

• AI-native tools like Cursor demanded native-speed response times (not interpreted bottlenecks).

Meanwhile, Go and Rust became the new foundations for dev tooling at scale.

Whether it was compilers, bundlers, or AI-augmented editors, speed mattered — and interpreted runtimes couldn’t deliver.

3. AI Fluency > Seniority

👉 TL;DR: AI-native engineers led hiring trends — even as layoffs continued.

2025 was the year AI became your co-worker. And if you couldn’t work with it, you were first on the layoff spreadsheet.

Big Tech slashed roles aggressively: Meta cut mid-level engineers, Amazon axed frontend teams, and Google eliminated platform teams while scaling its internal LLM infra.

But hiring didn’t stop. Jobs opened up for those with AI skills. Companies wanted engineers who could:

• Fine-tune open models

• Deploy LLMs in real products

• Build internal tools where AI wasn’t an add-on — it was the foundation

While talent left big tech, other industries — from healthcare to logistics — got to hire great engineers to implement AI features from scratch. Meanwhile, legacy apps had to be rebuilt for the AI era in The Great Rewrite: a multi-year wave of architectural change that will be underway for years to come.

Engineering leaders already deemed AI skills a necessity back in 2024, when 71% of leaders reporting they’d rather hire a candidate with AI fluency than seniority.¹

And by mid-2025, 36% of tech job postings required AI skills — mostly weighing on engineering roles.²

❓ What’s your AI upskilling plan? Whether it’s LangChain or CrewAI, explore our catalog at Generative AI hub.

4. Developers Demand AI That Knows Their Codebase

👉 TL;DR: Context > horsepower. Devs don’t just want the smartest model in the room anymore. They want the one that remembers where db.ts lives.

Anthropic’s MCP turned heads by delivering project-aware completions and code navigation that felt like it actually read your repo. It was like autocomplete… with context.

Meanwhile, Claude Opus 4.5 “compaction” feature self-summarizes previous interactions to maintain useful long-term memory.

That same mindset showed up across the board:

• Cody (Sourcegraph) indexed entire codebases and brought Git-level awareness to AI answers.

• Cursor let devs chat with their own code and watch the model edit real files in-place.

• Agent frameworks like CrewAI and OpenDevin started building smarter memory and context pipelines by default.

By the end of 2025, devs realized it’s not just about how smart a model is — it’s how well it understands your code too.

5. Invisible Infra Failed (and AI Exposed Its Limits)

👉 TL;DR: 2025 proved that traditional infra patterns can’t handle AI at scale. AI-native System Design went from niche skill to necessity.

This year saw some major outages:

• A tiny DNS bug at AWS triggered a cascading outage that brought down massive swaths of the internet.

• A misconfigured Cloudflare update nearly derailed Black Friday traffic across the globe.

These failures were largely linked to old automation problems hitting hyperscale.

But the most important infra lesson of 2025 came from AI-related failures.

In June, ChatGPT and Sora suffered the longest outage in company history (one of four major failures this year). OpenAI was scaling GPU-bound, stateful, real-time inference for millions — and most cloud infrastructure wasn’t built for that. The system simply hit its architectural limits.

Designing for Generative AI is a unique territory — and we’re charting it as we go.

GenAI workloads are:

• Expensive (GPU-hungry and time-sensitive)

• Stateful (every session matters)

• Uncacheable (every response is unique)

• Fragile under traffic (load-balancing becomes a chokepoint)

As GenAI swept throughout tech, we needed new skills and practices that actually worked for Gen AI workloads.

So AI System Design became a real discipline:

• Devs had to learn how to design for latency-aware routing, session migration, token-aware caching, and inference pipelines

• Infra teams started asking: “Can our system scale for GenAI — not just for users?”

• AI systems required new design patterns, not just bigger clusters

👀 By no coincidence, one of our most popular new courses this year was Grokking the Generative AI System Design.

6. The IDE got eaten by agents

👉 TL;DR: The IDE got absorbed into something faster, smarter, and a little bit feral.

Traditional IDEs couldn’t keep up with agentic workflows.

Developers migrated en masse to:

• Cursor

• Copilot Workspace

• Gemini CLI

• AI-native REPLs and shells

These tools went way beyond autocomplete. They debugged, explained, and occasionally rewrote your whole function without asking.

The file stopped being the center of gravity, and the agent became the dev environment.

Across the board, AI started making tiny steps toward getting baked into our interfaces:

• WHAMM let devs run every major model in a single browser tab.

• Google’s Generative UI hinted at a world where apps scaffold themselves in real time, adapting layouts, flows, and content dynamically based on user goals.

• GTA 6 previewed NPCs powered by LLMs, with emergent behaviors, memory, and interaction that felt alive.

Soon, AI will be the interface — fluid, dynamic, and co-designed in real time.

7. Rise of AI-Native Learning

👉 TL;DR: The best learning isn’t from tutorials, but real-time explainers, AI reviewers, and embedded walkthroughs.

AI collapsed the distance between idea and code, but it also widened the gap between code and comprehension. Devs can essentially generate code faster than they can understand it. That’s a problem.

That’s why we’re seeing a shift toward AI-native learning — not just content about AI, but content powered by it. Learning tools are being rearchitected around AI with real-time code reviewers, adaptive walkthroughs, and in-situ coaching that responds to each learner’s level.

Here’s how that’s playing out:

• Long-form video courses are fading. Interactive, contextual sandboxes are rising.

• Static docs are being replaced by smart explainers, walkthroughs, and in-situ AI coaching.

• AI-powered mock interviews are making interview prep more accessible than ever.

⏰ These trends were reflected in our learning community at Educative, where developers spent:

• 403,521 hours learning with alternative explanations and coaching in AI-powered courses

• 79,465 hours getting adaptive curriculum for interview prep via our Personalized Interview Prep tool

• 5,260 hours building portfolio-worthy projects with new tools in guided Projects

• 1,979 hours getting real-time feedback and interview practice in AI-powered Mock Interviews

AI-native learning is a new contract between developers and their tools: Learn faster with context, and get feedback immediately. That’s what we’re committed to delivering at Educative.

So What’s Next?

In 2026, we’ll be asking the harder questions about AI — whether the right models are chosen, the architecture actually supports them, or the systems we’re building can hold up under real-world pressure.

We’ll see:

• AI fluency as a baseline job requirement: Job postings will demand prompt engineering, fine-tuning, and agent workflows.

• Context-aware dev tools become standard: IDEs and agents that deeply understand your codebase, memory, and file structure.

• AI-native infrastructure will be mandatory: More teams will shift toward infra that’s optimized for GenAI, with a growing focus on latency, state, and fault-tolerant inference pipelines.

• Agents will creep into interfaces: Expect more tools, UIs, and even games to shift from static flows to dynamic, LLM-powered agents.

We look forward to keep learning and evolving with you in the year ahead.

With all the volatility that rocked 2025, I wanted to say thank you for reading, building, and learning with us. It’s an honor to be a part of your growth as the industry keeps changing beneath our feet, and we don’t take that lightly.

👉 For just a couple more days, you can lock in 50% off (or more) any Educative subscription to get AI-native learning for your New Year’s Resolutions.

Happy learning, and Happy New Year!

- Fahim

As we wrap up the year, I’m looking ahead to AI tools you can’t afford to skip in 2026. AI is moving at lightning speed — and I urge you to make AI skills a priority if you haven’t yet.

Microsoft and LinkedIn’s 2024 Work Trend Report found that 77% of leaders would rather hire a less experienced candidate with AI skills than a seasoned one without.

That was in 2024. Judging from my conversations with other leaders in the software industry, I can say for certain that in 2026, this number will go up dramatically.

So as we head into the new year, I’ve pulled together a forward-looking stack — the non-negotiable skills I think should be at the top of your wishlist if you want your systems (and your career) to age gracefully.

For each tool, I’ll cover:

• Use cases

• Why it matters

• My take from a hiring and leadership lens

I’ll also share my favorite courses and projects for each tool, so you can start building with the tools that will define the future.

Let’s dive in.

1. OpenAI Agent Stack

🧠 Use cases: Personalized AI copilots, automated workflows, support bots, and prototyping full-stack assistants—without rebuilding orchestration from scratch.

👨‍💻 Why you need it:

With persistent threads, tool use, memory, and system-wide orchestration, OpenAI has quietly shipped the blueprint for next-gen agents.

If you’re looking to build agents that plan, adapt, and execute, this stack gives you a serious head start, without requiring a custom orchestration layer.

💼 Hiring manager’s perspective:
When someone talks fluently about Assistants, memory, or tool calling, I know they’ve gone beyond playing with ChatGPT. They’ve touched real architecture. That’s the difference between AI hobbyists and engineers I’d trust to ship a production-ready assistant.

📚 Get hands-on with OpenAI:

• Course: Building with OpenAI: From APIs to Agents

• Project: Deep Research Through an AI Agent Using OpenAI

2. Claude Code

🧠 Use cases: Code review, bug explanation, writing tests, refactoring modules, generating docs from code, and multi-file planning.

👨‍💻 Why you need it:
Claude 3.5 and Opus 4.5 have become standout tools for developers who want more than just code completion. Claude’s ability to process large codebases, reason structurally, and explain tradeoffs makes it incredibly effective as a debugging partner and design reviewer.

It’s the rare model that feels like a thoughtful colleague — and when paired with an IDE like Cursor, it becomes a serious engineering multiplier.

💼 Hiring manager’s perspective:
Developers who’ve used Claude for real work often approach problems more clearly — not just generating code, but articulating systems. That clarity is gold in interviews, architecture reviews, and team communication.

📚 Get hands-on with Claude Code:

• Claude Code: Workflows and Tools

3. Google Agent2Agent (A2A) Protocol

🧠 Use it for: Scalable multi-agent workflows, agent coordination, collaborative AI tasks.

👨‍💻 Why you need it:
A2A is Google’s protocol for agent collaboration (think of it as gRPC for intelligent systems). It lets agents delegate tasks, share memory, and interact across environments with persistent context and roles. As agent-based systems become more modular and distributed, A2A is laying the foundation for large-scale coordination.

It’s not flashy, but it’s the infrastructure that will power complex agent ecosystems.

💼 Hiring manager’s perspective:

If someone brings up A2A, I know they’ve tried to scale agents beyond the toy level. That tells me they’ve thought about orchestration, not just outputs… and that’s the kind of thinking we need more of on real-world AI teams.

📚 Get hands-on with A2A:

• Course: Introduction to Google’s Agent2Agent (A2A) Protocol

• Project: Build an AI Travel Planner with Multi-Agent A2A Protocol

4. CrewAI

🧠 Use it for: Multi-agent swarms, task orchestration, intelligent pipelines.

👨‍💻 Why you need it:
Most multi-agent frameworks look great in demos but collapse under real complexity. CrewAI is built differently: it’s composable, Python-native, async-ready, and surprisingly stable at scale. It gives you a structured way to build agents that work together like a team: planning, delegating, and executing workflows without stepping on each other.

💼 Hiring manager’s perspective:
When I see someone bring up CrewAI in a conversation or interview, it tells me they’ve graduated from prompt experiments to systems thinking. They’ve taken the initiative to make agents work together, and that experience shows up quickly in how they talk about coordination and failure modes.

📚 Get building with CrewAI Projects:

• Build an Agentic Workflow with CrewAI and GitHub MCP Server Tools

• Build a Multi-Agent Job Search System with CrewAI and Python

5. MCP (Model Context Protocol)

🧠 Use it for: Shared memory, persistent state, multi-agent context-passing.

👨‍💻 Why you need it:

MCP solves one of the hardest problems in AI workflows: how do you make context portable?

With MCP, agents can persist memory, share state, and stay goal-aligned across tools and time. It enables everything from long-running assistants to multi-agent collaboration to stateful RAG systems.

If you’re stitching together multiple AI components, MCP is the memory bus that holds it all together.

💼 Hiring manager’s perspective:
Early adopters of MCP are shipping more stable agent pipelines, with better fallback, smarter retries, and more autonomy.

When someone understands MCP, they’ve likely wrestled with persistence, state leakage, and agent handoffs. That tells me they’ve worked on problems most devs haven’t touched yet.

📚 Get hands-on with MCP:

• Mastering MCP: Building Advanced Agentic Applications

6. LlamaIndex

🧠 Use it for: Data-grounded LLM apps, internal copilots, retrieval-based agents.

👨‍💻 Why you need it:
LlamaIndex is still the gold standard for building Retrieval-Augmented Generation (RAG) pipelines. It connects your data to any model and gives you full control over how queries, documents, and embeddings are managed. As hallucination risk becomes a real blocker for production AI, LlamaIndex is the fastest way to ground your models in reality.

It’s also one of the most actively evolving tools in the RAG ecosystem, and it plays nicely with tools like DeepSeek, Claude, and OpenAI.

💼 Hiring manager’s perspective:

If you walk into a System Design Interview and explain how you’d use LlamaIndex + RAG + memory layers, you instantly stand out from the crowd still writing vanilla prompts.

📚 Get hands-on with LlamaIndex:

• Course: Mastering LlamaIndex: From Fundamentals to Building AI Apps

• Project: Build a RAG Chatbot Using DeepSeek and LlamaIndex

7. Cursor AI

🧠 Use cases: Onboarding, code navigation, refactoring, debugging across large repos.

👨‍💻 Why you need it:
Cursor is an IDE with real intelligence. Unlike Copilot, it understands your repo. It tracks context across files, supports large-scale refactors, and makes onboarding to unfamiliar codebases radically faster.

For devs working in big, messy systems (and let’s be honest, that’s most of us), it’s become an essential second brain.

💼 Hiring manager’s perspective:
Engineers who use tools like Cursor tend to ramp faster, refactor smarter, and debug with more context. I’ve seen that translate directly into performance — especially on teams managing large legacy systems or complex modular stacks.

📚 Get hands-on with Cursor AI:

• Cursor AI for Enterprise: Modernizing Professional Development

8. DSPy

🧠 Use cases: Prompt pipelines, multi-step reasoning, experiment-driven AI systems.

👨‍💻 Why you need it:
DSPy marks the shift from handcrafted prompts to engineered AI behavior. Built by Stanford, it lets you define reasoning chains declaratively, run experiments, and optimize prompts the same way we optimize compiler output.

If you’re tired of brittle prompt chains and guesswork, DSPy brings structure, clarity, and testability to your LLM workflows.

💼 Hiring manager’s perspective:
When I talk to teams building serious AI products, DSPy keeps coming up — especially among engineers who care about correctness, iteration speed, and evaluation. If you want to stand out in AI infra interviews, knowing DSPy signals that you understand systems, not just syntax.

📚 Get started with this DSPy Project:

• Build an AI Financial Analyst Assistant with DSPy and Flask

9. n8n

🧠 Use cases: Internal tools, API automation, AI-powered workflows.

👨‍💻 Why you need it:
n8n gives you the power of low-code automation with the flexibility of pro-code extensions. It’s one of the fastest ways to build internal tools, wire up AI services, or connect APIs — and it now includes LLM-native nodes for smarter logic flows.

💼 Hiring manager’s perspective:
The fastest teams I see are using tools like n8n to prototype, automate, and iterate before committing to full builds. If you care about shipping speed, n8n belongs in your toolkit, and it works great for lean and cross-functional teams.

📚 Get hands-on with n8n:

• Automate Workflows with n8n

10. OpenRouter

🧠 Use cases: Model routing, fallback strategies, cost optimization, A/B testing.

👨‍💻 Why you need it:
OpenRouter gives you one unified API to access and switch between all major LLM providers — OpenAI, Claude, Mistral, DeepSeek, and more. It lets you optimize for latency, cost, performance, or even compliance, without rewriting your core logic.

💼 Hiring manager’s perspective:
If someone’s using OpenRouter, they’ve thought deeply about tradeoffs: price vs. performance, quality vs. fallback. That kind of architectural awareness is exactly what I want in someone designing AI workflows at scale.

📚 Learn by doing with this OpenRouter Project:

• Explore DeepSeek Models via OpenRouter

The question isn’t if these tools matter…

…It’s whether you’ll learn them before they become the new baseline.

The devs and orgs that embrace this mindset and invest in tool fluency will build faster, ship smarter, and spend less.

AI is moving fast, and the gap between playing with models and building real systems keeps widening. At Educative, we’re anticipating that gap, and building resources to help you keep up, ship faster, and stay sharp.

👉 Now is the perfect time to get ahead. You can get 50% off (or more) on any Educative subscription to get hands-on with the skills that matter most to you.

Lock in Your Discount

Which tools are you using? What’s missing from this list? Hit reply and let me know.

Next week we’ll be looking back on the year with 2025’s takeaways — and what they tell us about the years ahead.

Happy learning!

- Fahim

With 2025 coming to a close, it’s time for some bold bets on what comes next.

This will be the first installment of a new series on predictions for 2026: and the topic is AI Coding.

If a peer asked how AI has changed your work as an engineer, could you describe real production use cases without making it sound like you’re training your replacement?

That’s the tension sitting at the center of AI coding today.

AI is writing code, reviewing code, refactoring code, and stitching together workflows that used to take hours. It’s in pull requests, CI pipelines, internal tools, and even customer-facing systems. It’s no longer a novelty, but a legitimate part of the day-to-day work.

At the same time, the story around “AI replacing developers” is louder than ever.

This feels like the right moment to step back and map out where I believe the work is heading. These predictions are my attempt to put a stake in the ground, and I encourage you to check in with me a year from now and tell me how well (or poorly) these aged.

Prediction 1: Agentic System Design becomes the primary job

AI-powered applications are moving toward workflows where an agent handles the sequence of decisions, not just the final answer. Teams are building systems that can follow a goal, call the right tools, track what has already happened, and recover when something goes off-script. The shift is subtle but significant: the model produces language, but the agent produces behavior.

Most teams reach this point the same way. A single prompt works for a prototype, but breaks as soon as the task requires multiple steps or access to real systems. So engineers begin adding structure: a simple state machine, then a bit of memory, then a tool call, then a fallback. Before long, they’re designing a workflow rather than a message.

Frameworks like LangGraph and CrewAI reflect this direction. They treat an agent as a component with inputs, outputs, boundaries, and responsibilities — a small system living inside a larger one. The reliability of the overall product ends up depending more on how the workflow is designed than on the model itself.

Several trends will push this approach forward in 2026:

• Longer context windows raise new questions about what information an agent should keep or forget. Continuous evaluation becomes part of day-to-day maintenance as teams monitor how behavior drifts over time.

• Compute constraints encourage splitting large tasks into smaller, model-appropriate pieces. Multimodal input — voice, vision, and mixed formats — forces agents to make decisions instead of producing a single output.

• And with MCP gaining traction, it becomes easier to plug agents into tools and internal systems in a consistent, controlled way.

The result? Engineers will spend more time shaping how an agent thinks than crafting the perfect prompt. Planning steps, defining boundaries, choosing tools, and controlling what the agent pays attention to all become part of the normal workflow.

Note: For anyone who wants guided practice in these patterns, our Agentic System Design course walks through building and debugging real multi-step agents with tools, state, and guardrails.

Prediction #2: In 2026, debugging is second only to architecture

As teams rely more on AI, the path from a user request to a shipped feature runs through more layers than it used to.

There’s the model’s output. There’s the logic that decides which tools or APIs to call. And there’s the application itself, with all the state, latency, and constraints that come with real production systems. Each layer is powerful, and each layer can quietly introduce errors that don’t show up until later.

A failure today might begin as an innocent piece of retrieved context that’s slightly outdated, or a tool response the agent interpreted a bit too literally. The model might generate a refactor that passes tests, but changes the shape of an error path in a way no one expected. The application logic might still be correct, yet the agent guiding it has drifted from what the engineers assumed it would do.

When something breaks, the investigation looks different from traditional debugging.

1. Engineers trace behavior across reasoning steps, tool calls, cached context, and application state.

2. They read the agent’s decisions as carefully as they read their own code.

3. The root cause often lives at the intersection of layers rather than within a single one.

This kind of debugging becomes a standout skill. It blends System Design instincts with an understanding of how AI systems interpret the world. And the engineers who grow in this direction become the ones who keep complex systems steady.

TL;DR: Never accept an AI explanation without verifying its chain of thought. Engineers who learn this early are the ones who spot failures in larger, more complex AI-driven stacks.

Prediction #3: Web development will shift away from human-only interfaces toward machine legibility

More software is being consumed by AI systems before a human ever sees it. Agents summarize pages, follow user instructions, pull information from dashboards, and trigger actions inside applications. That means websites and APIs now serve two audiences: people, and the AI systems acting on their behalf.

This is a quiet shift, but a meaningful one. Interfaces built only for humans introduce ambiguity. A person can interpret unclear labels, navigate inconsistent patterns, or recover from a vague error. An AI system cannot. It needs structure, hints, and constraints. It needs a predictable shape to the information it’s allowed to use.

That’s why early experiments are appearing across the web:

• Some teams are adding llms.txt files to guide how models should interpret their content.

• Others are adding AI-focused metadata to clarify what’s important on a page.

• Many are exploring shadow APIs — simplified endpoints that expose information in a form retrieval systems can trust.

• And conversations around AI control headers show how teams are thinking about permissions long before anything touches production.

This isn’t about replacing UX, but adding a layer of design that makes systems safer and easier to understand when an AI is in the loop. Developers who work on customer-facing applications will spend more time making their systems readable to both humans and machines. The teams that invest here early will see fewer surprises when agents begin navigating their products at scale.

For engineers, this opens an entirely new design surface — one that rewards clean structure, clear semantics, and predictable patterns.

How to build the signals expected of senior engineers in 2026

Developers who grow in these areas stand out when hiring managers look for senior and staff-level qualities.

If you want a concrete way to lean into this shift, one option is to turn it into a focused learning sequence like this:

1. Start with Grokking the Modern System Design Interview to solidify your mental model for the architecture underpinning scalable systems.

2. Progress to Agentic System Design and Grokking Generative AI System Design to understand how models, retrieval, and agents fit into those systems.

3. Layer in a Cloud Lab like Building a Multi-Server MCP Using AWS Bedrock Agent so you can implement and debug these ideas in real applications.

All of that (and more) lives inside an Educative subscription — and you won’t find a better deal than our Year End sale.

Lock In Your Discount

Got any questions or topics you’d like me to cover? Hit reply and let me know.

Until then, happy learning!
— Fahim

We’re back to our AI essentials series this week with a focus on one of the most in-demand skills you can have: AI System Design.

AI systems are no longer just passively analyzing data — they’re generating content, making decisions, triggering actions, and occasionally causing messes in production.

Shipping AI isn’t like shipping traditional software. They hallucinate and go off-script. And yet companies are racing to integrate AI into products.

If a recruiter asked if you’re familiar with AI System Design, would you be ready?

AI System Design is how you turn unpredictable models into reliable products. Whether you’re building chatbots, copilots, or recommendation engines, knowing how to design for failure before it happens is what separates demos from production-ready systems.

Today, we’re demystifying AI System Design so you can start building with confidence — and stay competitive in the AI era.

You’ll learn:

• 4 key differences between traditional vs. AI System Design

• 3 most in-demand types of AI System Design (and which ones you need)

• How to get started with AI System Design today

Let’s get into it.

Key differences: Traditional vs. AI System Design

AI System Design is markedly different from designing traditional distributed systems. You’re no longer optimizing for correctness… you’re optimizing for acceptable unpredictability.

Before diving into the three most in-demand types of AI System Design, let’s look at 4 key differences that apply to every flavor of AI System Design. Once you see how these AI systems break, the design patterns will make a lot more sense.

1. Deterministic vs. Probabilistic Behavior

• Traditional systems: Same input = same output.

• LLM-based systems: Same input = it depends.

Outputs vary with temperature, context, prompt phrasing, or just model quirks. So you architect for consistency, not perfection.

That means:

• Using structured prompts and templates

• Tuning sampling parameters

• Validating outputs against expected formats (e.g. JSON schemas)

👀 Example: A chatbot that always returns valid JSON — even when the model “feels like explaining.”

2. One-and-done deploys vs. continuous feedback

• With traditional code, you ship once and monitor uptime.

• With AI, you ship... and then babysit it forever.

AI System Design requires feedback loops:

• Eval jobs that score LLM responses over time

• Users flagging bad outputs

• Retraining or prompt-tuning based on failure cases

It’s not just “is it up?”
It’s “is it still helpful, accurate, and aligned with user expectations?”

👀 Example: Your chatbot nailed answers in March but starts failing in May because the knowledge base got stale.

3. Static data vs. embedded knowledge

• In traditional apps, data lives in your database.

• In AI apps, data gets embedded into behavior (literally).

Embeddings, vector stores, and fine-tuning all turn your data into part of the model’s “mind.” That means your data pipelines now affect how the system thinks, not just what it retrieves.

That’s why you’ll need to:

• Re-embed content as it changes

• Keep track of embedding versions

• Treat your vector DB like prod infra, not an experiment

4. Input validation vs. output guardrails

• Old-school systems validate inputs: SQL injection, field formats, etc.

• With AI systems, you also have to guard the outputs.

Your model might go off-brand, say something unsafe, make up answers, or generate invalid responses (e.g. malformed JSON).

You’ll need output guardrails like:

• Output format enforcement (e.g. JSON schema validation)

• Content filters and post-processing

• Escalation logic based on confidence or risk

👀 Example: If confidence < 80% or the topic is out-of-scope, the assistant defers to a human.

3 Types of AI System Design

Let’s dig into what makes each subset of AI System Design essential (and unique).

1. Machine Learning System Design

🤖 For predictions, recommendations, and ranking systems

This is the most mature form of AI System Design. It predates LLM hype and still powers everything from product recommendations to fraud detection to churn scoring.

ML systems fail quietly (and usually after you’ve already shipped).

Here’s what typically goes wrong:

• Data drift: The world changes, but your model hasn’t

• Training-serving skew: Pipelines get out of sync and silently ruin performance

• Label leakage: Your model cheats during training and gets exposed in prod

• Silent degradation: Accuracy drops but no one’s watching

You won’t catch these in logs or alerts — they hide in dashboards until users start complaining or revenue dips.

That’s why ML System Design is all about rigor and observability. You’re going beyond building a model to build a feedback loop.

Core strategies to keep models reliable:

• Version everything — data, features, training runs, serving code

• Automate evaluation across real-time data slices to catch early signs of drift

• Enforce pipeline contracts between training and production

• Add monitoring for performance, input distributions, and prediction quality

👷 Who needs it: If you’re designing systems that need to stay correct over time — especially in dynamic environments — this is the foundation you build on.

2. Generative AI System Design

🤖 For systems that talk, write, summarize, and assist

What makes Gen AI systems tricky is that they don’t fail like traditional software.

These are the most common ways they go sideways:

• Hallucinations: Confidently false, made-up answers

• Prompt injection: Users hijack the system’s instructions

• Output issues: Broken JSON, excessive verbosity, weird formatting

• Tone drift: Sudden personality shifts that feel off

That means Generative AI System Design is about shaping unpredictability into something usable. You’re not aiming for perfection. Rather, you’re building enough structure to keep the weirdness inside the lines.

Key strategies to keep things on track:

• Structured prompt templates that reduce variability and support eval

• Retrieval (RAG) pipelines to ground output in actual content

• Schema checks and content filters before output reaches the user

• Fallbacks and escalation logic when confidence or context is low

As with traditional System Design, you can’t stop every failure — but you can contain them, catch them, and recover gracefully.

👷 Who needs it: You’ll lean on this constantly if you’re building anything that wraps an LLM in a real UI (e.g., support bots, internal tools, search assistants).

3. Agentic System Design

🤖 For systems that act, plan, and use tools on their own

Agentic systems are the closest thing to autonomous AI we have today. They plan, reason, call tools, and string together multi-step workflows.

This also means that when they break, they don’t just say something weird — they do something wrong.

The most common failure modes?

• Tool misuse: Calling APIs in the wrong order, with the wrong inputs

• Memory failures: Forgetting important state mid-task

• Infinite loops: Broken plans that never resolve

• Overconfidence: Acting on bad assumptions without escalating

This is the kind of execution risk that makes devs nervous to hit “Deploy.”

To tame this chaos, Agentic System Design focuses on structured autonomy: giving the agent space to act, along with rules to stay safe.

Key design strategies include:

• Explicit planning frameworks (e.g. LangGraph, state machines) to structure behavior

• Guardrails around tool use, including input/output validation and rate limiting

• Scoped memory management to track relevant state without overloading context

• Fallback and escalation logic to gracefully hand off when the agent’s out of its depth

👷 Who needs it: If you’re building AI copilots, autonomous task runners, or tool-using bots — this is the design discipline that separates “cool demo” from “real system.”

Which One Should You Learn?

Start with:

• Machine Learning System Design if you’re building recs, rankings, or anything “classic ML”

• Generative AI System Design if you’re creating chatbots, AI features, internal assistants

• Agentic System Design if you’re designing autonomous agents or tool-using copilots

If you’re trying to work up to Staff+ engineer levels, you do need to know a bit of each of these skills. But don’t try and tackle it all at once — take it one at a time.

Here’s 4 steps to get started with either subset:

1. Pick the AI System Design subset that best fits your use case

2. Learn the core patterns and failure modes

3. Build a small, scrappy prototype (and watch how it breaks)

4. Add guardrails, observability, and feedback loops

AI is messy by nature. Good System Design won’t make it perfect — but it will make it production-worthy.

If you want to go deeper, some of our most popular courses cover these subsets:

• Machine Learning System Design digs deeper into data pipelines, training, and more

• Grokking Generative AI System Design will walk you through designing various GenAI systems, like text-to-text, text-to-image, text-to-speech, and text-to-video

• Agentic System Design will get you hands-on with essential design patterns for agentic systems

All of these courses are included at 50% off (or more) in our Year End Sale 👇

Lock in Your Discount

Next week we’ll let you in on what to expect in 2026 at the intersection of AI and code.

Got any questions or topics you’d like me to cover? Hit reply and let me know.

Until then, happy learning!

- Fahim

We’re taking a break from AI this week for a postmortem on the Cloudflare outage — and why its timing near Black Friday served as a critical wakeup call for every e-commerce company.

On November 18, the internet stubbed its toe — hard.

Like classic distributed system chaos theory in action, a seemingly minor config change turned into a full-blown outage in minutes.

Services that rely on Cloudflare began failing to load thanks to a misconfigured internal update. Since Cloudflare handles proxying, caching, and traffic protection for a non-trivial slice of the web, its edge network spread the config change faster than your team merging directly into main on a Friday.

Cloudflare sits in front of countless origin servers, so when it breaks, healthy backends become unreachable. The result? Widespread HTTP 5xx errors.

At just days before Black Friday, this outage could have easily become a financial sinkhole. But it wasn’t. And that’s thanks to good System Design.

But what it was instead was a masterclass in tight coupling and shared infrastructure. Let’s break down what went wrong, how Cloudflare recovered, and the System Design lessons you can take away for peak-traffic readiness.

Why This Outage’s Timing Mattered

Black Friday and Cyber Monday are the highest-traffic days of the year, and minutes of downtime can cost millions.

During peak load, the effects of an outage are amplified because of factors like:

• Violation of change freezes: Most retailers lock down infra pre-BFCM. A global rollout days before peak breaks the “do not touch prod” rule and exposes fragility in upstream dependencies.

• Retry storms: During peak load, a single failure fans out—checkout, auth, and payment APIs get hammered harder than usual.

• Slow warm-up: Restoring service is slower under peak traffic. Cold caches, retry queues, and warm-up traffic all collide.

...so we’re lucky this outage didn’t happen a few days later.

But what’s more, we were lucky Cloudflare’s recovery playbook worked before the stakes were higher. This is what “resilience before peak” looks like in action.

How the Outage Unfolded

It all began with a routine configuration change. Within minutes, a faulty update propagated across Cloudflare’s edge network. The centralized control plane pushed the update to all edge locations simultaneously — and as the new config hit the data path, proxy processes began failing, leading to widespread 5xx errors.

Note: The control plane manages configuration; the data plane handles user traffic. A resilient system prevents control plane issues from corrupting the data plane.

The main disruption lasted just over an hour. Some services remained unstable for several hours as regional data centers recovered. Nodes would restart, restore partial functionality, then fail again. Because it mimicked a DDoS attack, the team first investigated external vectors. But they soon found the actual culprit: a bot management configuration update.

From there, Cloudflare’s engineers halted the deployment pipeline and began rollback.

The blast radius was enormous. Major platforms—X, Shopify, ChatGPT—were affected, alongside government and transportation services. During Black Friday or Cyber Monday, such an event could overwhelm downstream systems beyond recovery.

The diagram below illustrates how the faulty configuration propagated quickly, while recovery required a region-by-region stabilization process.

Here’s the TL;DR:

• Config bloat: A configuration update from Cloudflare’s bot management subsystem created a file with duplicated rows, which bloated it past expected size limits

• Ineffective validation: Schema and size checks didn’t catch the file, and the configuration was pushed to the global edge simultaneously rather than being tested on a subset of nodes.

• Inline fragility: Because bot detection runs inline on the proxy request path, this caused proxy processes to crash, taking large portions of Cloudflare’s edge network offline

The failure was swift, but the recovery process was complex and required stabilization on a region-by-region basis.

💡 Insight: Non-essential features such as bot management should never sit on the critical request path. When these components fail, the system should degrade gracefully rather than crashing the proxy.

How Cloudflare Recovered Through Detection, Rollback, and Stabilization

Once engineers identified the issue with the bot management subsystem, they halted the pipeline to prevent further propagation and initiated a rollback to the last known good configuration.

Nodes had to be restarted in waves. As servers came back online, they faced a thundering herd of retry requests from clients. This traffic surge posed a risk of a secondary outage.

💡 Pro tip: Implement exponential backoff with jitter in client-side retry logic to avoid thundering herds during recovery.

Managing this wave of retries is a standard challenge in system recovery. During a Black Friday event, the sheer volume of traffic would make this significantly harder to manage.

The graphic below illustrates the phased recovery and the traffic surge that recovering nodes had to handle.

Learn from the ultimate Black Friday success story

We’ve opened up a recent edition from our subscriber-only Educative Newsletter covering one of the ultimate success stories of Black Friday.

Shopify’s Black Friday Case Study

3 System Design Lessons for Resilience During Peak Traffic Events

Building resilient systems requires anticipating failure at every layer of the stack. Here’s how.

1. Strengthen configuration pipelines

Configuration updates require the same rigor as code deployments. Automated checks must prevent bad configurations from reaching production.

For example:

• Validation: Configuration data should be validated with the same rigor as external inputs. Schema checks, integrity checks, and size limits help detect anomalies such as unexpected growth in file size or missing fields before the data reaches the edge.

• Deployment: Configuration rollouts should use staged or canary deployment strategies. A small percentage of traffic or a limited set of regions should receive updates first. This limits the blast radius in the event of a configuration error and provides early warning signals.

• Atomicity: Ensure configuration updates are atomic. A system should only replace a configuration when the new version has passed validation and health checks.

The diagram below contrasts a fragile deployment process with a secure pipeline, highlighting where automated gates and canary phases intercept faulty configurations.

2. Isolate failure domains

The outage was global because a non-essential subsystem sat on the critical request path. Decoupling is essential for resilience.

• Fail open vs. fail closed: Auxiliary services such as bot detection, analytics, or logging should not be able to disrupt the primary request path. When these subsystems fail, they should default to allowing traffic through rather than blocking requests or crashing proxy processes.

• Decouple subsystems: Use asynchronous processing, bulkheads, and dedicated resource pools to prevent failures in one subsystem from cascading into others.

💡 Architectural insight: The bulkhead pattern partitions a system into isolated pools of resources, allowing for independent operation and management. A failure in one pool does not affect the others, preventing total system collapse.

The following diagram contrasts a fragile monolithic design with a more resilient bulkhead architecture.

3. Mitigate centralization risk

Relying on a single global provider introduces systemic risk. Redundancy helps limit the impact of provider-level failures.

• Multi-CDN strategy: An active-passive or active-active multi-CDN setup can route traffic to an alternative provider when the primary CDN experiences an outage.

• Automated failover: Failover systems must be automated and validated through routine testing using synthetic traffic or chaos engineering simulations to ensure reliability and effectiveness. Manual intervention is too slow during fast-moving outages.

A proactive approach requires a checklist of best practices.

Designing for the next outage

The November 18 Cloudflare incident is a brutal reminder: resilience isn’t about never failing — it’s about not taking everything else down with you.

The fix isn’t flashy: validate your configs, isolate your blast radius, and stop pretending your single-provider setup is “fine.” During peak traffic, assumptions break. The question is whether your system does too.

Had it happened a few days later, this incident could have been the outage story of the season. But even then, we still could’ve been saved thanks to staged rollbacks, edge isolation, and a capable incident response pipeline.

Want to dig deeper into the skills you need to build resilience?

Check out our top resources:

• System Design Deep Dive: Real-World Distributed Systems: Understand resilience by examining real-world hyperscalers like Amazon and Google.

• Scalability & System Design for Developers: A comprehensive Skill Path covering everything you need to know to design scalable systems for enterprise-level software.

You can still access these courses, plus Projects, AI Mock Interviews and more, with extended Cyber Monday offers (still available for a few more hours).

Lock in your discount

Next week we’ll return to AI with a deep dive on AI System Design — the different subsets, and which ones you need to know for your career.

Got any questions or topics you’d like me to cover? Leave a comment and let me know.

Until then, happy learning!

- Fahim

Last week, we grounded our models with RAG to stop the hallucinations.

Now, it’s time to let the machine do the work. We’re continuing our AI series by moving from smart answers to autonomous action.

The next frontier in AI development is less about making LLMs ‘smarter’, and more about increasing their ability to act autonomously.

This is where Agentic AI comes in.

But to fully understand AI Agents, we need to compare them to their predecessors:

• Traditional LLMs: These are specialized tools trained for specific tasks (like summarizing text). They’re good at many things, but limited in adapting to unforeseen scenarios.

• Compound Systems (like RAG): As we learned last week, compound systems connect multiple components (LLMs, databases, search tools) to solve bigger problems. However, they typically follow a fixed, predefined sequence of operations.

• AI Agents: Now for the really fun part. These are advanced compound systems with autonomy. Instead of following a rigid script, an Agent leverages the reasoning capabilities of a foundation model to plan, adapt, and choose its own tools to achieve a goal.

With that out of the way (cheers, Vince), let’s get to building.

Understanding the Agentic Workflow

AI agents have three key capabilities: reasoning, acting, and memory.

These capabilities enable AI agents to handle complex tasks by breaking them down into manageable steps, utilizing external tools, and leveraging stored information for personalized interactions.

The agentic workflow can be visualized in the following diagram:

In the above diagram, the process starts with a User Query, which is the initial question or problem posed by the user.

• The AI agent then proceeds to Plan (Reason), formulating a step-by-step strategy to address the query.

• The agent then moves to Action (Use Tools), utilizing various external tools such as search engines or calculators to gather necessary information and perform tasks.

• The Observe step involves the agent reviewing the results of its actions, and if needed, it may adjust its plan accordingly, looping back to the reasoning stage.

Finally, the process concludes with the Final Output, where the agent provides a comprehensive and tailored response to the user.

How Most Devs Are Building Today

With so many ways to build — OpenAI’s new Agents API, LangChain, LlamaIndex, CrewAI, AutoGen, and a growing list of lightweight libraries — it can be hard to know where to begin. Every framework advertises a different philosophy, a different abstraction layer, and a different idea of what an “agent” even is.

But when you strip away the branding and look at how developers actually get agents working in the real world, the pattern is surprisingly consistent. No matter which framework you choose, the first successful agent almost always boils down to the same simple setup: one model, one tool, one loop.

Everything else is just structure layered on top.

To keep things (relatively) simple, we’ll be working in python. Let’s begin.

1. Start With a Single-Agent Loop

The simplest agent you can build is:

• An LLM (GPT, Claude, Llama — pick your poison)

• A small set of tools (like an API)

• Instructions (‘don’t screw this up for me’)

• A loop that lets it keep thinking and acting until it’s done

Every framework (LangChain, CrewAI, OpenAI’s Agents SDK) is doing the same thing underneath:

If you can build this, you can build almost any agent system.

2. Give It One Useful Tool

Early wins usually come from one reliable tool. For example, a web search or internal API call.

Then you register that function as a tool the agent can call:

With just this, your agent can stop guessing and start doing something concrete.

3. Add Tight Instructions (Not a Vague Prompt)

A good agent behaves more predictably when its instructions are specific.

That’s all you need for a very capable first agent: one tool, clear rules, and a loop.

Once developers get their first agent running, the next layer is all about reliability: testing the workflow against real inputs, validating tool outputs, and adding lightweight guardrails so the agent doesn’t loop forever or take the wrong action.

These steps aren’t wildly complicated, but they separate a fun demo from something you can trust in production.

Connecting Systems: The Model Context Protocol (MCP)

As agent systems grow, they need a reliable way to talk to your tools, data sources, and internal APIs. Right now, every service speaks its own language — one for calendars, one for file systems, one for search, one for business apps. That quickly turns into a pile of brittle, custom integrations.

MCP (Model Context Protocol) addresses this by giving AI agents a standard way to connect to external systems.

You can think of it as a universal adapter:

• Your model runs inside an MCP host.

• It communicates through an MCP client.

• Tools and data sources sit behind MCP servers.

Once a service exposes an MCP server, any compatible AI agent can use it without bespoke glue code. That keeps your system modular, reusable, and easier to scale.

MCP defines three simple primitives:

• Tools: actions the agent can take

• Resources: read-only data the agent can fetch

• Prompts: reusable instruction templates hosted on the server

It’s still early, but MCP is already becoming the foundation for building larger, more interoperable agent systems. You’ll start seeing it show up everywhere: IDEs, assistants, multi-agent frameworks, and developer tooling.

Get Started With Agents Today

The transition from a smart prompt to an autonomous Agent is the biggest leap in modern AI engineering. If you can master Agents, you can build systems that truly automate major business processes.

Here are some resources to help you get started:

• 👉 Still need to home in on the basics? Find your footing with Generative AI Essentials

• 👉 Ready to command your first digital assistant? Dive deeper into the multi-agent paradigm with Build AI Agents and Multi-Agent Systems with CrewAI.

• 👉 Orchestrating multiple agentic systems? Explore MCP Fundamentals for Building AI Agents to learn more about the Model Context Protocol, the new standard for connecting AI assistants to systems and data.

You can access these courses (and more) at Black Friday prices right now:

View Black Friday Offers

Next week, we’ll take a break from AI to dive deep into a BFCM case study. After that, we’ll return to our AI series with an actionable overview of AI System Design.

Until then, let me know if there are any other topics you’d like me to cover in the comments.

Happy learning!

- Fahim

Hi there, I’m continuing my AI series with a focus on RAG today. If you don’t know why you need it or how to get started, you shouldn’t miss this.

Before we start, a quick reminder that your Early Bird Black Friday offers will be available on Educative for only one more week. This is a great opportunity to grab access to all of our courses on RAG, Agentic AI, and other critical AI topics. Use this link to view your offers.

Building with LLMs is exciting… until your app confidently tells a user that the Eiffel Tower is in Berlin.

Many developers hesitate to work with AI because of its unpredictability. And for good reason. Major hallucinations kill user trust fast.

But there’s a proven architecture to reduce those mistakes: Retrieval-Augmented Generation (RAG). Instead of asking the LLM to recall an answer, RAG involves giving your LLM the context needed to generate a better answer in real time.

The result is more accurate answers, fewer hallucinations, and way better UX.

Today, I’ll cover:

• How RAG actually works

• How to build your first pipeline

• Best practices and popular tooling

Let’s dig in.

How RAG works

At a high level, RAG combines retrieval and generation in a tight loop. Instead of just prompting an LLM, you’re plugging it into a pipeline that feeds it curated, context-relevant info in real time.

Here’s what’s happening in a RAG-enabled system when a user submits a question:

1. Query comes in

   The user asks something like: “What’s our refund policy for enterprise clients?”

   A clear, specific question = better retrieval = better answer.

2. Retrieval kicks in

   The system embeds the query, runs a similarity search on your vector DB, and pulls back relevant chunks of context.

3. Augmented prompt is built

   The original question + retrieved context + instructions (e.g. “Only answer using this info. Say ‘I don’t know’ if it’s not there.”)

4. LLM generates the response

   The model uses the prompt to generate a fluent, fact-grounded reply — now informed by your actual data.

5. User gets an answer they can trust

   Hallucination risk drops, confidence goes up. Users get useful answers, not confident guesses.

This is the conceptual flow of RAG, but to make it real, you need a few key pieces working together. Let’s walk through how to wire them up into an actual pipeline you can ship.

Your first basic RAG pipeline

A lightweight, functional RAG stack can be built with a handful of tools and some Python glue code.

Here’s a simple version you can stand up today in 5 steps.

1. Ingest your data

You need to make your docs searchable.

That means:

• Chunking them (e.g. 500-token chunks)

• Converting them into vector embeddings using a model like text-embedding-3-small

🛠️ Tools: LangChain, LlamaIndex, or a simple script using OpenAI + FAISS.

2. Store your vectors

These embeddings go into a vector database, which lets you search by semantic similarity instead of keywords.

🛠️ Tools:

• Open-source: FAISS, Chroma

• Managed: Pinecone, Weaviate, Qdrant

3. Handle the user query

You get a user question like: “What’s the SLA for our enterprise customers?”

You embed that question the same way you did your docs, and run a similarity search against your vector DB. This pulls back relevant chunks (say, a paragraph from your internal policy doc).

4. Construct the prompt

Now you build a prompt that includes:

• The user’s question

• The relevant retrieved context

• Instructions for how to answer (this part matters more than you think)

Example:

“Use the following context to answer the question. If the answer isn’t in the context, just say ‘I don’t know.’ Don’t make anything up.”

5. Call the LLM

You send the prompt to the model (OpenAI, Anthropic, Mistral, etc.) and get a response that’s:

• Grounded in your actual data

• Less prone to hallucination

• More trustworthy for users

And congratulations, you’ve built your first RAG pipeline.

RAG best practices

The difference between a scrappy prototype and a system users can actually trust often comes down to how well you manage the details.

Here are a few principles that’ll save you hours of debugging (and some deeply unhinged LLM outputs):

• Chunking size matters. Too big? You lose precision. Too small? You lose context. 300–500 tokens per chunk is a good default.

• “I don’t know” prompting is critical. Tell the model not to improvise when the answer isn’t in the context. Otherwise? It will.

• Eval early. Don’t wait until prod to find out your retrieval sucks. Manually check: “Does the retrieved context actually answer the query?”

• Don’t overcomplicate it. You can do basic RAG with a local FAISS index, 50 lines of Python, and OpenAI’s embedding API. Tools like LangChain are great — just know what’s under the hood.

Stick to these, and you’ll avoid 90% of the most common RAG faceplants.

When NOT to use RAG

RAG is powerful, but it’s not a silver bullet.

It won’t help if:

• You’re doing complex reasoning across multiple documents (that’s better for agents or multi-step planners)

• You need structured data from APIs or databases (use tools instead)

• Your data is always changing in real-time (consider hybrid approaches)

And if RAG doesn’t fit your use case, there are alternative approaches you could take:

• Tool use and API calling for tasks that rely on dynamic or structured data,

• Fine-tuning might be more efficient if your domain is stable but highly specific.

• A hybrid approach — using RAG where flexibility is needed, and fine-tuning or tools where precision matters.

Ground your models now, thank yourself later

If your app needs to ground its answers in real knowledge, you definitely need to learn RAG.

But don’t get blocked by tooling or perfectionism. Just build a scrappy prototype, test it with real users, and improve it from there.

The sooner you start, the sooner you stop shipping AI that just makes stuff up.

Here are some resources to help you get started:

• 👉 New to RAG? Fundamentals of Retrieval-Augmented Generation with LangChain shows you how to apply LangChain to implement RAG pipeline and build a frontend app for your pipeline with Streamlit.

• 👉 Ready for advanced RAG? Advanced RAG Techniques: Choosing the Right Approach covers different RAG approaches, post-retrieval optimization methods, and designing RAG-based chatbots.

This is your last week to access these courses and more at 50% off (or more) for our Early Bird Black Friday Sale.

View Early Bird Offers

Next week, we’ll demystify agents and how you can grok them to future-proof your career.

Any questions about the future of development or AI? Leave a comment and let me know.

Happy learning!

- Fahim

A lot of developers have tried ChatGPT or Copilot. But when it comes to actually building something real, they’re not confident. If that’s you, you’re not alone.

…but you are falling behind. LLMs are here to stay, and if you don’t learn how to build with them fast, the next few years will leave you in the dust.

To help you catch up, I’ll spend most of my efforts through the new year to address the AI topics that readers have requested the most: from RAG through 2026 must-have tools. But today I’m kicking it off with an overview of what makes building with AI fundamentally different.

LLMs have transformed our development model.

In the 2010s, APIs became our go-to for connecting with third-party services. Building something like Uber didn’t mean reinventing the wheel for every feature—you stitched together services like Google Maps for directions and Stripe for payments. The API model was reliable and boring (in a good way).

But when you use LLMs via API, the output isn’t always precise.

Take the classic GPT 4o mistake: misspelling “strawberry” with three R’s:

You’d never expect this kind of mistake from a legacy API or SaaS platform (imagine if Google Maps guessed a location with 70% confidence). But with LLMs, this uncertainty is baked in.

Still, that’s not stopping any companies from adopting AI. 92% of companies plan to invest more in AI over the next three years. And if you don’t learn the skills to work with AI reliably, you’ll be competing in an AI-hungry market against developers who have.¹

Grokking AI is all about grasping how to work with its unpredictability.

To help you get there, I’ll cover this today:

• 3 LLM components that drive unpredictability under the hood

• The skills you need to build resiliently with AI

Let’s get started.

3 components that make LLMs powerful (and weird)

The unpredictability you see in LLMs is a direct consequence of how they work under the hood. Let’s talk about how these 3 components contribute to that.

1. Transformers

At the heart of every LLM is the transformer — a neural network architecture built around a mechanism called attention. Attention lets the model weigh which parts of the input matter most when making a prediction.

Unlike older models that process text word by word, transformers read the entire input (called the context window) at once. Then, they predict what comes next based on the patterns they’ve seen before.

This means that transformers don’t understand meaning. Rather, they extend patterns. You’re getting what the model thinks is a likely continuation of the input, but not necessarily a logical result.

This is what causes weird behavior:

• The same input can produce different outputs, depending on the surrounding context.

• Minor prompt changes can cause big differences in completions.

• The model isn’t “following instructions” — it’s guessing what looks right next.

2. Tokenization & Sampling

Before an LLM can generate output, it first breaks down the input text into tokens — small chunks of text that may be whole words, word fragments, or even just characters — in a process called tokenization.

For example, the word unpredictability might be split into several tokens like un, predict, and ability. Rather than seeing sentences, the model sees token sequences.

To generate output, the model generates a probability distribution over its vocabulary for the next token. This is called sampling: picking the next token at random, weighted by its probability.

That randomness is what makes LLMs feel creative — but it also makes them unpredictable:

• Change the temperature, and you change how random the output is.

• Use different sampling strategies (like top-k or nucleus sampling), and the model may complete the same prompt in very different ways.

• Even punctuation or emoji can split into multiple tokens and skew the results.

While you can force consistency by setting the temperature to zero, doing that often kills the flexibility and fluency that make LLMs useful in the first place.

3. Training Signals

Most LLMs are trained in two big phases:

1. Pretraining: The model learns language by predicting the next token on massive text datasets.

2. Fine-tuning: Humans step in and say “this output was helpful,” using techniques like Reinforcement Learning from Human Feedback (RLHF) to make the model more aligned with human preferences.

This fine-tuning and RLHF process makes a model more polite, friendly, and cooperative. But it also introduces some major issues:

• It’s rewarded for sounding helpful, not for being factually correct.

• It might confidently hallucinate an answer instead of saying “I don’t know.”

• Sometimes it’ll refuse to answer a valid query to avoid “sounding risky.”

This means you’re not just debugging your code anymore — you’re also debugging the ghost of every human preference that got baked into the training loop. Fun.

AI skills to mitigate uncertainty

You can’t change how LLMs work, but you can still build reliably with them if you learn skills and strategies that help mitigate their pitfalls.

The most in-demand AI skills are a direct response to the quirks we unpacked:

• Prompt engineering: Steers transformer behavior by reducing ambiguity and guiding attention.

• Retrieval-Augmented Generation (RAG): Injects real-world context to override hallucinations from pretraining or RLHF.

• Output validation and filtering: Catches bad outputs caused by token sampling randomness or training misalignment.

• Tool use and orchestration: Delegates tasks the model is bad at (math, APIs, logic) to more reliable systems.

• System Design for failure: Assumes the model will mess up and builds in retries, backups, and fallbacks.

• Evaluation strategies: Measures performance when “correctness” is fuzzy or probabilistic.

If you’ve heard of these skills but didn’t understand why they’re useful, you do now.

Reducing uncertainty in LLMs (and your career)

The unpredictability in LLM outputs is a natural side effect of their architecture. If you can embrace uncertainty instead of fighting it, you’ll be ahead of the developers still treating LLMs like APIs.

Start now, with the skills that matter most:

• Learn prompt engineering to steer model behavior.

• Add basic validation to catch bad outputs before they reach users.

• Use RAG to ground your model in facts when accuracy matters.

• Bring in tools and fallback logic as your system grows.

• And when you’re deploying to real users, track what’s working with evaluation.

You don’t need to master all of this overnight. But if you want to build with AI, now’s the time to move. The LLM wave isn’t slowing down — and waiting until the tooling is “mature” or the models are “perfect” is a gamble you will lose.

To keep learning:

• You can get hands-on experience with everything from prompting to RAG to fine-tuning to building agents in our comprehensive Skill Path: Become an LLM Engineer.

• If you’re ready for more advanced AI skills, explore our entire catalog of Generative AI resources.

For a limited time, you can access these courses and more at a 50% discount (or more) with our Early Bird Black Friday Sale.

View Early Bird Offers

Over the next few weeks, we’ll discuss some of the applied techniques you need to engineer successful AI systems, including:

• How to set up your first RAG pipeline

• How to create reliable agentic systems.

• Designing for failure with AI System Design

Until then, do you have any burning questions about AI and software development? Leave a comment and let me know.

Happy learning!

— Fahim

Hi there,

This week I wanted to do something a little different.

In light of the big AWS crash last week, we decided to make this week’s “post mortem” edition of our Educative Newsletter free for everyone. The Educative Newsletter is a perk for Educative Unlimited subscribers — who received insights on the crash the very next morning.

You can see a preview below, or use this link to read the whole thing. Enjoy!

When (and why) Amazon $hat the bed

At the start of last week, the internet was literally falling apart.

Netflix lost its chill.

DoorDashers went hungry.

Redditors around the world thought they were shadow banned from their favorite online communities.

And spookiest of all? You couldn’t even send a Slack.

(Hey, maybe that last part wasn’t so bad).

You likely experienced the AWS us-east-1 outage that broke the internet in your own way. But do you have a complete understanding of what actually caused the outage, and how it all went down?

Read Now

When systems break, we learn

The internet is only as strong as the systems we design to hold it together.

If this kind of postmortem fascinates you, or if you’re thinking about how to make your own architecture more resilient, our courses dig deep into the principles that separate fragile systems from fault-tolerant ones.

Now’s the perfect time to learn something that won’t vanish when us-east-1 does again.

• Grokking Modern System Design Interview

• Distributed Systems for Practitioners

• Learn the A to Z of Amazon Web Services

And you’ll also get topical post-mortems like today’s feature in your inbox as a subscriber.

🎃 And remember — don’t forget to grab 50% off your Educative Unlimited subscription before the Halloween sale ends on October 31st.

Happy learning!

- Fahim

The Halloween countdown continues.

With that, we have an interview horror story that’ll leave even the most seasoned devs clutching their rubber duck — followed by some juicy MAANG news.

But first, don’t forget to take advantage of our most aggressive sale of the year: here’s the link to get 50% off your Educative unlimited subscription.

Last week, a few of you replied to my request for interview horror stories. Thank you all for sharing.

Reading through them, this one stood out — not because it’s rare, but because it happens a lot. And I think I have some antidotes to share with you.

“I was preparing for my first big MAANG interview: a Software Engineer (L3) role at Meta. I spent MONTHS doing technical prep for process management, distributed systems, and database scaling. I built mock APIs, designed fake backends, even ran through a few system design scenarios just in case. I really felt like I knew my stuff.

I crushed the coding round and the system design interview was probably my strongest. But then came the behavioral round.

Honestly, I thought it’d be the easy part. I had always gotten on well with teammates from previous roles, and never had an issue talking about my experience.

But every behavioral question hit harder than I expected: a time I failed, disagreed with a lead, or missed a deadline. And I froze. And when I came to, my answers were scattered, too technical, and didn’t really tell a story.

A couple days later, the rejection email hit my inbox. In typical fashion, the recruiter didn’t give an exact reason. But deep down I knew: I bombed because of the behavioral interview.”

If you’ve ever struggled through a behavioral interview, you’re not alone.

Most engineers underestimate it, but as an interviewer myself, I can tell you that the behavioral round is a bigger decision-maker than ever before. Trust me: in today’s saturated market, you’re competing with hundreds of developers who have the same technical skills. So what makes you a real hire for the team is the sense of what kind of team player you are — how you communicate, prioritize, and support your team when things go sideways.

The good news? There’s a system for this part too. I’ll give you three strong frameworks.

3 behavioral frameworks every engineer should know

1. STAR (Situation, Task, Action, Result)

This is the standard template most interviewers expect. It’s perfect when you need to walk through a project, challenge, or achievement from start to finish.

• Situation: Give context (what was happening, who was involved).

• Task: Explain your specific responsibility or goal.

• Action: Describe what you did to tackle it.

• Result: Share the measurable or meaningful outcome.

👉 Why it works: STAR keeps your answer structured and easy to follow. It shows that you understand sequencing and can separate what you did from what happened around you — a key sign of self-awareness.

2. SOAR (Situation, Obstacle, Action, Result)

Some stories aren’t about steady progress, they’re about friction. SOAR helps you explain what went wrong and how you navigated through it.

• Situation: Set the stage.

• Obstacle: What got in your way (technical, interpersonal, or organizational).

• Action: How you responded to the challenge.

• Result: What you achieved or learned in the process.

👉 Why it works: SOAR highlights resilience and adaptability. It’s ideal when your story involves conflict, resource constraints, or a project that didn’t go as planned.

3. CAR (Context, Action, Result)

Other questions don’t need a full four-part breakdown. That’s when CAR shines: it helps you keep things tight and relevant.

• Context: Briefly explain the setup.

• Action: Focus on what you did.

• Result: Close with impact or takeaway.

👉 Why it works: CAR trims the fat. It’s great for rapid-fire follow-ups or when you’re short on time and need to move quickly through multiple examples.

At first glance, these frameworks might look interchangeable. But the key is in the subtleties and knowing when to use each (especially under a time constraint). Here’s how you can practice:

• Run a few mock interviews and challenge yourself to answer every question using a different framework.

• Record your responses and note which one feels most natural for you.

• Refine, repeat, and get comfortable shifting between them in real time.

That way, when the real interview hits, you’re not guessing which story structure fits — you’re choosing it with confidence.

AI-enabled coding rounds are officially a thing

Speaking of interview rounds at Meta...

There have been some murmurs that Meta’s new AI-enabled coding round is officially live, replacing one of the traditional onsite coding rounds. Here’s what it involves (big thanks to reddit user drCounterIntuitive for the breakdown):

• 60-minute CoderPad session with an AI-assist chat window (GPT-4o mini, Claude 3.5 Haiku, or Llama 4 Maverick; you can switch models). Somewhat similar interface to github copilot’s chat window, but simplified.

• One thematic question with multiple checkpoints or stages (so it can be a multi-part question), not two separate LeetCode problems

• You get a mini multi-file codebase (for Python: multiple .py files plus requirements.txt)

• You can run and debug code in real time (no dry-running needed)

• The interview started appearing early October 2025 for SWE and ML; likely rolling out to Production Engineers soon

Survive the season

Behavioral, technical, or AI-assisted — every interview round tells a story. The best engineers know how to prepare for all of them. You can keep your skills sharp and your prep structured with our most popular resources:

• Grokking the Behavioral Interview

• Grokking Modern System Design Interview

Stay prepared and stay out of the dark out there.

Happy learning!

- Fahim