Framework changes: reproduce, diagnose, migrate

Dependency churn is the ongoing change in libraries, integrations, runtimes and hosted services that an application depends on. It can break imports, alter behavior, or retire a service while your own source code stays unchanged.

The engineering goal is controlled change: reproduce the current system, understand the specific incompatibility, and migrate with evidence. Freezing every dependency forever leaves security and support problems; upgrading everything without evaluation creates a different risk.

First distinguish the failure

An error message is evidence, not a complete diagnosis. An abstract-method error is not proof that “upgrade the core and adapter to their newest versions” is the right repair. First inspect their declared compatibility and reproduce the failure in an isolated environment.

A concrete import example

A tutorial may show:

from llama_index import VectorStoreIndex

A current core import is:

from llama_index.core import VectorStoreIndex

But replacing that line alone does not migrate the application. Check its reader, embedding, model, vector-store, and workflow packages. Independently versioned integrations do not need identical version numbers; they need a compatible resolved set.

LlamaIndex's configuration migration describes moving from ServiceContext to Settings, and also shows local component configuration. Global settings can be convenient, but mutating a global model or identity per request risks cross-request interference. Prefer explicit per-component dependencies when requests need different configurations.

LangChain's v1 migration guide documents create_agent and the relocation of legacy functionality to langchain-classic. “Old import” does not always mean “deleted feature.” A compatibility package can support a staged migration, with an owner and removal deadline.

Interview tip: Say which layer changed—import, adapter, state format, model behavior, or hosted API. “The framework broke” is too broad to guide a repair.

What a reproducible AI application records

A package lockfile pins software dependencies; it cannot keep a retired hosted API alive or make a probabilistic model deterministic.

Lock and synchronize deliberately

For a Python application using uv, commit its project metadata and lockfile. In a clean, intended project environment:

uv sync --locked
uv run --locked python -m pytest

--locked rejects a project/lockfile mismatch instead of silently resolving a new lock. --frozen uses the lockfile without that freshness check; it is not an equivalent spelling. See uv locking and syncing. The test command assumes pytest is part of the project's selected dependency groups.

For an npm application with a checked-in lockfile:

npm ci
npm test

npm ci requires agreement between the manifest and lockfile and replaces the installed dependency tree. Use it in the intended checkout/CI workspace, not as a harmless read-only diagnostic in someone else's active environment.

Other package managers have their own locked-install workflows. Python's PEP 751 specifies pylock.toml; that standard does not mean every tool natively uses the same lockfile or that arbitrary formats are interchangeable. Respect the repository's existing tooling.

Exact pins on direct dependencies alone do not capture transitive dependencies. Conversely, adding redundant exact pins everywhere is not necessary when a suitable lockfile already resolves the application. Maintain explicit compatibility constraints where they communicate a real requirement.

Hosted lifecycle changes require a different response

As reviewed in September 2026, OpenAI's official notices include:

Use the official deprecation notice and the Assistants migration guide. These dates describe specific product surfaces; they do not establish that all hosted agent or evaluation capabilities are disappearing.

Migration guides can themselves contain transitional examples. Check whether the proposed replacement has a separate lifecycle notice. Keep prompt text, evaluation cases, tool definitions, and business state exportable so a vendor product change does not strand the application's essential assets.

Interview design: upgrade the interview-coaching service

Assume a service retrieves lessons, generates feedback, and saves a study plan. A provider adapter update changes tool-call handling, while existing plans and conversations must remain readable.

Functional requirements

1. Reproduce the currently deployed version and its behavior.
2. Identify required API, schema and state changes.
3. Run old and new implementations on representative saved inputs.
4. Deploy gradually, observe results, and support a defined recovery path.
5. Preserve user access, saved plans and legitimate in-progress work.

Non-functional requirements

1. No cross-account data exposure during comparison or migration.
2. No duplicate external writes from replayed requests.
3. Explicit limits for evaluation cost and production error rates.
4. Traceable source, dependency, model and dataset versions.
5. A rollback plan compatible with any database or state changes.

flowchart LR
    A[Known production baseline] --> B[Isolated migration branch]
    B --> C[Resolve compatible dependencies]
    C --> D[Contract and behavioral checks]
    D --> E{Acceptance criteria met?}
    E -->|No| B
    E -->|Yes| F[Shadow reads or restricted canary]
    F --> G{Production gates pass?}
    G -->|Yes| H[Expand and monitor]
    G -->|No| I[Rollback or forward repair]
    H --> J[Retire compatibility path]

Migration sequence

1. Capture the baseline: package lock, runtime, explicit model configuration, schemas, representative cases and observed failures.
2. Read the migration contract: identify breaking changes and compatible core/adapter ranges. A supported direct upgrade can be better than mechanically installing every historical release.
3. Change one coherent boundary: avoid combining a framework migration, embedding-model replacement and prompt rewrite unless they are inseparable.
4. Apply supported automation: review codemod output as a code change; codemods do not prove behavioral equivalence.
5. Verify contracts: tool arguments/results, stream events, cancellation, retry limits, access filtering and stored-state compatibility.
6. Evaluate behavior: compare correctness, groundedness, task completion, latency and total cost under the same inputs and budgets.
7. Roll out gradually: shadow only operations that can be safely duplicated. Route mutations to one authoritative path.
8. Close the migration: remove temporary adapters after the recovery window and document the new operating baseline.

A canary threshold is a product decision. For illustration, a team might require zero observed cross-account access violations, no regression in a specified critical-case suite, and an agreed p95 latency budget. An observed zero is not a mathematical guarantee of zero risk; keep production detection and recovery.

Find the dangerous shortcuts

Framework, raw SDK, or application adapter?

A direct SDK is useful when a small set of calls is enough. A framework can justify itself through workflow, retrieval, integration, or observability capabilities. A narrow application-owned adapter can isolate the model boundary, but it must preserve streaming, errors, cancellation and tool semantics that the product actually uses.

These are tradeoffs, not a maturity ladder. Raw SDKs also change. Provider-native structured output and tool calling share ideas but not identical semantics. A nominally compatible endpoint does not establish feature parity. Use the framework selection guide to compare a concrete workload.

Hypothetical migration economics:

engineering effort = 60 hours × $100/hour = $6,000
measured monthly operating saving        = $750
simple payback                           = 8 months

This excludes support-risk reduction, new capabilities, opportunity cost, and ongoing maintenance. A mandatory service shutdown may justify migration without a positive cost-saving payback. State the actual reason instead of inventing a financial win.

Learn the concepts that survive an API rename

Before using a tutorial, inspect its date, dependency files, runtime, provider/model requirements and matching versioned documentation. Decide whether to reproduce its historical environment or implement its concepts with current APIs. An old tutorial is not automatically useless, and a new tutorial is not automatically correct.

Interview questions and answer checks

1. An import fails after a fresh install. What do you inspect first? The actual interpreter/environment, installed packages, traceback and documented import—not just the tutorial date.
2. Must core and integration package numbers match? No. Their declared compatibility constraints and the resolved set must agree.
3. Does a lockfile make model output reproducible? It helps reproduce client software; hosted model behavior, data and nondeterminism remain separate.
4. Why can an API-compatible migration still fail? Output quality, retries, streaming, defaults and persisted state may change.
5. When is shadow traffic unsafe? When duplicated work can cause external effects or disclose data to an unauthorized destination.
6. Why might rollback need a forward repair? New data/state may be incompatible with old code, or a retired external service may no longer be available.
7. What should remain application-owned? Business contracts, authorization, essential state, prompt/evaluation assets and acceptance criteria.
8. How do you close a migration? Verify outcomes, monitor the rollout, remove temporary paths on schedule, and record the new baseline and remaining deadlines.

Final notes

Use reproduce → classify → migrate → evaluate → release → monitor. Learn framework APIs well enough to work effectively, while grounding decisions in stable concepts. The strongest protection against churn is an understood system with explicit contracts and a tested path for change.