Skip to content

fix: close all pipeline components even when an earlier close raises - #13845

Open
TejasLamba2006 wants to merge 1 commit into
deepset-ai:mainfrom
TejasLamba2006:fix/pipeline-close-all-components
Open

TejasLamba2006 wants to merge 1 commit into
deepset-ai:mainfrom
TejasLamba2006:fix/pipeline-close-all-components

Conversation

@TejasLamba2006

Copy link
Copy Markdown
Contributor

Related Issues

Proposed Changes:

PipelineBase.close() and PipelineBase.close_async() previously iterated over self.graph.nodes without isolating per-component errors. If an earlier component raised during cleanup (for example a LinkContentFetcher whose keep-alive client belonged to a closed event loop), the loop exited immediately and skipped closing all remaining components in the pipeline.

This PR wraps each component's close() / close_async() call in try ... except Exception (leaving BaseException subclasses such as asyncio.CancelledError and KeyboardInterrupt untouched), logs a warning for each failure, continues closing the remaining components, and re-raises the first exception once all components have been attempted.

How did you test it?

  • Added unit tests in test/core/pipeline/test_pipeline_lifecycle.py covering both Pipeline.close() and Pipeline.close_async() (including async components, sync-fallback components, and multi-component failures).
  • Ran pytest test/core/pipeline/test_pipeline_lifecycle.py, ruff check, ruff format --check, and mypy.

Notes for the reviewer

Only Exception instances are caught inside the loop so task cancellation (asyncio.CancelledError) still propagates immediately.

Checklist

  • I have read the contributors guidelines and the code of conduct.
  • I have updated the related issue with new insights and changes.
  • I have added unit tests and updated the docstrings.
  • I've used one of the conventional commit types for my PR title: fix:, feat:, build:, chore:, ci:, docs:, style:, refactor:, perf:, test: and added ! in case the PR includes breaking changes.
  • I have documented my code.
  • I have added a release note file, following the contributors guidelines.
  • I have run pre-commit hooks and fixed any issue.

@TejasLamba2006
TejasLamba2006 requested a review from a team as a code owner November 20, 2026 18:15
@TejasLamba2006
TejasLamba2006 requested review from bogdankostic and removed request for a team November 20, 2026 18:15
@vercel

vercel Bot commented Nov 20, 2026

Copy link
Copy Markdown
Contributor

@TejasLamba2006 is attempting to deploy a commit to the deepset Team on Vercel.

A member of the Team first needs to authorize it.

@github-actions github-actions Bot added P2 Medium priority, add to the next sprint if no P1 available topic:tests topic:core type:documentation Improvements on the docs labels Nov 20, 2026
@github-actions

Copy link
Copy Markdown
Contributor

Coverage report

Click to see where and how coverage changed

FileStatementsMissingCoverageCoverage
(new stmts)
Lines missing
  haystack/core/pipeline
  base.py
  pipeline.py
Project Total  

This report was generated by python-coverage-comment-action

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

P2 Medium priority, add to the next sprint if no P1 available topic:core topic:tests type:documentation Improvements on the docs

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Pipeline.close_async skips a healthy component after an earlier close error

1 participant