Repository navigation
Hibernate 7: prevent outer session connection closure during nested withNewSession lazy loading - #16763
Hibernate 7: prevent outer session connection closure during nested withNewSession lazy loading#16763borinquenkid wants to merge 7 commits into
Conversation
jamesfredley
left a comment
There was a problem hiding this comment.
Not approving. The nested-transaction case in the new spec is covered. A disconnected outer session that already has synchronization, and no DataSource holder, can still bind a connection that the inner transaction then treats as its own.
jdaugherty
left a comment
There was a problem hiding this comment.
Not approving yet. The reported case is fixed: WithNewSessionLazyLoadSpec fails on 8.0.x with the NullPointerException from the issue and passes here. But the place where the connection is acquired breaks a case that works on 8.0.x today. The details are on James's thread. The other comments are smaller.
grails-data-hibernate5 has the same bug. WithNewSessionLazyLoadSpec, ported to grails-data-hibernate5-core on 8.0.x, fails with HibernateJdbcException: could not prepare statement, caused by PooledConnection has already been closed. 7.0.x has the same executeWithNewSession code in H5, so that fix belongs on 7.0.x in a separate PR. If you don't plan to pick it up, could you open an issue for it?
| Connection con = logicalConnection.getPhysicalConnection(); | ||
| if (con != null) { | ||
| try { | ||
| con.getMetaData(); |
There was a problem hiding this comment.
This call is what makes the fix work, and the comment above doesn't say why. getPhysicalConnection() returns the TransactionAwareDataSourceProxy handle, and the proxy only takes a target connection on its first real method call. getMetaData() is that call. When I replaced it with a call the proxy answers itself (hashCode()), WithNewSessionLazyLoadSpec failed again with PooledConnection has already been closed. Please say that in the comment, so the call doesn't get removed as dead code.
getMetaData() also makes the default LazyConnectionDataSourceProxy (dataSource.lazy: true) check out a pooled connection, and the request session holds it until it closes. That happens on every withNewSession while the request session is disconnected. It includes a request whose first database work is the withNewSession and whose session has nothing to lazy-load. After withNewSession { WnsllAuthor.count() } on an unused request session, the Tomcat pool reports 1 active connection on this branch and 0 on 8.0.x. con.getAutoCommit() binds the proxy's target the same way, but the lazy proxy answers it without fetching from the pool. With that change the spec still passes, and the pool stays at 0 until the request session runs a statement. (With dataSource.lazy: false, both calls check out a connection.)
The SQLException is logged at debug. If this call fails, the original bug comes back with nothing in the log at the default level. Logging it at warn would make that visible.
There was a problem hiding this comment.
Done in b986f36. The comment on the priming call now says that getPhysicalConnection() returns the TransactionAwareDataSourceProxy handle, that getAutoCommit() is the first real call that makes the proxy bind its holder, and that it must not be removed as dead code. The lazy proxy keeps an idle session from checking out a pooled connection, and priming only runs for a lazy source.
| } catch (HibernateException ex) { | ||
| throw convertHibernateAccessException(ex); | ||
| DataAccessException dae = convertHibernateAccessException(ex); | ||
| throw (dae != null ? dae : new HibernateSystemException(ex)); |
There was a problem hiding this comment.
After the change to convertHibernateAccessException, it can't return null. Both branches fall back to SessionFactoryUtils.convertHibernateAccessException, which ends with new HibernateSystemException(ex), and nothing in the repository overrides the method. So this second fallback never runs, and no test covers it. I'd drop it.
This path does need a test, though, because the issue's NullPointerException came from here. With the fix in place, the new spec no longer reaches it, and the new GrailsHibernateTemplateSpec test calls convertHibernateAccessException directly. Next to doExecute wraps HibernateException as DataAccessException, something like this:
when:
template.execute { sess -> throw new GenericJDBCException('msg', new SQLException('untranslatable')) }
then:
thrown(HibernateJdbcException)On 8.0.x that throws the NullPointerException. On this branch it throws HibernateJdbcException.
There was a problem hiding this comment.
At bb56460 convertHibernateAccessException has one fallback, in the JDBCException branch, for when the Spring translator returns null; the second fallback in doExecute is gone. GrailsHibernateTemplateSpec covers translation through doExecute. Please tell me if you still want the direct convertHibernateAccessException test from your snippet.
| DataAccessException dae = convertJdbcAccessException((JDBCException) ex, jdbcExceptionTranslator); | ||
| return dae != null ? dae : SessionFactoryUtils.convertHibernateAccessException(ex); | ||
| } | ||
| if (GenericJDBCException.class.equals(ex.getClass())) { |
There was a problem hiding this comment.
This branch can't run. GenericJDBCException extends JDBCException, so the instanceof JDBCException check above catches it first. The new test passes a GenericJDBCException, but it goes through the first branch. Could you delete this branch instead of adding the fallback to it?
There was a problem hiding this comment.
Removed. There is no GenericJDBCException branch in GrailsHibernateTemplate at bb56460; the instanceof JDBCException check handles it.
|
Updated in commit a8501b9:
|
jamesfredley
left a comment
There was a problem hiding this comment.
The earlier comment is fixed. The outer connection is acquired before the holder is saved. That thread is resolved. Not approving. A SUPPORTS scope that only calls withNewSession can still close the connection the outer session keeps.
jdaugherty
left a comment
There was a problem hiding this comment.
The round-1 items are fixed. The acquisition runs before the holder and the synchronizations are saved. It uses getAutoCommit(), the comment says why, and a failure is logged at warn. The dead GenericJDBCException branch and the doExecute fallback are gone, and the new execute test covers the path that threw the NullPointerException.
Not approving yet. James's new thread reproduces, and it breaks code that works on 8.0.x. The details are on his thread.
grails-data-hibernate5 has the same bug, so please fix it there too. The issue's scenario fails on 7.0.x: after the block, the request session's next query throws HibernateJdbcException, caused by PooledConnection has already been closed. executeWithNewSession there has the same shape, and its convertHibernateAccessException can return null in the same way. H5 is on 7.0.x, so that fix goes in its own PR against 7.0.x and merges up from there. Please include the same tests, with the SUPPORTS cases from this PR.
|
Updated in commit cba1b7f:
Also opened #16783 to address the same issue in |
jdaugherty
left a comment
There was a problem hiding this comment.
The synchronization fixes James's thread. A SUPPORTS scope that only calls withNewSession leaves the outer session usable again, with or without an inner transaction, read-only or read-write. The issue's lazy load under SUPPORTS now works too, also when nothing touches the outer session inside the scope. The new feature fails without the synchronization, so it guards the fix. A REQUIRES_NEW after the withNewSession, a scope that throws, and two withNewSession calls in one scope also leave the outer session usable, and no pool connection leaks.
Two non-blocking notes inline.
| } catch (SQLException e) { | ||
| LOG.warn("Could not initialize physical connection: {}", e.getMessage(), e); | ||
| } | ||
| if (txResources.isSynchronizationActive()) { |
There was a problem hiding this comment.
Non-blocking: the comment above explains the acquisition but not this registration, and without a reason it reads as redundant with the session synchronization. One line would keep it: when nothing in the scope uses the outer session, no session synchronization is registered, and the JDBC ConnectionSynchronization would close the connection the outer session still holds.
There was a problem hiding this comment.
Done in b986f36. The registration in NonTransactionalSynchronizationOuterSessionStrategy.prepare() now carries your one-line reason: when nothing in the scope uses the outer session, no session synchronization is registered, and Spring's JDBC ConnectionSynchronization would close the connection the primed session still holds.
| new WnsllAuthor(name: 'Tenar').save(flush: true) | ||
| } | ||
| } | ||
| assert WnsllBook.count() == 1 |
There was a problem hiding this comment.
Non-blocking: this query registers the session synchronization, so this feature passes without the new synchronization too. Without it, the feature covers the issue's scenario under SUPPORTS, which is the case that needed it.
There was a problem hiding this comment.
The WithNewSessionLazyLoadSpec feature "request-bound session remains usable after nested withNewSession inside SUPPORTS transaction without in-scope query" leaves the scope without touching the outer session and queries it afterwards, so it needs the synchronization. The older feature that queries inside the scope is still there as well.
|
Pushed commit
@jdaugherty if everything looks good, please approve when you have a moment. |
jamesfredley
left a comment
There was a problem hiding this comment.
The connection is acquired before the holder is saved, and a SUPPORTS scope now disconnects the outer session before Spring closes it. Those threads are resolved. Not approving. beforeCompletion still disconnects an active outer transaction, so a commit can report success and leave the inserted row gone. With lazy wrapping off, the outer session also checks out a connection it never uses.
jamesfredley
left a comment
There was a problem hiding this comment.
The connection holder is still saved after the connection is acquired. The disconnect no longer runs on an active outer transaction, and an idle session no longer checks out a connection when lazy wrapping is on. Not approving. With lazy wrapping off, the disconnected outer session is not primed. A SUPPORTS scope that then starts a REQUIRED transaction can close the connection that primed session keeps. This is a bugfix, so 8.0.x can keep it.
| // to bind the proxy without eagerly checking out a connection from the pool. When lazy wrapping is off | ||
| // or the data source is non-transaction-aware, acquiring the connection eagerly would check out a pooled | ||
| // connection unnecessarily and cause a single-connection pool to time out. | ||
| if (this.transactionAware && this.lazyConnection && |
There was a problem hiding this comment.
This priming runs only when lazy wrapping is on. With dataSource.lazy=false and transactionAware still true, a disconnected outer session is not given its own connection. A lazy load inside withNewSession { withTransaction { ... } } then takes the inner transaction's connection, and the outer session keeps that connection after the inner transaction closes it. The idle outer session must not check out a pooled connection, and a real use of that session must not share the inner transaction's connection.
There was a problem hiding this comment.
Addressed in bb56460, with a different shape than the thread asked for. An idle outer session is not primed at all when dataSource.lazy=false. On restore, a session that was not connected on entry and is connected at the end is disconnected, so it never keeps the inner transaction's connection. WithNewSessionNonLazyPoolSpec runs against a one-connection pool with lazy=false, including "lazy loading in inner transaction with non-lazy pool drops closed connection on restore".
| } | ||
|
|
||
| @Override | ||
| public void beforeCompletion() { |
There was a problem hiding this comment.
Inside a SUPPORTS scope, this beforeCompletion does not run when a REQUIRED transaction suspends that synchronization. The REQUIRED transaction reuses the connection this scope primed and closes it, so a later query on the outer session fails with a closed connection. Disconnect the primed session when the synchronization is suspended.
There was a problem hiding this comment.
Addressed in commit 1b4eaaf:
- Encapsulated outer session lifecycle management into explicit OuterSessionLifecycleStrategy implementations (ActiveTransactionOuterSessionStrategy, NonTransactionalSynchronizationOuterSessionStrategy, DefaultOuterSessionStrategy, and NoOpOuterSessionStrategy).
- Implemented suspend() in NonTransactionalSynchronizationOuterSessionStrategy to call logicalConnection.manualDisconnect(). When an inner REQUIRED transaction begins and Spring suspends outer synchronizations, the primed session drops its connection immediately, preventing the inner transaction from reusing or closing it.
- Implemented beforeCompletion() to disconnect before Spring's ConnectionSynchronization closes the physical connection when no inner transaction suspends the scope.
- Added dedicated unit tests in OuterSessionLifecycleStrategySpec covering each strategy and interaction invariants in isolation, and added integration coverage in WithNewSessionLazyLoadSpec for a SUPPORTS scope executing a nested withNewSession followed by a REQUIRED transaction.
1b4eaaf to
c9eeb39
Compare
|
Rebased onto We have refactored outer session lifecycle handling in
|
jamesfredley
left a comment
There was a problem hiding this comment.
A disconnected transaction-aware outer session is primed, and a SUPPORTS scope that then starts a REQUIRED transaction disconnects that connection. Not approving. A non-lazy data source still checks out a connection for an idle outer session, and a closed bound session is left in place.
| return; | ||
| } | ||
|
|
||
| Connection con = logicalConnection.getPhysicalConnection(); |
There was a problem hiding this comment.
With transactionAware=false, lazy=false, a one-connection pool, and an open disconnected request session, withNewSession { withTransaction { /* query */ } } checks out the only connection for the unused outer session. The inner query then waits until the pool times out. The same priming in NonTransactionalSynchronizationOuterSessionStrategy.prepare() does this too. Prime a transaction-aware source, and skip a source that cannot lend its connection.
There was a problem hiding this comment.
Priming is now gated on a lazy connection proxy (lazyConnection) in both NonTransactionalSynchronizationOuterSessionStrategy.prepare() and the restore path, so a non-lazy source, transaction-aware or not, never checks out its only pooled connection for an idle outer session. The gate is the lazy proxy rather than a transaction-aware check; if you want it keyed on the transaction-aware proxy instead, tell me which case it misses. The one-connection lazy=false scenarios are in WithNewSessionNonLazyPoolSpec.
| boolean hasConnectionHolder = txResources.getResource(dataSource) != null; | ||
| boolean hasSynchronization = txResources.isSynchronizationActive(); | ||
|
|
||
| if (!hasOpenSession && !hasConnectionHolder && !hasSynchronization) { |
There was a problem hiding this comment.
A closed session still bound, with no connection holder and no synchronization, selects the strategy that never unbinds that holder. executeWithNewSession() then throws IllegalStateException because a value is already bound, and the callback does not run. Use the no-op strategy only when no holder is bound, and unbind a holder whose session is closed.
There was a problem hiding this comment.
Fixed in bb56460. OuterSessionLifecycleStrategy.from returns the no-op strategy only when no session holder is bound, no connection holder is bound and no synchronization is active. A bound holder whose session is closed goes to DefaultOuterSessionStrategy, which unbinds it and rebinds it afterwards. Covered by "withNewSession runs and rebinds holder when outer request session was closed before scope" in WithNewSessionNonLazyPoolSpec.
|
Updated branch
|
…ecuteWithNewSession Isolate outer session management across nested withNewSession scopes using dedicated OuterSessionLifecycleStrategy implementations: - ActiveTransactionOuterSessionStrategy suspends and restores active transactions symmetrically without prematurely disconnecting physical connections. - NonTransactionalSynchronizationOuterSessionStrategy primes connections only when lazyConnection is enabled, disconnects on restore if not physically connected at prepare, and coordinates with Spring session synchronization. - DefaultOuterSessionStrategy unbinds and rebinds session holders when synchronization is inactive or when the bound session was closed prior to the scope. Route new session execution through execute(callable::call) to preserve flush modes and translate Hibernate exceptions to Spring DataAccessException. Guard against null DataSource lookups in transaction resource operations.
67880b4 to
bb56460
Compare
|
Rebased onto
|
… synchronization is registered Say in the code why getAutoCommit() must stay (it is the first real call that makes TransactionAwareDataSourceProxy bind its holder) and why the scope-end synchronization exists (no session synchronization is registered when the scope never uses the outer session). Correct the DefaultOuterSessionStrategy javadoc, which still described priming.
…pened execute() asked the SessionFactory for the current session, which registered the new holder with the surrounding synchronization. A transaction manager that began inside the callback (a named connection's withTransaction) then saw a holder synchronized with a transaction and opened a second session, so the session handed to the closure had no active transaction. The callback now runs on the session withNewSession bound, with the same exception translation and flush handling. Claude-Session: https://claude.ai/code/session_01Dmki4SQGwRSj8pRJ5KCWyX
|
@jdaugherty @jamesfredley head b715c03: |
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## 8.0.x #16763 +/- ##
==============================
==============================
🚀 New features to boost your workflow:
|
|
@jdaugherty @jamesfredley merged 8.0.x into the branch (1370979, no conflicts, no other change) so CI runs without the unrelated upstream failures; the round-3 items are still answered by b715c03, ready for re-review. |
…cleanup of the new one throws
|
@jamesfredley @jdaugherty the Hibernate 7 |
jamesfredley
left a comment
There was a problem hiding this comment.
Not approving. The outer session is put back when cleanup throws. The callback now runs through a path that keeps a read-only session read-only inside a write transaction, and that flushes after the callback has already committed.
|
|
||
| return callable.call(newSession); | ||
| final Session scopedSession = newSession; | ||
| return doExecute(callable::call, false, scopedSession); |
There was a problem hiding this comment.
With the open-session read-only setting on, this sets the new session read-only before the closure runs. A write transaction inside the closure reuses that session, so a change is discarded even though the commit succeeds. When the template flushes eagerly and the closure commits its own transaction, the flush after the callback throws because no transaction is active, after the work has already committed. Keep the session writable for that write, and flush only while a transaction is active.
…s only inside a transaction The callback ran through the template's execute path, which marked the new session read-only when the open-session read-only setting was on, so a write transaction the closure committed on that session was discarded, and which flushed after the callback even when the closure had already committed its own transaction, so the flush threw for a missing transaction. The new session is not marked read-only, and it is flushed after the callback only while a transaction is active. Claude-Session: https://claude.ai/code/session_01Dmki4RzWwreEC2GkkyFnvL
…sconnected outer session On a lazy data source with a pool of one connection, the strategy for an outer session inside a synchronization scope opened a physical connection for that session before the nested session began. A transaction in the closure then waited for a connection that never came. The outer session stays disconnected now; the synchronization still disconnects it if it connects later. Claude-Session: https://claude.ai/code/session_01Dmki4RzWwreEC2GkkyFnvL
|
@jamesfredley Both findings are fixed in the new commit. The callback ran through the template's
The same push removes the prime you found in #16784 from the Hibernate 7 strategy: |
✅ All tests passed ✅🏷️ Commit: 414737a Learn more about TestLens at testlens.app/docs. |
jamesfredley
left a comment
There was a problem hiding this comment.
Requested changes. A disconnected outer session can still keep a closed connection. An empty withNewSession also fails when the session is not allowed to ask for its transaction.
| } | ||
| } | ||
|
|
||
| private static boolean hasActiveTransaction(Session session) { |
There was a problem hiding this comment.
The session is not allowed to call getTransaction. An empty withNewSession used to finish. It now throws. Check whether a transaction is active without that call.
Fixes #16571.
Problem
When a transaction on a request-bound Hibernate session completes, Spring's
disconnectOnCompletion(session)disconnects the session's physical JDBC connection, allowing it to be returned to the connection pool until needed.If the application subsequently opens a nested
withNewSessionand inside it starts a transaction (withTransaction), Spring'sTransactionSynchronizationManagerbinds the inner transaction's physical connection to the thread under thedataSourceresource key.If an entity loaded earlier in the outer request session is touched inside that nested scope (for example, accessing a lazy association like
book.author.name), the outer session detects that it is physically disconnected and requests a connection from thedataSource. Because thedataSourceis wrapped in Spring'sTransactionAwareDataSourceProxy, it returns the thread-bound connection of the inner transaction.When the inner transaction completes, it closes its connection. The outer session is left holding that closed connection, and subsequent operations fail with:
Additionally, if
convertJdbcAccessExceptionreturnednullbecause the database vendor error code could not be mapped,GrailsHibernateTemplate.doExecutewould throwNullPointerExceptionwhen trying to throw the result ofconvertHibernateAccessException.Change
GrailsHibernateTemplate.executeWithNewSession, before unbinding the existing session holder and entering the new session scope, check whether the outer session is open but physically disconnected (!logicalConnection.isPhysicallyConnected()). If so, acquire and initialize its physical connection (con.getMetaData()) from the data source pool beforehand. This ensures the outer session holds its own independent connection and does not borrow the inner transaction's thread-bound connection during lazy loading.GrailsHibernateTemplate.convertHibernateAccessException, fall back toSessionFactoryUtils.convertHibernateAccessException(ex)ifconvertJdbcAccessExceptionreturnsnull.GrailsHibernateTemplate.doExecute, fall back tonew HibernateSystemException(ex)ifconvertHibernateAccessExceptionreturnsnull.Tests
WithNewSessionLazyLoadSpec: reproduces the scenario where a lazy association on an outer request-bound session entity is loaded inside a nestedwithNewSession { withTransaction { ... } }, and verifies that the outer session remains open and usable for subsequent queries and updates.GrailsHibernateTemplateSpec: added test verifying fallback toSessionFactoryUtilswhen JDBC exception translation returns null.