Skip to content

fix: sync option holdings with the brokerage in live mode - #9937

Open
Romazes wants to merge 9 commits into
QuantConnect:masterfrom
Romazes:feature-clear-expired-option-holdings
Open

Romazes wants to merge 9 commits into
QuantConnect:masterfrom
Romazes:feature-clear-expired-option-holdings

Conversation

@Romazes

@Romazes Romazes commented Oct 23, 2026 •

Copy link
Copy Markdown
Collaborator

Description

In live mode an expired option contract stays in the portfolio until a restart, and an early assignment on a short contract is never booked. Lean clears or reduces an option holding only in BrokerageTransactionHandler.HandleOptionNotification, on a brokerage OptionNotification with the new position. The backtesting brokerage raises it from the delisting at expiry, and Interactive Brokers raises it from its portfolio updates. No other brokerage plugin raises it (Alpaca, Charles Schwab, Clear Street, Public, Tastytrade, Tradier, TradeStation, WeBull, Eze, Wolverine, Bloomberg FIX, TerminalLink, Samco, Zerodha), and AlgorithmManager tracks delistings only in non-live mode. Charles Schwab support and the Alpaca docs confirm those APIs send nothing when a contract expires, so the plugins have nothing to raise it from.

Seen on a Charles Schwab deployment trading 0DTE SPXW (plugin issue 123): seven contracts stayed in Portfolio after the 16:00 ET expiry with Invested = true, every expiry added a set, and a restart before Schwab's overnight cleanup loaded them back:

BrokerageSetupHandler.Setup(): Has existing holding: SPXW  261019P07590000: 7.0 @ USD0.2 - Market: USD0.025

Now ProcessSynchronousEvents in live mode syncs the option holdings with the brokerage once a day, inside the cash sync window (07:45 ET) and its no-fill guard, before the cash read. When Lean holds an option it reads GetAccountHoldings once and runs every invested option whose quantity differs through HandleOptionNotification with the brokerage quantity: 0 for a contract past expiry whatever the brokerage still reports, otherwise the quantity the brokerage holds, 0 when it no longer lists the contract. That is the same path a brokerage position update takes, so expiry books an exercise, assignment or expired worthless through the option exercise model, and a smaller short position before expiry books an early assignment, full or partial, with the shares moved. The fills are applied synchronously, so the next pass finds no difference and books nothing more. One holdings read a day, only when an option is held, repeated once about 10 seconds later on a day with a booking because the cash sync's fill check asks for a resync. A failed read is logged and retried at the next time step like a failed cash read, with the same five attempts in a row before the sync throws. A Start syncing option holdings... trace line before the read and a Brokerage option holdings count N. Exit line after the pass show it in the live log. No per-brokerage switch, no new state, backtests keep the delisting path.

After a redeploy between the close and 07:45 ET the underlying of a held option has no price yet: BrokerageSetupHandler seeds a price only for the held security, the live feed drops off-hours data, and the first bar arrives at 09:30 ET. The exercise model reads underlying.Close to decide in the money, and at 0 every put is in the money by the full strike and every call is worthless. So before the notification the sync seeds the underlying with GetLastKnownPrice from the history provider when its price is 0; when history returns nothing the contract is skipped for the day with an error line instead of being settled at 0.

The Interactive Brokers handler stays: its position updates book expiry and assignment the same day, and the daily sync then finds no difference. Not covered, as before: a long contract exercised outside Lean with no exercise order in Lean.

Related Issue

https://github.com/QuantConnect/Lean.Brokerages.CharlesSchwab/issues/123

Motivation and Context

An expired contract kept in the portfolio keeps Invested true, so the algorithm's own position guards blocked trading the next session. Nothing could close it: the engine refuses orders on an expired contract and the brokerage rejects them.

Requires Documentation Change

No

How Has This Been Tested?

  • BrokerageTransactionHandlerTests.ProcessSynchronousEventsClearsExpiredOptionHoldingsOnlyInLiveMode, six cases of a SPY option held past expiry with 100 shares of the underlying, two daily passes at 08:00 ET inside the cash sync window, the first the day after expiry, the second a day later and finding nothing to book. Live mode books one exercise order and the option quantity goes to 0: OTM leaves the shares at 100, an ITM long call or short put ends with 200, an ITM long put or short call ends with 0. Backtest mode books nothing and the quantity stays 7.
  • BrokerageTransactionHandlerTests.ProcessSynchronousEventsBooksEarlyAssignmentFromBrokerageHoldings, three cases: a short SPY call ITM the day before expiry, the test brokerage no longer lists the contract, lists -3, or lists the same -7. Full assignment ends with 0 contracts and -600 shares, partial with -3 and -300, equal books nothing. The second daily pass, on expiry day before the close, books nothing more.
  • BrokerageTransactionHandlerTests.ProcessSynchronousEventsSeedsUnderlyingPriceFromHistoryWhenMissing, three cases of a SPY 450 call whose underlying has no price, with a history provider fake that returns a 455 bar or nothing: a short call no longer listed at the brokerage is seeded and assigned in full (0 contracts, -600 shares); with no history it is skipped (-7 contracts, 100 shares); an expired long call is seeded and exercised (0 contracts, 200 shares). Without the seed a call at price 0 is worthless, so the first and third cases fail on the previous code.
  • BrokerageTransactionHandlerTests.ProcessSynchronousEventsRetriesTheOptionHoldingsSyncWhenTheBrokerageReadFails, three cases of an expired SPY 450 call in the money with a brokerage fake whose holdings read throws a given number of times: one or four failed reads book the exercise on the next pass at the same clock time (0 contracts, 200 shares); five failed reads throw the cash sync attempts exception on the fifth pass and book nothing.
  • The existing OptionExpirationEmitsOrderEvents and early assignment cases cover the exercise outcomes and show a second notification for the same contract adds no order.

Types of changes

  • Bug fix (non-breaking change which fixes an issue)
  • Refactor (non-breaking change which improves implementation)
  • Performance (non-breaking change which improves performance. Please add associated performance test and results)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to change)
  • Non-functional change (xml comments/documentation/etc)

Checklist:

  • My code follows the code style of this project.
  • I have read the CONTRIBUTING document.
  • I have added tests to cover my changes.
  • All new and existing tests passed.
  • My branch follows the naming convention bug-<issue#>-<description> or feature-<issue#>-<description>

🤖 Generated with Claude Code

- the delisting at expiry is processed only in backtesting, so in live an expired option stayed in the portfolio until restart unless the brokerage sent a position update, which only IB does
- the transaction handler now runs every held option past its expiry through the option notification path each time step, the same path a brokerage position update with zero takes
- Lean books the exercise or the expiry itself, so no brokerage call and no per-brokerage switch
@Romazes Romazes self-assigned this Oct 23, 2026
- Securities.Values is a cached snapshot, so the handler can run inside the loop and the intermediate symbol list was an extra pass for nothing
@Romazes
Romazes force-pushed the feature-clear-expired-option-holdings branch from 4057e20 to 27653ba Compare October 23, 2026 15:48
- a scan of every security on each live time step was too frequent for a contract that only expires at the close, so the check moved inside the daily cash sync window, one scan a day with no new state
- it runs before the cash read, so the cash an expiry books is replaced by the brokerage figure right after
@Romazes
Romazes force-pushed the feature-clear-expired-option-holdings branch from 27653ba to d95c68d Compare October 23, 2026 15:49
- the expiry date only covers contracts held to expiry; an in the money American style short option can be assigned on any day before, and a brokerage without a position stream reports nothing, so Lean kept the full short and no shares
- the daily pass now reads the brokerage holdings once, only when an option is held, and runs every option whose quantity differs through the same path a brokerage position update takes, which already books full and partial early assignment and moves the shares
- a contract past expiry is cleared whatever the brokerage still reports, Charles Schwab keeps it until an overnight cleanup
- a failed holdings read is logged and skipped for the day, same as a failed cash read, so one REST error does not stop a live algorithm
- the test brokerage got a settable holdings list, the backtesting one mirrors Lean's own portfolio and never differs; the full assignment case leaves the contract out of the list, the way a brokerage reports a closed position
@Romazes Romazes changed the title fix: clear expired option holdings in live mode fix: sync option holdings with the brokerage in live mode Oct 26, 2026
- after a redeploy between the close and 07:45 ET the underlying of a held option has no price: setup seeds only the held security and the live feed drops off-hours data, so the exercise model read 0 and called every put in the money and every call worthless
- the sync now takes the last known price from history when the underlying has none, and skips the contract for the day when history has nothing, so no contract is settled at a zero price
- the sync tests run their second pass on the next day; the cash sync flag of the same day had been skipping it, so the pass that finds no difference was never exercised
- the daily pass left nothing in the live log, and the first proof on Charles Schwab will be a log: a start line before the read and an exit line with the option holdings count, the pattern of the old-order cleanup in the same handler
- the holdings read returns every security type while the lookups are by option symbol, so only option rows go into the dictionary and the count in the exit line means option holdings
- a failed holdings read only returned, so when the cash read then succeeded the sync window closed and the option sync was lost for the day; it came back the same day only when the cash read failed too
- the read now counts as a failed sync attempt: retried at the next time step with the same five-attempt limit as the cash sync, so a failed cash read also repeats the holdings read on each retry
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant