I. Introduction

On December 21, 2022, The Nasdaq Stock Market LLC (“Nasdaq” or “Exchange”) filed with the Securities and Exchange Commission (“Commission”), pursuant to Section 19(b)(1) of the Securities Exchange Act of 1934 (“Act”) ^[1] and Rule 19b-4 thereunder,^[2] a proposed rule change to replace the static holding period requirements for Midpoint Extended Life Orders and Midpoint Extended Life Orders Plus Continuous Book with dynamic holding periods. The proposed rule change was published for comment in the Federal Register on January 10, 2023.^[3] On February 22, 2023, pursuant to Section 19(b)(2) of the Act,^[4] the Commission designated a longer period within which to approve the proposed rule change, disapprove the proposed rule change, or institute proceedings to determine whether to disapprove the proposed rule change.^[5] On March 9, 2023, the Exchange filed Amendment No. 1 to the proposed rule change, which amended and superseded the proposed rule change as originally filed. On April 7, 2023, the Commission provided notice of filing of Amendment No. 1 and instituted proceedings to determine whether to approve or disapprove the proposed rule change, as modified by Amendment No. 1.^[6] On July 6, 2023, pursuant to Section 19(b)(2) of the Act,^[7] the Commission designated a longer period on proceedings to determine whether to approve or disapprove the proposed rule change.^[8] On July 18, 2023, the Exchange filed Amendment No. 2 to the proposed rule change, which amended and superseded the proposed rule change as amended by Amendment No. 1. The Commission received comments on the proposed rule change.^[9] The Commission is publishing this Notice and Order to solicit comment on Amendment No. 2 in Sections II and III below, which sections are being published verbatim as filed by the Exchange, and to approve the proposed rule change, as modified by Amendment No. 2, on an accelerated basis.

II. Self-Regulatory Organization's Statement of the Terms of Substance of the Proposed Rule Change

The Exchange proposes to amend Rules 4702(b)(14) and (b)(15) of the Exchange's Rulebook to replace the static holding period requirements for Midpoint Extended Life Orders and Midpoint Extended Life Orders Plus Continuous Book with dynamic holding periods. This Amendment No. 2 ( printed page 62851) supersedes the original filing and Amendment No. 1 ^[10] in their entireties.

The text of the proposed rule change is available on the Exchange's website at https://listingcenter.nasdaq.com/​rulebook/​nasdaq/​rules, at the principal office of the Exchange, and at the Commission's Public Reference Room.

III. Self-Regulatory Organization's Statement of the Purpose of, and Statutory Basis for, the Proposed Rule Change

In its filing with the Commission, the Exchange included statements concerning the purpose of and basis for the proposed rule change and discussed any comments it received on the proposed rule change. The text of these statements may be examined at the places specified in Item IV below. The Exchange has prepared summaries, set forth in sections A, B, and C below, of the most significant aspects of such statements.

A. Self-Regulatory Organization's Statement of the Purpose of, and Statutory Basis for, the Proposed Rule Change

1. Purpose

The Exchange proposes to amend Rules 4702(b)(14) and (15) of the Exchange's Rulebook to replace the static 10 millisecond holding period requirements for its Midpoint Extended Life Order (“M-ELO”) and Midpoint Extended Life Order Plus Continuous Book (“M-ELO+CB”) Order Types with dynamic holding periods (“Dynamic M-ELO and M-ELO+CB” or collectively, “Dynamic M-ELO”).

Background

In 2018, the Exchange introduced the M-ELO, which is a Non-Displayed Order priced at the Midpoint between the National Best Bid and Offer (“NBBO”) and which is eligible for execution only against other eligible M-ELOs and only after a minimum of one-half second passes from the time that the System accepts the order (the “Holding Period”).^[11] In 2019, the Exchange introduced the M-ELO+CB, which closely resembles the M-ELO, except that a M-ELO+CB may execute at the midpoint of the NBBO, not only against other eligible M-ELOs (and M-ELO+CBs), but also against Non-Displayed Orders with Midpoint Pegging and Midpoint Peg Post-Only Orders (“Midpoint Orders”) that rest on the Continuous Book for at least one-half second and have Trade Now enabled.^[12]

When the Exchange designed M-ELO, it originally set the length of the Holding Period at one-half second because it determined that this time period would be sufficient to ensure that likeminded investors would interact only with each other, and with minimal market impacts. The Exchange believed that the longer length of the M-ELO Holding Period and its simplicity in design would provide greater protection for participants than they could achieve through competing delay mechanisms.

In 2020, however, the Exchange shortened the length of the Holding Period to 10 milliseconds.^[13] The Exchange did so after studying two years of actual use and performance of M-ELOs, as well as customer feedback. That is, the Exchange came to understand that, while users of M-ELO and M-ELO+CB are less concerned with achieving rapid executions of their Orders than are other participants, they are not indifferent about the length of time in which their M-ELOs and M-ELO+CBs must wait before they are eligible for execution. Indeed, participants informed the Exchange that in certain circumstances, such as when they sought to trade symbols that on average had a lower time-to-execution than a half-second, they were reticent to enter M-ELOs or M-ELO+CBs. They indicated that the associated Holding Periods for these Order Types were longer than necessary to achieve the desired protections and that, during the residual portion of the Holding Periods, they risked losing out on favorable execution opportunities that would otherwise be available to them had they placed a non-MELO order.

Based upon this feedback, the Exchange studied the potential effects of reducing the length of the Holding Periods for both M-ELOs and M-ELO+CBs (as well as for Midpoint Orders that would execute against M-ELO+CBs). Ultimately, the Exchange determined that it could reduce the Holding Periods to 10 milliseconds without compromising the protective power that M-ELO and M-ELO+CB are intended to provide to participants and investors.^[14] Thus, the Exchange determined that shortening the Holding Periods to 10 milliseconds for M-ELOs and M-ELO+CBs would increase the efficacy of the mechanism while not undermining the power of those Order Types to fulfill their underlying purpose of minimizing market impacts. At the same time, the Exchange determined that a reduction in the Holding Periods to 10 milliseconds would dramatically add to the circumstances in which M-ELOs and M-ELO+CBs would be useful to participants. In its M-ELO Timer Approval Order, the Commission agreed with the Exchange:

The Commission notes that, with the proposed ten-millisecond Holding Period and Resting Period, M-ELOs and M-ELO+CBs would continue to be optional order types that are available to investors with longer investment time horizons, including institutional investors. The Commission also believes that the proposal could make M-ELOs and M-ELO+CBs more attractive for securities that on average have a time-to-execution of less than one-half second and, for investors who currently do not use M-ELOs and M-ELO+CBs for these securities, provide optional order types that could enhance their ability to participate effectively on the Exchange. The Commission notes that, if market participants determine that the proposal would make M-ELOs and M-ELO+CBs less attractive for their particular investment objectives, such market participants may elect to reduce or eliminate their use of these optional order types. Moreover, as noted above, the Exchange will continue to conduct real-time surveillance to monitor the use of M-ELOs and M-ELO+CBs to ensure that such usage remains appropriately tied to the intent of the order types. If, as a result of such surveillance, the Exchange determines that the shortened ( printed page 62852) Holding Period does not serve its intended purpose or adversely impacts market quality, the Exchange would seek to make further recalibrations.^[15]

For similar reasons and with even better potential results for participants, the Exchange now proposes to further refine the length of the Holding Periods for M-ELOs and M-ELO+CBs, this time through the application of innovative and patent pending machine learning technology.

Dynamic M-ELO

After receiving feedback from participants that even 10 millisecond Holding Periods for M-ELO and M-ELO+CB may, at times, exceed what is necessary to accomplish the underlying intent of these Order Types, the Exchange began to experiment with making further refinements to the duration of the Holding Periods. Ultimately, the Exchange concluded that shorter Holding Periods could achieve the same, if not better results for participants in terms of mark-outs, but not in all circumstances. That is, where prices of the underlying securities are stable, and not subject to imminent unfavorable changes, M-ELOs and M-ELO+CBs face lower risks of confronting spread-crossing orders, such that shorter Holding Periods could suffice to protect M-ELOs and M-ELO+CB from such orders. In periods of heightened price volatility, however, M-ELOs and M-ELO+CBs also face heightened risks, such that longer Holding Periods would continue to be beneficial in protecting M-ELOs and M-ELO+CBs from such risks. Thus, the Exchange determined that another across-the-board reduction of the static 10 millisecond Holding Periods would be sub-optimal because it could impact the performance of the M-ELO and M-ELO+CB Order Types during periods of heightened volatility.

In light of these observations, the Exchange tasked its artificial intelligence and machine learning laboratory (the “AI Core Development Group”) to explore whether it could employ these innovative technologies to optimize the length of M-ELO and M-ELO+CB Holding Periods during various states of price volatility, and then to vary the lengths of the Holding Periods dynamically during the lifecycles of M-ELOs and M-ELO+CBs, with the objectives of improving the performance of these Order Types while also further reducing opportunity costs.

As the Exchange explains in greater depth in the attached White Paper,^[16] the AI Core Development Group proceeded to develop an artificial intelligence-based timer control system that will achieve these objectives.^[17] The AI Core Development Group did so by using reinforcement learning techniques—machine learning paradigms which develop optimal solutions to problems over time by taking actions to solve them, generating feedback on the results of such actions, applying that feedback to direct and improve the next round of solutions, and then repeating the feedback loop until the paradigm achieves optimized solutions.

In this instance, the AI Core Development Group applied reinforcement learning techniques to a simulation of the M-ELO Book that it constructed using a representative data set from the first quarter of 2022 (the “Training Period”). The Training Period data consisted of 380 out of the 6,257 symbols on the M-ELO Book (accounting for approximately 67 percent of M-ELO volume). The symbols chosen reflect both actively-traded and thinly-traded securities, and both low-priced and high-priced securities.

The AI Core Development Group then developed a machine learning model and applied it to the Training Period data. The Group programmed the model to value the achievement of higher fill rates or lower mark-outs than that which occurred in a historical simulation of M-ELOs and M-ELO+CBs involving the Training Period data.^[18] The Group then programmed the model to seek to achieve its goals by taking one of five possible actions with respect to the duration of the Holding Periods at 30 second intervals ^[19] for each symbol during each trading day of the Training Period. That is, at each 30 second internal, the model evaluated market conditions for each symbol over the prior 30 second period and either kept the Holding Periods the same, increased/decreased them by 0.25 milliseconds, or increased/decreased them by 0.50 milliseconds.^[20] After each decision-making round, the model utilized the results to inform its actions at the next 30 second increment.

In making its decisions, the model (again, drawing upon a combination of historical SIP and M-ELO-specific data) considered 142 categories of data points.^[21] A confluence of data points that correlated with an increase in volatility tended to cause the model to increase the durations of Holding Periods, including increases in the standard deviation of NBBO prices, the number of unique participants placing sell orders on M-ELO and M-ELO+CB, and the volume-weighted average of the NBBO spread. Conversely, a confluence of data points that correlated with greater price stability tended to cause the model to decrease the durations of Holding periods, such as an increase in the median and max number of shares per trade and the number of resting bids left in the M-ELO and M-ELO+CB Book.

The AI Core Development Team produced variations of its model that prioritized achievement of the lowest mark-outs, the highest fill rates, and a blend of these two objectives.^[22] Through ( printed page 62853) a process of learning and experimentation involving a combination of historical and simulated data, the AI Core Development Group settled on a Dynamic M-ELO model that achieved substantial simulated performance improvements for users of M-ELO and M-ELO+CB—both in terms of mark-outs and fill rates—as compared to the static 10 millisecond Holding Periods. As the White Paper explains in greater detail, Dynamic M-ELO yielded an average combined volume-weighted (simulated) improvement of 31.7 percent, including a 20.3 percent increase in fill rates and a 11.4 percent reduction in mark-outs.^[23] The White Paper provides a more fulsome explanation of these improvements.^[24]

Based upon these exciting results, the Exchange now proposes to amend Rule 4702(b)(14) and (15) to replace the static 10 millisecond timers applicable to M-ELO and M-ELO+CB with Dynamic M-ELO Holding Periods. Using the Exchange's “proprietary assessment of market conditions” ^[25] and patent pending technology, the Dynamic M-ELO system will evaluate and, as it deems necessary, adjust the length of the Holding Periods for each symbol comprising M-ELOs and M-ELO+CBs (and Midpoint Orders on the Continuous Book that opt to interact with M-ELO+CBs after resting on the Book) every 30 seconds throughout the Market Hours (each such 30 second interval, a “Change Event”). In so doing, Dynamic M-ELO will help participants to achieve a more optimized blend of the underlying purposes of the M-ELO and M-ELO+CB Order Types: protection against adverse selection (low mark-outs) without sacrificing opportunities to achieve high-quality executions (high fill rates).

A proposed M-ELO or M-ELO+CB with a Dynamic Holding Period will operate as follows. At the outset of Market Hours (approximately 9:30:00 a.m.), the Exchange will impose initial Holding Periods of 1.25 milliseconds for M-ELOs and M-ELO+CBs in all symbols. Thereafter, Holding Periods for a given symbol will become eligible to change dynamically from the initial duration beginning at 9:30:30 a.m. and then at 30 second intervals thereafter during Market Hours. The Exchange will then apply to the M-ELO or M-ELO+CB Order a Holding Period that is of the duration that prevailed at the time of entry. For example, if participant A enters a M-ELO for symbol XYZ at 9:30:25 a.m., then Holding Period for that M-ELO will be 1.25 milliseconds. If at 9:30:30:00 a.m., the System decides to lower the duration of the Holding Period by 0.50 milliseconds, and then participant B enters a M-ELO for symbol XYZ at 9:30:45 a.m., then the System will assign a 0.75 millisecond Holding Period to participant B's M-ELO. To be clear, the System will determine Dynamic M-ELO Holding Periods independently for M-ELOs and M-ELO+CBs in each symbol.

During normal market conditions, the range of potential Holding Period durations for M-ELOs and M-ELO+CBs will be between 0.25-2.50 milliseconds, with the Holding Period duration being eligible to change by increments of either 0.25 or 0.50 milliseconds at each Change Event. Thus, if the Holding Period for a M-ELO in symbol XYZ is set at 0.75 milliseconds at 2:22:11 p.m., and at 2:22:41 p.m., the System determines to increase the duration of the Holding Period, it may do so only by 0.25 or 0.50 milliseconds during that event.

When a Change Event occurs, and the System determines to adjust the duration of a Holding Period for a symbol, that adjustment will apply, not only to all M-ELOs and M-ELO+CBs for that symbol entered within the 30 second period after the Change Event occurs, but also to M-ELOs and M-ELO+CBs entered prior to the Change Event with unexpired Holding Periods (with applicability retroactive to the time of Order acceptance). Thus, if a participant enters a M-ELO in symbol XYZ at 1:14:299 p.m., and the prevailing Holding Period applicable to that M-ELO is 2 milliseconds, and at 1:14:30 p.m., the System modifies the Holding Period to be 1.5 milliseconds, then the M-ELO will become eligible to execute at 1:14:3005 p.m. This is the case because the M-ELO will have already expended 1 millisecond of its Holding Period as of the time of the Change Event; thereafter, the M-ELO will need to rest only another 0.5 milliseconds to become eligible to execute under the new 1.5 millisecond Holding Period (as measured from 1:14:299 p.m.). This last feature ensures that the M-ELO Book maintains time priority among M-ELOs and M-ELO+CBs in a dynamic environment. That is, it ensures that no M-ELO or M-ELO+CB with an unexpired Holding Period at the time of a Change Event will end up becoming eligible to execute later than a M-ELO entered after the Change Event which has a shorter Holding Period applicable to it.

If at any time, the System detects extraordinary instability in a symbol, then the System will activate a “stability protection mechanism” to provide an extra layer of protection to M-ELO and M-ELO users from the heightened risks of adverse selection that exists during such periods of instability.^[26] The stability protection mechanism will override the prevailing Holding Periods for M-ELOs and M-ELO+CBs in a symbol experiencing extraordinary instability and immediately increase the duration of those Holding Periods to 12 milliseconds for a period of 750 milliseconds. The System may activate the stability protection mechanism even between Change Events. The System will evaluate, at each NBBO update, whether market conditions remain extraordinarily unstable and, if so, it will restart the 750 millisecond Stability Protected Period and maintain the 12 millisecond Holding Period until conditions stabilize. Once the System determines that market conditions have stabilized ( i.e., all measurements for the symbol are at or below the threshold value throughout the duration of the prevailing Stability Protected Period), the System will revert the duration of the Holding Periods to that which prevailed as of the Change Event that ( printed page 62854) occurred immediately prior to the activation of the stability protection mechanism or, if the stability protection mechanism was active when a Change Event occurred, to the duration selected at the immediately preceding Change Event. The System will then proceed to reevaluate the duration of the Holding Periods as per the regular schedule of Change Events.

The following is an illustration of the operation of the stability protection mechanism. At 11:10:04 a.m., the prevailing Holding Period for M-ELOs in symbol XYZ is 1.5 milliseconds. At the same time, the NBBO for symbol XYZ updates. The System looks back at the prior three seconds of trading in symbol XYZ and finds that during that period, the highest observed NBBO midpoint was $10.05, and the lowest was $10.00, such that the difference between these two values is a range of $0.05. The System then looks back at trading behavior for symbol XYZ during the immediately preceding trading day. In doing so, the System calculates the value of the threshold that would have caused the symbol to be deemed extraordinarily unstable for one percent of the trading day; the System determines that this threshold value is a range of $0.03. The System then compares the $0.03 threshold to its measurement of the prior three seconds of NBBO changes ($0.05), and concludes that over these past three seconds, the symbol is extraordinarily unstable. Accordingly, the System activates the stability protection mechanism and the Holding Period for M-ELOs in symbol XYZ immediately increases to 12 milliseconds for a period of 750 milliseconds. However, 5 milliseconds after the Stability Protection Period commences, the NBBO updates again, thus prompting the System to repeat its assessment of the stability of the symbol in light of the update. This reassessment reveals that the symbol remains unstable, such that a new Stability Protection Period of 750 milliseconds begins at that time (overriding the pre-existing Period). Over the course of this new Stability Protection Period, the NBBO shifts two more times, but each of the ensuing reassessments indicate that the NBBO ranges for the symbol have fallen below the $0.03 threshold. The Stability Protection Period elapses 750 milliseconds after it began with the symbol remaining stable. Thus, the Holding Period reverts to 1.5 milliseconds.

If the Exchange halts trading in a symbol, then upon resumption of trading, any new M-ELO or M-ELO+CB in that symbol and any pending M-ELO or M-ELO+CB in that symbol with an unexpired Holding Period will be subject to a new 12 milliseconds Holding Period (running from the time when trading resumes) until the next scheduled Change Event, at which point the System may determine to adjust that Holding Period to a duration within the range applicable under normal market conditions.^[27] If, however, the System determines that extraordinary instability in the symbol exists, it will instead determine to activate the stability protection mechanism and maintain the duration of the Holding Period at 12 milliseconds for another 750 milliseconds. This design will help to ensure that M-ELOs and M-ELO+CBs receive added protection coming out of halt conditions.^[28]

The Exchange notes that same dynamic process described above will also apply to and govern the time periods during which Midpoint Orders on the Continuous Book must rest before they will become eligible to interact with M-ELO+CBs (provided that participants have opted for their Midpoint Orders to interact with M-ELO+CBs). Thus, the same Holding Period duration that the System sets for a M-ELO+CB in a symbol during Regular Market Hours will also be the length of time that a Midpoint Order must rest on the Continuous Book must rest before it may interact with a M-ELO+CB.

Apart from these impacts of Dynamic Holding Periods, M-ELOs and M-ELO+CBs will continue to behave as they do now in all respects, and as set forth in Rules 4702(b)(14) and (15).

It is important to note that within the parameters discussed herein and in the White Paper, the Exchange will continue to re-train Dynamic M-ELO and M-ELO+CB on a weekly basis (outside of market hours) so that the model will continue to learn from and act upon the basis of more recent SIP and M-ELO book data sets, and further improve its performance over time. The retraining process should not result in dramatic or unpredictable changes to the behavior of Dynamic M-ELO. The retraining process will not retrain the model from scratch each week; rather, it will retain the model's existing data inputs, knowledge base, and objectives—all without alteration. Retraining will result in new behaviors only as needed to address new scenarios that the model did not confront previously, and even then, only in a manner designed to further optimize outcomes, i.e., reduce mark-outs or increase fill rates. If the System assesses that a retrained model would be worse than the existing model in achieving its objectives, then the System will continue to use the existing model and discard the retrained model. This retraining process is a standard and accepted practice for use of deep learning models; it helps to ensure that deep learning models not only work well, but that they continue to work well in dynamic circumstances.^[29]

The Exchange will not modify the underlying structure of Dynamic M-ELO and M-ELO+CB without first obtaining the Commission's approval to do so, including modifications to the data elements the model considers in making decisions about Holding Period durations, the conditions under which the model may adjust the duration of Holding Periods, the frequency with which the model my adjust the Holding Periods, the range of Holding Period durations available to M-ELOs and M-ELO+CBs, the increments by which Holding Periods may change at any given Change Event, and the procedures for triggering, maintaining, and ending 12 millisecond Holding Periods during times of extraordinary instability.^[30] Although the Exchange will seek Commission approval prior to changing any of the data elements that the model considers, the Exchange will not seek Commission approval prior to retraining ( printed page 62855) the model to adjust the weighting it applies to those data elements.

To aid investors in understanding and evaluating Dynamic M-ELO, Nasdaq will continue to publish weekly and monthly transparency statistics on Nasdaqtrader.com, as it does now, about the performance of its M-ELOs and M-ELO+CBs, including statistics listing the weekly numbers of shares and trades in M-ELOs by symbol, weekly aggregated M-ELO share and trade data, and monthly aggregated block data.^[31] Nasdaq also will continue to disclose monthly data on Nasdaq.com, as it does now (the M-ELO Monthly Report), about M-ELO and M-ELO+CB mark-outs (quote stability by time horizon) and fill rates.^[32] Moreover, Nasdaq will add statistics to the M-ELO Monthly Report about how frequently, on average, the System changes Holding Period durations for the top decile, median, and bottom decile of symbols, as measured by monthly M-ELO and M-ELO+CB trading volumes. Nasdaq will retain copies of each historical iteration of its models as part of its books and records, and make them available to the Commission upon request, should it wish to examine them to understand how the model changes over time. Furthermore, Nasdaq will publish an equity trader alert in advance of deploying a retrained version of Dynamic M-ELO whenever Nasdaq has reason to anticipate that the retrained version will produce results that differ materially from the prior version, i.e., a projected change in mark-outs or fill-rates of 10% or more in either direction.

The Exchange acknowledges that systems necessary to implement Dynamic M-ELO, including the systems proposed that include model development and retraining processes, are “SCI Systems” within the meaning of Regulation Systems Compliance and Integrity (“Reg. SCI”),^[33] and that the Exchange, as an SCI Entity, remains responsible for compliance with all requirements of Reg. SCI, including, without limitation, to have policies and procedures reasonably designed to ensure that its SCI Systems operate in a manner that complies with the Act and the rules and regulations thereunder and Exchange's rules and governing documents, among them a plan for assessments of the functionality of SCI Systems designed to detect systems compliance issues, including by responsible SCI personnel and by personnel familiar with applicable provisions of the Act and the rules and regulations thereunder and Exchange's rules and governing documents.

Implementation

The Exchange intends to make the proposed change effective for M-ELOs and M-ELO+CBs in the Second or Third Quarter of 2023, but that time frame is subject to change. The Exchange will publish a Trader Alert in advance of making the proposed change effective.

2. Statutory Basis

The Exchange believes that its proposal is consistent with Section 6(b) of the Act,^[34] in general, and furthers the objectives of Section 6(b)(5) of the Act,^[35] in particular, in that it is designed to promote just and equitable principles of trade, to remove impediments to and perfect the mechanism of a free and open market and a national market system, and, in general to protect investors and the public interest, by allowing for more widespread use of M-ELOs and M-ELO+CBs.

When the Commission approved the M-ELO and the M-ELO+CB, it determined that these Order Types are consistent with the Act because they “could create additional and more efficient trading opportunities on the Exchange for investors with longer investment time horizons, including institutional investors, and could provide these investors with an ability to limit the information leakage and the market impact that could result from their orders.” ^[36] Nothing about the Exchange's proposal should cause the Commission to revisit or rethink this determination. Indeed, the proposal will not alter the fundamental design of these Order Types, the manner in which they operate, or their effects.

Even with Dynamic M-ELO Holding Periods, M-ELOs and M-ELO+CBs will continue to provide their users with protection against information leakage and adverse selection—and they will do so at levels which are substantially undiminished from that which they provide now.^[37]

At the same time, however, the proposal will benefit market participants and investors by reducing the opportunity costs of utilizing M-ELOs and M-ELO+CBs. The proposal, in other words, will re-calibrate the lengths of the Holding Periods so that M-ELOs and M-ELO+CBs will operate in the “Goldilocks” zone—their Holding Periods will not be so short as to render them unable to provide meaningful protections against information leakage and adverse selection, but the Holding Periods also will not be too long so as to cause participants and investors to miss out on favorable execution opportunities. Nasdaq believes the proposal will render M-ELOs and M-ELO+CBs more useful and attractive to market participants and investors, and this increased utility and attractiveness, in turn, will spur an increase in M-ELO and M-ELO+CB use cases on the Exchange, both from new and existing users of M-ELOs and M-ELO+CBs. Ultimately, the proposal should enhance market quality by increasing opportunities for midpoint executions on the Exchange.

As Nasdaq explained above, the Proposal will operate within strict, well-defined, and transparent parameters. Although it will undergo weekly retraining (outside of market hours),^[38] such retraining will aim to improve the performance of the model in achieving its twin objectives; retraining will not alter the inputs, objectives, or basic design parameters of Dynamic M-ELO without prior Commission approval.^[39] Moreover, the Exchange will not deploy a retrained model if it fails to achieve performance improvements. To aid ( printed page 62856) investors in evaluating Dynamic M-ELO, the Exchange will publish statistics about its performance, including as to mark-outs and fill rates, as well as statistics about how frequently the System changes Holding Period durations. To further facilitate accountability, the Exchange will retain each historical iteration of its model as part of its books and records, and make such information available to the Commission, upon request. The Exchange will also publish equity trader alerts whenever retraining will result in a performance change of 10% or more.

Nasdaq notes that the twin objectives it prescribes for the model involve the absolute values of mark-outs and fill rates; they are not designed to further the performance of any participant or any category of participant. Furthermore, Nasdaq performed internal tests of its AI model to detect indications of harmful bias in its performance results, and such tests concluded that no such indications exist. That is, the Exchange reviewed the impact on fill rates and mark-outs of Dynamic M-ELO, as compared to the “static” M-ELO, for those firms that accounted for more than 95% of M-ELO activity on the Exchange during Q1 2022.^[40] The Exchange analyzed results both in an absolute and a relative sense. Testing revealed that all participants experienced at least some improvements in fill rates and mark-outs when using Dynamic M-ELO versus static M-ELO, with the volume-weighted average improvement being aligned with the results expressed in the White Paper. We detected no material variations that might suggest that a particular participant or category of participant ( i.e., nature of firm; size of firm) benefitted from Dynamic M-ELO functionality to an extent that was unreasonably disproportionate to the benefits that other participants experienced. Thus, Nasdaq believes the model is objective, is designed to, and does avoid bias and discrimination.^[41]

The Exchange notes that use of Dynamic M-ELOs and M-ELO+CBs remains voluntary for all market participants. Accordingly, if any market participant feels that the dynamic Holding Periods are still too long or too short or because competing venues offer more attractive delay mechanisms, then the participants are free to pursue other trading strategies or utilize other trading venues. They need not utilize Dynamic M-ELOs or M-ELO+CBs.

Furthermore, the design of Dynamic-MELO would constitute an “established, non-discretionary” method that is consistent with the definition of an exchange, as set forth in SEC Rule 3b-16.^[42] The Commission stated as follows when it adopted Rule 3b-16:

Nothing in the Reg. ATS Adopting Release or in any of its illustrative examples suggests that Dynamic M-ELO would constitute an exercise of discretionary behavior. Dynamic M-ELO will handle and execute Orders according to published, pre-determined rules that are disclosed to the public and which provide reasonable notice of how the Order Type will behave.^[44] To the extent that the design of the System permits variation in the Holding Periods for such Orders, it does so by design. The range of potential variations, the objectives that such variations are intended to achieve, and the factors that determine when such variations may occur are also predetermined and set forth in the Exchange's Rules or otherwise disclosed to the public. The mere fact that the System may apply different weights over time to the factors it uses to determine whether and by how much to vary a Holding Period does not mean that the System will act with discretion in the same sense that a human being could be said to be exercise independent judgment when deciding whether and how to handle an order.^[45] Even when the System makes decisions about changing the Holding Periods, the System will operate pursuant to a mathematical algorithm from which it cannot deviate—an algorithm that is programmed to achieve pre-defined and pre-disclosed objectives.^[46]

The Exchange notes that it will continue to conduct real-time surveillance to monitor the use of M-ELOs and M-ELO+CBs to ensure that such usage remains appropriately tied to the intent of the Order Types. If, as a result of such surveillance, the Exchange determines that the Dynamic M-ELO Holding Periods do not serve their intended purposes, or adversely impact market quality, then the Exchange will seek to make further re-calibrations.

Nasdaq does not believe that the design of Dynamic M-ELO lends itself to potential manipulation by a single participant or a small group of participants because the System makes determinations regarding Holding Periods based upon prevailing market-wide conditions for a given symbol, rather than the behaviors of particular participants with respect to that symbol, or the activity of participants in M-ELOs involving that symbol. Manipulation of the System also would be difficult to accomplish given the large number of variables that factor into the System's decisions to change Holding Periods during Change Events, as well as the different weights that apply to each such factor, which as described above, the System may vary over time. Any benefits that a participant might derive from manipulating the duration of Holding Periods would likely be small and outweighed significantly by the difficulty and cost of affecting such manipulation. Nevertheless, the Exchange will surveil for indications of manipulation and act accordingly if it detects such indications.

B. Self-Regulatory Organization's Statement on Burden on Competition

The Exchange does not believe that the proposed rule change will impose any burden on competition not necessary or appropriate in furtherance of the purposes of the Act. To the contrary, the Exchange believes that this proposal will promote the competitiveness of the Exchange by rendering its M-ELO and M-ELO+CB Order Types more attractive to participants.

The Exchange adopted the M-ELO and M-ELO+CB as pro-competitive measures intended to increase participation on the Exchange by allowing certain market participants that may currently be underserved on regulated exchanges to compete based on elements other than speed. The proposed change continues to achieve this purpose. With Dynamic M-ELO Holding Periods, both M-ELOs and M-ELO+CBs will afford their users with a level of protection from information leakage and adverse selection that is better from what is achievable at present.^[47] At the same time, the Dynamic Holding Periods will increase opportunities to interact with other like-minded investors with longer time horizons while also lowering the opportunity costs for participants that utilize M-ELOs and M-ELO+CBs, particularly for securities that trade within the “Goldilocks” zone. In sum, the proposed changes will not burden competition, but instead may promote competition for liquidity in M-ELOs and M-ELO+CBs by broadening the circumstances in which market participants may find such Orders to be useful. With the proposed changes, market participants will be more likely to determine that the benefits of entering M-ELOs and M-ELO+CBs outweigh the risks of doing so.

The proposed change will not place a burden on competition among market venues, as any market may adopt an order type that operates similarly to a M-ELO or a M-ELO+CB with Dynamic M-ELO Holding Periods.

C. Self-Regulatory Organization's Statement on Comments on the Proposed Rule Change Received From Members, Participants, or Others

No written comments were either solicited or received.

IV. Discussion and Commission Findings

After careful review, the Commission finds that the Exchange's proposal to adopt Dynamic M-ELO is consistent with the requirements of the Act and the rules and regulations thereunder applicable to a national securities exchange.^[48] In particular, the Commission finds that the proposed rule change, as modified by Amendment No. 2, is consistent with Sections 6(b)(5) and 6(b)(8) of the Act.^[49] Section 6(b)(5) of the Act requires that the rules of a national securities exchange be designed, among other things, to prevent fraudulent and manipulative acts and practices, to promote just and equitable principles of trade, and, in general, to protect investors and the public interest, and not be designed to permit unfair discrimination between customers, issuers, brokers, or dealers.^[50] Section 6(b)(8) of the Act requires that the rules of a national securities exchange not impose any burden on competition that is not necessary or appropriate in furtherance of the purposes of the Act.^[51]

Nasdaq's Obligation To Sufficiently Explain Its Proposed Rule Change

The burden to demonstrate that a proposed rule change is consistent with the Act and rules and regulations thereunder is on the self-regulatory organization (“SRO”) proposing a rule change.^[52] Each proposed SRO rule change must be “accompanied by a concise general statement of the basis and purpose of such proposed rule change.” ^[53] As described in more detail below, several commenters argued that the proposal did not provide sufficient information with respect to the operation of Dynamic M-ELO, or that the information provided was not “clear and comprehensible,” as required by Form 19b-4. For the reasons articulated below, the Commission believes that Nasdaq has provided clear and comprehensible information on the overall operation of Dynamic M-ELO and the role of the machine-learning model and demonstrated that the proposal is consistent with the Act. Several related comments addressed this issue; these comments and Nasdaq's responses are discussed below, followed by the Commission's analysis.

One commenter stated that the initial filing would establish “a dangerously vague standard for describing how exchange-hosted complex algorithmic ( printed page 62858) order types operate.” ^[54] In response to public comment, Nasdaq added more details describing the operation of Dynamic M-ELO to both the filing and public record since this proposed rule change was initially submitted to the Commission, and Nasdaq also provided additional legal analysis to support Dynamic M-ELO's consistency with the Act. Prior to the filing of Amendment No. 1, a commenter stated that although “Nasdaq shared some of the 142 features of their formula,” Nasdaq should reveal all of these features so that prospective users may evaluate how the model works.^[55] Similarly, another commenter stated that the public cannot provide meaningful comment on the proposal without knowing all categories and parameters of the proposed Dynamic M-ELO.^[56] In its response to these comments, the Exchange, among other things, provided the specific 142 data elements that will be weighed by the machine-learning model as both an appendix to its first letter in response to comments,^[57] and as Exhibit 3B to its Amendment No. 1 filing.

In response to the Exchange's disclosures in Exhibit 3B of Amendment No. 1, one of these commenters stated that the list of data elements was not “clear and comprehensible” as is required by the Form 19-4, but rather “vague, confusing, and perfunctory.” ^[58] This commenter also stated that the disclosed data elements included unexplained terms ( e.g., “baseline simulated,” “action simulated,” and “synthetic mark-out”).^[59] In a subsequent comment letter, this commenter reiterated these points; the commenter specified that the commenter's concern is that Nasdaq's rule text does not disclose information about its methods for assessing market conditions and that “Nasdaq should carefully detail its methods in its rulebook, just like other exchanges have done, and Nasdaq should also thoroughly disclose its methods in its filing text.” ^[60] In its second response to comments ^[61] and the revised Exhibit 3b to Amendment No. 2, Nasdaq expanded and “simplified” the explanation of these 142 data elements. The Exchange also added to the proposed rule text a definition of the term “proprietary assessment of market conditions” to explain how the machine-learning model will evaluate those 142 data elements.^[62]

Furthermore, Nasdaq attached, as Exhibit 3A to its proposed rule change, the White Paper written by its AI Core Development Team that explains, among other things, how Dynamic M-ELO's machine-learning functions were developed and tested. The White Paper includes a general discussion of the type of model implemented in the proposed system, in this case a reinforcement learning model,^[63] as well as citations to academic research behind the Double Deep Q-Network algorithm that is the basis for the algorithm used in Nasdaq's model.^[64] The White Paper also describes the ways in which Nasdaq's implementation of the proposed model differs from the model and training in the academic research, providing both an English summary and a pseudocode description of differences in model training implemented by Nasdaq.^[65]

One of the commenters stated that the White Paper is not easily understood by most market participants and that referencing the White Paper in the filing is an “unacceptable substitute” for a “plain English” explanation of the proposal in Form 19b-4.^[66] In response, Nasdaq explained that it drafted the filing to provide a general understanding of Dynamic M-ELO and how it will behave, and the more detailed information and explanation in the White Paper are meant to support the filing.^[67]

The Commission agrees with comments and the Exchange that there is an extent to which the proposed changes will introduce an unavoidable degree of uncertainty with respect to the use of these order types. The deep reinforcement learning model that will determine the dynamic holding periods for each symbol for M-ELO and M-ELO+CB orders will be implemented through established, non-discretionary methods,^[68] but it is so complex that its complete details are, for most intents and purposes, not readily intelligible, and it would be immensely difficult for the Exchange or any market participant to precisely predict the holding periods that will be generated by the model for any given symbol at any particular time. Nevertheless, as further discussed below, the Commission believes that the Exchange has provided information sufficient for the Commission and public to understand the design, operation, and limits of the proposed changes to these order types, and the role of the machine-learning model therein.

While the holding periods under the proposal would be dynamic, Nasdaq has precisely articulated both the nature of changes that would be permissible under the proposal, and the limits to those changes. Nasdaq described when changes might occur (every thirty seconds throughout the trading day), the initial default holding period for all symbols (1.25 milliseconds), the permissible increments by which a holding period might change in each symbol (0.25 or 0.50 milliseconds), and the outer bounds of permissible holding period lengths (0.25 milliseconds at the short end, and 2.50 milliseconds at the long end). Nasdaq also described the conditions of “extraordinary instability” in a symbol when these holding periods would not apply, and when the holding periods would be overridden by the proposed “stability protection mechanism” (with a holding period of 12 milliseconds for at least 750 milliseconds). The Commission believes that these details provide sufficient information to understand the range of potential holding periods that may be applied when M-ELO or M-ELO+CB orders are entered or resting on the order book, the changes that may occur, and the limits to those changes. ( printed page 62859)

Nasdaq has also described the role of the machine-learning model in its proposal. The model will determine whether, by which increment, and in which direction to adjust the holding period for each symbol throughout the trading day.^[69] In its Form 19b-4, White Paper, and response letters, Nasdaq described the goals towards which the model is optimized: reducing mark-outs and increasing fill rates.^[70] Nasdaq's White Paper includes a detailed discussion of model choice, development, and training, including citations to relevant other research.^[71] Nasdaq also provided several iterations of a list of data elements that the model will ingest and use, including a glossary defining terms used in the descriptions.^[72] Nasdaq affirmed that in operation during market hours, the data used would be calculated based on intraday market data.^[73] One version of these lists included Nasdaq's estimates of the tendencies of data elements to affect model outcomes.^[74] Nasdaq's White Paper also included an “explainability study” that assessed both the effects of individual data elements on model performance and the effects of interactions between individual data elements.^[75] Across its filing and incorporated exhibits, aspects of the model's operations and design are described in different formats and with different levels of specificity—for example, the filing and exhibits include “plain English” descriptions, mathematical definitions, and pseudocode. Together, this set of information allows the Commission to understand the type of decision the model will implement, the goals the model aims to achieve, which model type is implemented and how it was developed, the range of data types and data sources used by the model, and estimates of the manner in which those data may affect model outcomes.

Nasdaq also explains how and when the machine-learning model will be retrained. Nasdaq will retrain the model weekly, outside of market hours. Retraining will incorporate market data obtained during the week from the equity consolidated data feeds and M-ELO order book. A retrained model will only be promoted to production if it improves upon the model objectives compared to the prevailing model.^[76] Furthermore, the Exchange explained that the machine-learning model is consistent in its behavior from day-to-day during periods when it is not undergoing retraining, “such that its decisions when presented with given set of facts and circumstances in a given security on day 1 should be the same as they would be on day 2.” ^[77] The Exchange also stated in its initial response to comments that “[e]ven after the system undergoes retraining, which will occur on a weekly basis (and not during market hours), system behavior should not change dramatically or in unexpected ways from week-to-week.” ^[78] As noted above as well, the Exchange also represents that outside of set retraining periods, “the System will operate pursuant to a mathematical algorithm from which it cannot deviate—an algorithm that is programmed to achieve pre-defined and pre-disclosed objectives.” ^[79] Nasdaq also will publish equity trader alerts when it anticipates that a model update may change mark-outs or fill rate by 10% or more in either direction.^[80] By including this set of information, Nasdaq has provided the Commission and public with information that allows them to understand how frequently the model will be retrained, the data used for retraining, and the criteria that will be used to determine whether to update the production model based on retraining. This information allows the Commission to understand when the proposed model may change and when it will remain constant, the circumstances under which a change would be implemented, and circumstances under which the public will receive notice of significant changes in the model's anticipated outcomes.

In addition, a commenter stated their belief that the Exchange's proposal for Dynamic M-ELO would result in the exercise of discretion by a national securities exchange because the machine-learning model's decisions would vary over time based on the following: (1) varying parameter values; and (2) results of retraining cycles.^[81] The commenter stated that by making it possible for Dynamic M-ELO to behave differently when confronted by the same market conditions before and after the model is retrained, Nasdaq's model would be exercising discretion that is more akin to a broker than an exchange.^[82]

The commenter claimed that Dynamic M-ELO would operate outside of established non-discretionary methods, which require “fully disclosed procedures operating in a strictly linear, invariant, and deterministic fashion.” ^[83] Additionally, the commenter stated that Nasdaq would be exercising discretion with Dynamic M-ELO to alter a participant's material order terms.^[84] The commenter claimed that Nasdaq would be using undisclosed data such as the ( printed page 62860) buyer and seller counts and recent trade sizes.^[85] Further, the commenter stated:

In Amendment No. 1, Nasdaq added language to address these concerns.^[86] Among other things, Nasdaq stated that to the extent that the design of Dynamic M-ELO permits variation in the Holding Periods for such orders, it does so by design, and the “mere fact that the System may apply different weights over time to the factors it uses to determine whether and by how much to vary a Holding Period does not mean that the System will act with discretion in the same sense that a human being could be said to be exercise independent judgment when deciding whether and how to handle an order.” ^[87] Additionally, Nasdaq stated the following in its second response to comments:

Based on Nasdaq's representations described above, Dynamic M-ELO would operate pursuant to pre-determined, programmed procedures that would dictate order interaction and the terms for trading for each Dynamic M-ELO order entered on the Nasdaq trading facility. While the Exchange's procedures include conditions that, if satisfied under certain circumstances, might result in different outcomes for different M-ELO orders, such conditions and circumstances, if pre-determined, pre-defined, and programmed into the Exchange's trading facility, would be considered established and not discretionary. For example, according to the Exchange, Dynamic M-ELO may apply different pre-determined weights over time to pre-determined factors it uses to determine whether and by how much to vary a Holding Period.^[91] In such an event, Dynamic M-ELO will operate pursuant to pre-determined procedures and programmed mathematical algorithm from which it cannot deviate to “achieve pre-defined and pre-disclosed objectives.” ^[92] Further, the procedures governing Dynamic M-ELO and use of M-ELO orders will be established before the beginning of each trading day. For example, Dynamic M-ELO will use preset methods to evaluate and weigh specific data elements to determine the dynamic holding periods. Such pre-set methods will be established during the prior retraining period, and outside regular trading hours, and will not vary intra-day until adjusted at the next retraining period.

Given the pre-determined, programmed procedures and rules that Nasdaq has proposed to dictate trading for Dynamic M-ELO, the Commission does not believe that Dynamic M-ELO is designed to provide Nasdaq with judgement and flexibility, and therefore, discretion over the handling or execution of a M-ELO order entered on the Exchange.

Unfair Discrimination

Section 6(b)(5) of the Act requires that the rules of a national securities exchange not be designed to permit unfair discrimination between customers, issuers, brokers, or dealers. In several letters, one commenter stated that Nasdaq inadequately explains how it will monitor and, if necessary, adjust Dynamic M-ELO to ensure no unfair discrimination.^[93] Initially, this commenter emphasized what they perceived to be silence on the part of Nasdaq with regard to whether Dynamic M-ELO will discriminate among categories of participant types.^[94] In its response to these comments, Nasdaq initially added a new representation to the filing in Amendment No. 1, stating that that Dynamic M-ELO is not designed to further the performance of any participant or any category of participant, but instead has twin objectives—the absolute values of mark-outs and fill rates. In Amendment No. 2, Nasdaq expanded on this representation by adding the following:

In Amendment No. 2, Nasdaq also affirmed that it will periodically review its model to ensure that it continues to perform in accordance with the Exchange's rules and that it has not introduced any harmful bias in favor of or against any participant or class of participants.^[95]

In response to the above, the commenter submitted a fourth comment letter, in which they questioned the approach Nasdaq took to demonstrate that there is not bias against any one ( printed page 62861) participant or class of participants.^[96] The commenter, among other things, expressed concern about Nasdaq conducting its analysis using data for firms that accounted for 95% of M-ELO activity during Q1 of 2022 rather than all M-ELO activity. The commenter states that Nasdaq did not describe how it determined the 5% of activity during that period to exclude from its analysis.^[97] For example, the commenter states that it is not clear whether Nasdaq excluded firms with large orders and trades, and the commenter opines that discarding any data could exclude activity that has qualitative or quantitative differences from the rest.^[98]

In response to this comment, Nasdaq represented that it conducted a supplemental analysis of the initially-excluded data—which were the activity of the least-active M-ELO firms from the control period of its initial analysis—to confirm whether its initial conclusions held for those participants.^[99] Nasdaq explains that the individual variations among the previously excluded participants was higher than that for the original batch of data, but that, based on simulated data, each of these participants would have experienced the same or better fill rates during the testing period if they had utilized Dynamic M-ELO.^[100] Based on this supplemental data analysis, Nasdaq concluded that there is no apparent biases for the Dynamic M-ELO, even among the least active M-ELO participants.^[101]

The Commission concludes that Nasdaq has adequately demonstrated that the proposal is not designed to permit unfair discrimination consistent with Section 6(b)(5) of the Act. Through the White Paper, amendments, and response letters, Nasdaq has demonstrated that it has analyzed the anticipated or simulated effects of the proposed change on all current M-ELO users, and that this work did not indicate that particular firms or classes of firms are anticipated to unfairly benefit from or be harmed by the proposed Dynamic M-ELO functionality.

Prevention of Fraudulent and Manipulative Acts and Practices, Just and Equitable Principles of Trade, and the Protection of Investors and the Public Interest

Section 6(b)(5) of the Act also requires that the rules of a national securities exchange be designed to prevent fraudulent and manipulative acts and practices, to promote just and equitable principles of trade, and, in general, to protect investors and the public interest. In Amendment No. 2, Nasdaq addressed whether the Dynamic M-ELO is designed to prevent fraudulent and manipulative acts and practices. Nasdaq states that the design of Dynamic M-ELO does not lend itself to potential manipulation by a single participant or a small group of participants because the machine-learning model makes determinations regarding Holding Periods based upon prevailing market-wide conditions for a given symbol, rather than the behaviors of particular participants with respect to that symbol, or the activity of participants in M-ELOs involving that symbol. Nasdaq further states that manipulation of the machine-learning model would be difficult to accomplish given the large number of variables that factor into the machine-learning model's decisions to change Holding Periods during Change Events, as well as the different weights that apply to each such factor, which as described above, may vary over time. Furthermore, Nasdaq states that any benefits that a participant might derive from manipulating the duration of Holding Periods would likely be small and outweighed significantly by the difficulty and cost of effecting such manipulation.

The Exchange, in Amendment No. 2, also sets forth representations regarding how it will surveil its market after Dynamic M-ELO is implemented. First, Nasdaq represents that it will review the machine-learning functionality and operation periodically to affirm that it continues to perform in accordance with the Exchange's rules and has not introduced any harmful bias in favor of or against any participant or category of participants.^[102] Nasdaq also represents above that it will surveil for indications of manipulation and act accordingly if it detects such indications.

The Commission finds that the proposed rule change is designed to prevent fraudulent and manipulative acts and practices consistent with Section 6(b)(5) of the Act. The Commission agrees that aspects of the Dynamic M-ELO design reduce opportunities for manipulation or are likely to make manipulation costly or difficult. The model's operation depends on 142 data elements, which are each likely to have effects on model outcomes of differing magnitudes and in different directions. Many of these data elements are also based on market-wide data, in some cases spanning periods of days,^[103] which are likely themselves difficult for market participants to manipulate. Given these design features, it appears likely that manipulating the duration of Dynamic M-ELO holding periods in any given symbol or group of symbols would be an extremely complex undertaking. In light of this complexity, and the size of M-ELO activity relative to the market for NMS stocks,^[104] Nasdaq's assertion that the potential benefits of manipulating the dynamic holding periods for these order types would be outweighed by the cost and complexity of manipulation also appears reasonable. Nasdaq has also represented that it intends to surveil the proposed order types for manipulation. This ongoing surveillance, to ensure the appropriate use of Dynamic M-ELO by Exchange Members and behavior by the machine-learning model, is important to the successful implementation of Dynamic M-ELO and appears appropriately tailored to the accomplish the intent of the M-ELO and M-ELO+CB order types.

Furthermore, the Commission finds that overall structure of Dynamic M-ELO—particularly, the static numerical constraints set forth in the proposed rule text—is designed in general, to protect investors and the public interest and promote just and equitable principles of trade pursuant to Section 6(b)(5) of the Act. As described above, the model will continuously engage in dynamic analysis of current market conditions during trading hours, and outside of market hours, it will retrain with the goal of improving the overall performance of Dynamic M-ELO. These dynamic aspects of the proposal, ( printed page 62862) however, are constrained by the static numerical thresholds set forth in the proposed rule text. For example, the initial Holding Periods for each trading day will be 1.25 milliseconds, the overall range for any Holding Period must be between 0.25 and 2.50 milliseconds during normal market conditions, and the Holding Period can only change by either 0.25 or 0.50 milliseconds at each Change Event during normal market conditions. Regardless of how the model analyzes the current market or changes the weighting of the data elements as a result of its retraining, Dynamic M-ELO cannot operate outside of the static numerical ranges and limitations or minimums set forth in the rule text. As such, the Commission finds that Nasdaq has designed Dynamic M-ELO to operate in a manner that in general protects investors and the public interest and promotes just and equitable principles of trade in accordance with Section 6(b)(5) of the Act.

Compliance With SRO Recordkeeping and Reporting Obligations

One commenter queried whether Nasdaq could maintain an adequate audit trail given the potential for frequently shifting Holding Periods for Dynamic M-ELO.^[105] In response, Nasdaq states that it will retain copies of each iteration of its system as part of its books and records and will disclose publicly statistics relating to Dynamic M-ELO performance.^[106] Nasdaq additionally represented that it will publish weekly and monthly Dynamic M-ELO performance statistics, which would include the weekly numbers of shares and trades in M-ELOs by symbol, weekly aggregated M-ELO share and trade data, and monthly aggregated block data, on Nasdaqtrader.com.^[107] Nasdaq also indicated it would add statistics to its existing M-ELO Monthly Report, which discloses quote stability by time horizon, about how frequently, on average, its system changes Holding Period durations for the top decile, median, and bottom decile of symbols, as measured by monthly M-ELO and M-ELO+CB trading volumes.^[108]

Nasdaq also added a representation to the filing, addressing how it would comply with its recordkeeping obligations.^[109] Nasdaq states that it will retain copies of each historical iteration of its models as part of its books and records, and make them available to the Commission upon request, should it wish to examine them to understand how the model changes over time.^[110] Nasdaq also states that it will publish an equity trader alert in advance of deploying a retrained version of Dynamic M-ELO when Nasdaq anticipates the retrained version will produce results that differ materially from the prior version.^[111] Based on these representations, the Commission finds that Nasdaq has met its burden to demonstrate that it will comply with all relevant exchange recordkeeping requirements and obligations when it implements Dynamic M-ELO. In addition, the Commission notes that Nasdaq must comply with its reporting obligations under Rule 613 of Regulation NMS ^[112] and the National Market System Plan Governing the Consolidated Audited Trail (“CAT NMS Plan”) ^[113] with respect to Dynamic M-ELO, which requires it to record and electronically report to the central repository the material terms of each order and each reportable event.^[114]

Nasdaq's Obligation To File Proposed Rule Changes Relating to Dynamic M-ELO

Prior to the filing of Amendment No. 1, a commenter stated that it was unclear what types of changes to the model would lead Nasdaq to seek approval from the Commission via an SRO rule filing.^[115] As explained above,^[116] Nasdaq represents that it will not modify the underlying structure of Dynamic M-ELO without first obtaining the Commission's approval to do so, including modifications to the data elements the model considers in making decisions about Holding Period durations, the conditions under which the model may adjust the duration of Holding Periods, the frequency with which the model may adjust the Holding Periods, the range of Holding Period durations available to M-ELOs and M-ELO+CBs, the increments by which Holding Periods may change at any given Change Event, and the procedures for triggering, maintaining, and ending 12 millisecond Holding Periods during times of extraordinary instability. In contrast, the Exchange states that it will not seek Commission approval prior to retraining the model to adjust the weighting it applies to those data elements pursuant to the weekly retraining process.

Section 19(b)(1) of the Act ^[117] and Rule 19b-4 thereunder ^[118] require an SRO to file a proposed rule change with the Commission whenever it seeks any proposed change in, addition to, or deletion from the rules governing the SRO and its members' activities on the SRO. As discussed above, the proposal sets forth the specific data elements that Dynamic M-ELO will use during the trading day. Furthermore, the proposed rule change sets forth when the machine-learning model will retrain and the extent to which the retraining can and cannot cause the machine-learning model to update Dynamic M-ELO's operation during subsequent trading days.^[119] In addition, the proposal sets forth the operation of Dynamic M-ELO, such as the potential range for a Holding Period, how often Dynamic M-ELO reevaluates market conditions for a given security to adjust a Holding Period, and the increment by which a Holding Period may be changed. Nasdaq represents that it will not change any of these aspects of the proposal or any other function of Dynamic M-ELO without first filing a proposed rule change.^[120] Nasdaq does, however, state that it would not file a proposed rule change in connection with the operation of the machine-learning model's weekly retraining and the results of that process.

Based on the foregoing, the Commission believes that Nasdaq has adequately responded to the commenter's concern. Nasdaq will need to file a proposed rule to make any changes, additions, or deletions to the operation of Dynamic M-ELO as approved herein. Nasdaq has delineated when it would file a proposed rule change to alter the operation of Dynamic M-ELO, and when the machine-learning model would retrain and adjust the weighting it applies to the data elements without it filing a proposed rule change. Specifically, Nasdaq's proposed rule change and rule text reflect the 142 data elements Dynamic M-ELO will consider when determining the Holding Period for a security and the goals Nasdaq will consider when weighing those data elements ( i.e., reducing mark-outs and increasing fill rates) but does not set forth the relative weighting of each those individual data elements. Though ( printed page 62863) the structure of the proposal does not disclose of the exact weighting for each of the 142 data elements, it does set forth the two goals Nasdaq will consider when weighing those data elements initially and during each weekly retraining, which provides information as to how those 142 factors will be used in determining the Holding Period for a security. Based on how the proposed rule sets forth the goals that will govern each retraining, the Commission believes that Nasdaq's delineation of when it would and would not file a proposed rule change to alter the operation of Dynamic M-ELO is consistent with Nasdaq's rule filing obligation. The Commission agrees that the weekly retraining to optimize the weighting of the 142 data elements considered by Dynamic M-ELO to best achieve those goals within the rule's parameters would not necessitate the filing of a proposed rule change with the Commission because those adjustments would be reasonably and fairly implied by the proposed rule. However, to the extent Nasdaq seeks to change, add to, or delete from the rule's construct in connection with the weekly retraining, it would first be required to file a proposed rule change with the Commission.

V. Solicitation of Comments on Amendment No. 2 to the Proposed Rule Change

Interested persons are invited to submit written data, views, and arguments concerning whether Amendment No. 2 is consistent with the Act. Comments may be submitted by any of the following methods:

Electronic Comments

• Use the Commission's internet comment form (https://www.sec.gov/​rules/​sro.shtml); or
• Send an email torule-comments@sec.gov. Please include file number SR-NASDAQ-2022-079 on the subject line.

Paper Comments

• Send paper comments in triplicate to Secretary, Securities and Exchange Commission, 100 F Street NE, Washington, DC 20549-1090.

VI. Accelerated Approval of Proposed Rule Change, as Modified by Amendment No. 2

The Commission finds good cause to approve the proposed rule change prior to the 30th day after the date of publication of Amendment No. 2 in the Federal Register . Amendment No. 2 does not include any material changes to the operation of the proposed Dynamic M-ELO and its machine-learning model. In Amendment No. 2, the Exchange: (1) adds the defined term “proprietary assessment of market conditions” to the proposed rule text, which consolidates certain details and explanations about how the machine-learning model would operate from prior versions into a single defined term; (2) revises the list of factors provided in Exhibit 3b to include expanded and “simplified” explanations of the terminology used therein; (3) adds a representation that the systems used to operate Dynamic M-ELO and machine-learning model are “SCI Systems” and thus subject to compliance with Regulation SCI; and (4) expands the legal analysis to address comments regarding unfair discrimination and the exercise of impermissible discretion by the Exchange.

The Commission finds that Amendment No. 2 raises no novel regulatory issues that have not previously been subject to comment and is reasonably designed to prevent fraudulent and manipulative acts and practices, to promote just and equitable principles of trade, and, in general, to protect investors and the public interest, and not be unfairly discriminatory, or impose an unnecessary or inappropriate burden on competition. Amendment No. 2 does not alter the proposed operation or any material features of Dynamic M-ELO, which operation and features have been subject to two rounds of public comment. In response to public comment, the revisions to the proposal contained within Amendment No. 2 provide additional clarification and details regarding how Dynamic M-ELO and the machine-learning model will operate, as well as additional legal analysis to support the Exchange's position that the proposal is consistent with the Act. Accordingly, pursuant to Section 19(b)(2) of the Act,^[121] the Commission finds good cause to approve the proposed rule change on an accelerated basis prior to the 30th day after publication of notice of the filing of Amendment No. 2 in the Federal Register .

VII. Conclusion

It is therefore ordered, pursuant to Section 19(b)(2) of the Act,^[122] that the proposed rule change (SR-NASDAQ-2022-079), as modified by Amendment No. 2, be, and it hereby is, approved on an accelerated basis.

A system uses established non-discretionary methods either by providing a trading facility or by setting rules governing trading among subscribers. The Commission intends for “established, non-discretionary methods” to include any methods that dictate the terms of trading among the multiple buyers and sellers entering orders into the system. Such methods include those that set procedures or priorities under which open terms of a trade may be determined. For example, traditional exchanges' rules of priority, parity, and precedence are “established non-discretionary methods,” as are the trading algorithms of electronic systems. Similarly, systems that determine the trading price at some designated future date on the basis of pre-established criteria (such as the weighted average trading price for the security on the specified date in a specified market or markets) are using established, non-discretionary methods.^[43]

that (a) by determining the universe of data the system consumes, (b) by programming how the system thinks, (c) by controlling and supplying the information with which it thinks, and (d) by setting the goals and programming the nature and extent of its actions, and when it does all this to determine (e) when and in which prescribed intervals to set an ever variable time-in-force term for an order, a term which (f) dictates when to expose an order to the market to find contra-side interest, then without question Nasdaq is exercising control, judgment, and discretion over its customer orders.

It is also worth noting that presently, exchanges like Nasdaq already employ non-linear, non-deterministic functionalities, like the randomized timers it uses to resolve certain unavoidable race conditions that arise in the order handling process. Nasdaq employs these functionalities with the knowledge of the SEC, and without any suggestion that they somehow transform Nasdaq into a broker.^[88]

Furthermore, as noted above, the Exchange represents that outside of set retraining periods, “the System will operate pursuant to a mathematical algorithm from which it cannot deviate—an algorithm that is programmed to achieve pre-defined and pre-disclosed objectives.” ^[89] The Exchange explains that outside of the set retraining periods “the System will behave predictably from day to day, such that its decisions when presented with given set of facts and circumstances in a given security on day 1 should be the same as they would be on day 2.” ^[90]

Furthermore, Nasdaq performed internal tests of its AI model to detect indications of harmful bias in its performance results, and such tests concluded that no such indications exist. That is, the Exchange reviewed the impact on fill rates and mark-outs of Dynamic M-ELO, as compared to the “static” M-ELO, for those firms that accounted for more than 95% of M-ELO activity on the Exchange during Q1 2022 . . . . The Exchange analyzed results both in an absolute and a relative sense. Testing revealed that all participants experienced at least some improvements in fill rates and mark-outs when using Dynamic M-ELO versus static M-ELO, with the volume-weighted average improvement being aligned with the results expressed in the White Paper. We detected no material variations that might suggest that a particular participant or category of participant ( i.e., nature of firm; size of firm) benefitted from Dynamic M-ELO functionality to an extent that was unreasonably disproportionate to the benefits that other participants experienced. Thus, Nasdaq believes the model is objective, is designed to, and does avoid bias and discrimination.

All submissions should refer to file number SR-NASDAQ-2022-079. This file number should be included on the subject line if email is used. To help the Commission process and review your comments more efficiently, please use only one method. The Commission will post all comments on the Commission's internet website ( https://www.sec.gov/​rules/​sro.shtml). Copies of the submission, all subsequent amendments, all written statements with respect to the proposed rule change that are filed with the Commission, and all written communications relating to the proposed rule change between the Commission and any person, other than those that may be withheld from the public in accordance with the provisions of 5 U.S.C. 552, will be available for website viewing and printing in the Commission's Public Reference Room, 100 F Street NE, Washington, DC 20549 on official business days between the hours of 10:00 a.m. and 3:00 p.m. Copies of the filing also will be available for inspection and copying at the principal office of the Exchange. Do not include personal identifiable information in submissions; you should submit only information that you wish to make available publicly. We may redact in part or withhold entirely from publication submitted material that is obscene or subject to copyright protection. All submissions should refer to File Number SR-NASDAQ-2022-079, and should be submitted on or before October 4, 2023.