Logbook 4
A Logbook entry written by @tpierrain
Like the previous time, we started to discuss with Cyrille, and code after with Mendel.
Topic of the day was:
Hexadecimal?!? No. Hexagonal? for sure! For those that needed a recap on what the Hexagonal -or port and adapters- architecture is => click HERE(beware: shameless bragging!!!)
Ok then: let's get back to our mob thing:
After having whiteboard-ed our hexagon, we identified its 3 ports and adapters:
-
SmartOrderRouting API (investor-side)
-
MarketDataFeed API (markets-side)
-
OrderRouting API (markets-side)
We were trying then to figure out the form of the third one: the OrderRouting port/adapter. Should it be a protocol or an OO API as we started to implement since the beginning? Should we introduce a kind of common layer upon all the various kind of markets to handle and protect from race conditions in the way we received the notifications/information? ... Wait a minute! Did you just said "race condition"?!? (you may ask "why" here).
Yes, you should... (in fact it will help me if you do it right now...)
- "Why?!?"
What does it mean? Cyrille illustrated that to us through the whiteboard:
Every time a client (like our SOR) is sending an Order to a Market, there are many steps involved on the Market side:
-
The Order book of the Market is updated, and that triggers a potential Execution via the Market's Matching Engine
-
This Order(s) matching will then send an Order(s)Updated message (Message 1 on the diagram) to the OrderRouting Market API used by our SOR.
-
Depending on the Market, an optional Execution notification message (Message 1bis) may also be sent to the OrderRouting Market API used by our SOR.
-
In every case, this Order matching will also lead to a Trade generation on the market side (next step after the Execution). To do so, the Market have to collect some needed extra data to be sent as part of the TradeInfo message (Message 2). This TradeInfo message is sent to the connected OrderRouting Market API of the involved Market client (i.e. our SOR).
-
On the Market (data) Feed side of the considered Market, this new execution may lead to a LastExecutedPrice update notification (Message 3) forwarded to every MarketFeed API connected to this Market (i.e. not via the OrderRouting API this time).
-
But this new trade will impact more deeply the FeedEngine of the Market, that will also send a FeedUpdated notification (Message 4) to every connected APIs (once this new execution will have impacted all related prices, available quantities/depths for this Instrument, etc).
Sounds logical right? But in reality
For a given Market you may receive messages 1, 2, 3, 4, with another Market you will receive 1, 1bis, 3, 4, 2 ... etc. It is also worth-noting that our SOR solver is mainly interested by 3 and 4.
Anyway. Our SOR must somehow be able to cope with those Market discrepancies. The question Cyrille, Mendel and I have been asking was:
- WHERE to handle those functional race conditions?
And our answer so far:
- Somewhere part of our business logic, and not embedded in a kind of Market APIs wrapper.
'Cause we don't want a leaky abstraction layer hiding from us some of the good Markets information and capabilities...
Ok. Cyrille left us, and Mendel and I started to code.
Funny thing: so far we were talking about the OrderRouting and MarketFeed APIs, but in fact we started to work on the SmartOrderRouting public API on the Investor side.
Indeed, we started to build a C# raw (i.e. without any middleware nor HTTP involved) port, and an adapter to translate from a flat DTO world ... to our internal business logic (OO world).
We continued to build it in TDD mode, but we've been asking tons of questions during the elaboration of this new public API for the SOR: how to expose our events in a non-OO world? How to do it without creating functional race conditions (where you post an investor instruction, and the corresponding answers/events are raised before you had the opportunity to subscribe to them, etc.).
As of today, you can have a look at:
- Our new service/API interface (BTW, we created a new project for SOR interfaces)
but also to
Note: We didn't have much time to work on it (lot's of talking before, even if it was necessary). So don't be cruel with us ;-)
Well guys... I think this is enough for today's Logbook entry. Next session, with (almost) the entire crew will be much more interesting regarding this SOR hexagonalization topic.
To be continued thus...
A Logbook entry written by @tpierrain
Without Cyrille, but with Ozgur and Tomasz this time, Mendel and I started to explain what we did during the previous session.
In fact, it turned into to a larger discussion. Even more: into a usefull clarification of what hexagonal architecture (a.k.a. Ports and Adaptors) meant to every one of us.
Yeah, because it was obvious: Mendel and I weren't clear enough for Tomasz and Ozgur to quickly grasp what we did last monday (and what we intended to achieve in the long run). Thus, we -as a mob- decided to take a pencil, and to go to the whiteboard for clarification.
Everyone then tried to explain to the others the kind of hexagonal architecture he was willing to implement. As you will see below, we weren't exactly at the same page ;-)
Drew something that I tried to reproduce here in order to better grasp what was in stakes (note: the original white board picture is also displayed at the end of this post):
What do we have here: one Market on the top-right corner, one Investor (i.e. user of our hexagonal SOR) on the bottom-left. Lollipop representation for interfaces. The center of the (hexagonal) SOR for the applicative and domain code. And the weighted border of the hexagon for the infrastructure code landscape.
Every port/adapter combo being drawn as a blue-dashed rectangle, they all expose whether:
- A public interface to the external investor users (mainly outside->inside interactions)
or
- An interface to sustain interactions with the Markets from the inside of the hexagon (mainly inside->outside interactions).
While Tomasz was drawing his diagram (slighly different as this one, I must admit), I was bothered by something in his model: all the ports/adapters were somehow connected to each others with lines. Like if they were consistent. As a whole.
According to me, there is no need to share anything between the external public API of the SOR (on the investor-side), and what the SOR needed to play with for its external interactions with the Markets. I don't even want to add a coupling between those 2 different kind of port/adaper blocks (the left ones and the right ones).
This is where I decided to take the pencil, and to draw the Mendel-and-I version of the (ports and adapters) hexagonal architecture. At least, that's what I thought at the time... ;-)
Let's discover this:
Reproduced here just below, my graphical version was trying to highlight the isolation between all those sets ot port/adapter.
Ok. All those port/adapter may share part(s) of the SOR business logic (at the center of the hexagon), but not even necessary.
Drawing my version had 2 consequences:
-
Tomasz realized that we didn't get what he wanted to explain with his diagram. He clarified then that the lines on his diagram were here to suggest a kind of pipeline with some port/adapters. According to him, the debate was much more related to the boundaries of our hexagon and where port/adapters instantiation & composition should be done.
-
Funnier thing: Mendel and I just realized that we didn't share exactly the same view of what we built together ;-P
In particular regarding the SmartOrderRoutingService (i.e.: our first port/adapter for me; something slightly different for Mendel...) Nice transition to present:
Attached to the YAGNI principle, I was considering our SmartOrderRoutingService as the first port/adapter for our SOR (a raw C# port/adapter on the investor-side).
Mendel considered it as a kind of common layer for any port/adapter (to be added) on the investor-side.
Graphically speaking, Mendel's proposal could be represented like this. With the SmartOrderRoutingService port/adapter represented as the blue dashed form on the left (with 2 different interfaces pluged-in):
After those clarifications, we continued to talk about which version of the hexagonal architecture to implement all together. In fact, we really couldn't manage to share the same vision of the technical solution to be implemented (every one of us probably still being stucked somehow in our own mental model ;-)
Not able to collectively agree on the solution? => we decided to focus -at least- on its objectives:
And in fact we quickly agreed on 3 points to be implemented, whatever how we will code it:
- Our SOR business logic must be fully encapsulated. Why? Because we need to be able to switch from one implementation (e.g. C# LMAX-like) to another (e.g. F#) without breaking the public API (the one to be used by our common acceptance test harness).
- As a good side-effect, this will allow every of our SOR technical implementation to freely evolve without breaking these (a kind of anti-corruption layer effect)
-
We need to expose a public interface (understand "contract") for the investors (DTO oriented, and not OO)
-
The first user of this public interface will be our upcoming acceptance test harness Important to have in mind in order to continue to respect the YAGNI principle.
Ok... Near 1 hour of lunch-whiteboard-discussion... and still no test/code written ;-(
Mendel suggested that we got back to the code... and we all agreed on that ;-)
But first things first. We all decided to update our "next steps" backlog file (the equivalent of Kent Beck's tiny piece of paper while he's working). Why's that? Because we didn't have lots of remaining time to code today. Thus, it seemed important to us to secure our next session productivity (by saving us some time to re-synch on our objectives next time).
-
To introduce the concept of (financial) instrument (mandatory to support the concept of Market data feeds)
-
To introduce the notion of market data and the MarketData API & Port/Adapter (starting from our acceptance tests)
-
To introduce the OrderRouting API & corresponding port/adapter
-
To create our acceptance test harness (i.e. a console application generating a report after each runs session)
Once these next steps written within our backlog file, we started to code by introducing the notion of Instrument Identifier. But quite shortly:
__Ding-dong! End of the session :-( __
As previously said in this post, here is the picture of our original whiteboard (part of it being erased before we took this snapshot):
