Binance websocket connection update time limitation

[opal hare]

Proposal to get the BestBid and BestAsk (Level 1) prices real-time from ws bookticker stream instead from the depth ws stream with the 100 ms update interval.

Full description is below:
https://github.com/hummingbot/hummingbot/issues/6014

[mint anchor]

Would you not need to be able to place orders in less than 100ms for this to be actionable?

[opal hare]

No, as at the time when OnTick() method is called you don't know the current BestBid and BestAsk. The order placement frequency can be any.

[opal hare]

the tick size can be any, but that is the interval when the orders will be sent to Binance.
On tick size iteration the HBot will send limit order to the BestAsk price 19450, but the real price on exchange is BestBid 1950 and BestAsk 1951 because the updates comes with 100 ms. The limit order Sell 19450 will be filled now as market on exchange and placed to BestAsk.
How to make the markets if you don't have the right prices?

[round jungle]

Yes, Real-time order book data is an essential function for market maker. How many milliseconds do you make a onTICK() request for BTC/USDT before?

[mint anchor]

This is possibly my poor understanding, but wouldn't market makers place ask/bid orders with a spread around a definition of current price? Anyway, I was just curious

[opal hare]

That is simple, the BestBid and BestAsk and it sizes are used in calculation of the definition of current price.

And execution algorithm should place the order to BestBid and BestAsk if the conditions are met.

[opal hare]

No actual bestbid and bestAsk prices = no actual definition price.

[mint anchor]

What surprises me is that this time sensitivity would become critical in the context of market making, with protection spread and 1 tick/s - Again, I just wanted to understand better and I suspect that the real benefit of this proposal resides more on the potentiality of HB rather than traditional MM. For example, I can definitely understand it better when one adds place/cancel orders on WS in asynchronous fashion. This actually reminds me of a strategy I had started and stopped developing because I needed to have a cascade of orders needing fast execution to chain-down and thought it may not work with HB

[opal hare]

Need the asynchronous update of the data when available from exchange. Order placing frequency doesn't matter as even for each 1 second or 5 min or whatever onTick() iteration, the Hbot doesn't know the current bid ask prices as it outdated for 100ms.

[patent basin]

GG @opal hare, you just advanced to level 7!

[opal hare]

The limitation of the tick size is another question if the updates of data will be real-time.

[placid gyro]

@vagrant beacon @junior walrus @mint anchor @winter obsidian @devout steeple I tag you here since you are the most active devs on the community

what do you think about this proposal?

I think that we can't override the top bid and top ask since we are using the diffs to update the full order book

so we have to create another object if we want to keep track of this

or we can create a data_feed to handle this new channel

but I would like to understand what do you think about this overall

[winter obsidian]

i think we should have a new data feed called realtime_ticker or something as an optional module for exchanges, make the connector bid/ask price functions call that if it exists, if not fetch the top price from the order book

[mint anchor]

Second. Would it make sense to have it run in its own thread/process/socket-connected app?, this may be quite a busy fellow on high-volatility market

[winter obsidian]

Yeah I think it should go to the async_call_scheduler which appears to run in a thread pool

this could pave the way for one of our next steps with orchestration which was to use shared order book feeds - make the connector price functions more modular

[mint anchor]

I think @vagrant beacon would voice his preference for a separated socket-connected standalone app. At my end, the experiments with the async concurrency with time accuracy made me skittish of using async for anything needing less than several 100s of milliseconds reliably - I have not looked at the async_call_scheduler though

[winter obsidian]

socket connected app would inherently add latency although that is what we planned with mqtt

all of hummingbot is async 🙂 async concurrency and time accuracy are affected by how many tasks are sharing the same scheduler

if the price feed is running in it's own thread with the async_call_scheduler then the async tasks are only affected by what's in that thread

[mint anchor]

Right, I tend to agree. That's what I used to tell him, but somehow he left a lingering doubt as to whether the socket-connection would not perform well. It can only be settled by live testing though, I was very surprised when I saw the Clock being so inaccurate, but with the widely common HB tick size of 1s, this turns out not to be an issue. - As a side, the implementation of the clock in its own thread seemed to be much better, to confirm my support for giving the asyn_call_scheduler a try

[devout steeple]

I would recommend to implement it as a standalone process that broadcasts data to the listening bots via a message broker.
This is the plugin-based approach that we introduced with the MQTT layer.
Distribute resource-hungry tasks...
Message brokers such as RabbitMQ and EMQX can handle A LOT of traffic.

[patent basin]

GG @devout steeple, you just advanced to level 7!

[devout steeple]

That is exactly an application example of the PROS of the new communication layer for hummingbot client.
Lets start building such "plugins" is my line!

[mint anchor]

Not sure I understand your opinion, you 👍🏾 async_call_scheduler while also recommending standalone + inter-process messaging

[devout steeple]

👍 for the position (Means true that for me 🙂 )... not for my recommendation...
Both solutions will work, I recommended the external one

[mint anchor]

Hopefully @vagrant beacon will burst in any time to release me out of that sitting-in-the-middle ... sometimes it's good, sometimes you suddenly realize that it is not a comfy chair, but a sushi chef's tuna knife ... 👀 😱

[vagrant beacon]

I do not mind whether it's socket-based or not but it has to be a subscription/push-based (as @devout steeple pointed out, with a proper comms layer it can be interprocess shared memory, named pipes, mailslots, threads, internal events, raw TCP, raw UDP, other slower protocols on top of TCP/IP such as MQTT, HTTP... whatever) ... that way... even if the actual decision-making bot subscribes to multiple types of feed (orderbook, tick, etc)... the most recent update will be the actual source of truth

[vagrant beacon]

Of course, if it is a feed that will be read by more than 1 client, then an external process is ideal

Having said that, it is important to not be too fixated on the technology (or worse, just the library) and lose sight of the big picture... in this case, what we want is to always have the most up-to-date best bid/ask... which means... whichever feed gave us the most recent price (be it orderbook diff, full orderbook snapshot or top of book feed... all come with timestamp), we take that as the actual top of book, regardless

Of course, if the 2nd feed pushes tick data with a timestamp that is earlier than our last valid timestamp, we disregard it and not update the top of book... and conversely, if a new update to the orderbook comes in and has a more recent timestamp, after we update with the snapshot or the diff, the side effect of that update is that it MUST also update the value of top of book

[devout steeple]

other slower protocols on top of TCP/IP such as MQTT, HTTP... whatever

Bidirectional transports, such as MQTT, wont introduce any noticable latencies, as the channels are kept alive after the initial handshake

I have studied and used MQTT in my PhD, for resource constrained environments with heavy data transmissions, in the context of IoT and Cyber-Physical Systems...

Though, to be honest, the best performance-wise solution would be Redis...
I am letting outside the conversation any custom created protocols over plain UDP transports...

[devout steeple]

Our intention with @winter obsidian is to enable selection of the communication middleware for bots (aka the Broker Bridge that was introduced in the latest development branch) - MQTT, AMQP and Redis.

[winter obsidian]

all in on comms-based approaches but since the issue at hand here is a 100ms bottleneck i thought using an external service would incur latency as opposed to a separate thread

[devout steeple]

We should measure it to be honest.
Also we should gather the requirements. We want to have the data in-strategy at a 100ms interval exactly?
If latencies occur, we can hold a buffer and send batches to the strategy.
Context switching within a process also introduces latencies...

[winter obsidian]

@devout steeple no 100ms is the current interval and this was about having as close to 0 as possible using the real-time ticker from binance

[placid gyro]

idk if it's a good idea to replace the top bid and ask with this stream. Mainly because is retrieving the prices of the top of the book, and if the bid cross the ask for example, you will need to delete rows from the current OB and you are going to have like missing or fake volume around that price

[mint anchor]

So as a learning project, I have been playing with the Clock running on thread. Could I use the new implementation to experiment with having it as a external process that communicates with the rest of HB?

[vagrant beacon]

I don't get what you meant by this

[round jungle]

@vagrant beaconHe said the best bid and ask came from prices of the top of the book, which at a 100ms interval . So you must build other source system to get real-time order book first. Then you can choose any mode.

[vagrant beacon]

Hmm... but

I think that we can't override the top bid and top ask since we are using the diffs to update the full order book
seems to imply that the internal top of book values will not be in-sync with the order book updates

[round jungle]

Yes, so need external real-time full or the top 20 order book data source. then you can choose and do anything. This is basic tool for markert maker.@vagrant beacon

[patent basin]

GG @round jungle, you just advanced to level 2!

[vagrant beacon]

What I was trying to say was... if the order book was updated (even if it was via diff), the latest internal snapshot post-diff should contain the latest top of book

and a change of the values in order book should trigger a change of top of book values

But perhaps I misunderstood what @placid gyro was saying lol

[round jungle]

No,the new external order book cannot be updated via diff if you want real-time data,I think it only come from websocks, not use API request?@placid gyro

[opal hare]

I recommend you to study the API docs first and how the order book updates comes from the websocket Binance API.

[opal hare]

I think the first step is to synchronize the onTick with the diff stream OB update, than the HB will have the actual real-time slice of data of the Top 1 when the onTick is called.

[vagrant beacon]

or if we set some ground rules, for instance... saying that if we get real time top of book data, we will update our internal representation of top of book as well as the timestamp of last update... at the same time, if order book gets updated and the timestamp is more recent then the 'side effect' of that is that the top of book changes propagates to the same internal object (of top of book)

That way there shall only be a single source of truth, regardless of when the data is read

[round jungle]

@opal hareFede understood it is difficult, you call him

[vagrant beacon]

... instead of only sync-ing things on onTick

for now of course, if it's the same websocket connection (but with different message types... order book diff, top of book stream, etc), things should come in the right order... but it is good to still store the timestamp so that things still "automatically" work if we decide to read realtime data from a different stream

does that make sense?

[opal hare]

you are right, it is the same ws connection with 2 streams, this should works fine.

and this approach is not required to develop external module.

[round jungle]

Fede understood hummingbot code system and worked at HFT quant company before. He kown all of your problem.

[opal hare]

and async_call_scheduler also needed in addition to onTick

to user the Top level data and react async to its changes

[vagrant beacon]

Yup... so if we do choose to not subscribe to the real time stream, then the event that gets triggered when real time stream is pushed to the bot is never called, that's all... everything else should just work

[opal hare]

yes, without any changes in the code.

[round jungle]

@opal hareYou can try it😄

[vagrant beacon]

Not that it makes much of a difference but I have been developing HFT systems for 2 decades now for tradfi 🙂

But of course, Fede is definitely infinitely more conversant in the Hummingbot codebase than almost any of us

[round jungle]

What is tradfi

[vagrant beacon]

Defi = decentralised finance = the current crypto world
Tradfi = traditional finance = the rest of the financial markets...

[round jungle]

Got it, Furtures ?

[vagrant beacon]

Options, Futures, Bonds, Warrants, FX, Stocks

[opal hare]

So I vote for this implementation, that is optimal solution to fix.

[vagrant beacon]

But depending on how the Hummingbot codebase is written (I did not go that much in-depth to read them all), that may or may not be optimal for the Hummingbot architecture

[round jungle]

@vagrant beaconYour Tradfi system OB real-time?

[vagrant beacon]

OB?

[opal hare]

Sounds cool, what do you think about the https://github.com/sstoikov/microprice model to use as fair value of price

[round jungle]

OB=order book@vagrant beacon

[vagrant beacon]

it depends on the strategies that we deploy for that particular instrument/season... just like when @opal hare talked about micropricing... for some of our strategies we don't even need the actual tick data because some of our strategies only rely on synthetic prices... there are various 'fair price' and micro price models... which may take into account risk-free rates, expiration (for futures and options contracts), order book imbalance, 'dust', etc

[patent basin]

GG @vagrant beacon, you just advanced to level 24!

[vagrant beacon]

Sometimes we even directly incorporate multiple indicators to affect the tick prices in order to generate a single stream of synthetic price (in place of tick price)... so that the bot only need to work on 1 data stream and need not even look at indicators... the synthetic prices are usually generated by our code in FPGAs

[round jungle]

FPGAs? Great

You should use c++ for FPGA

[opal hare]

That is the next level HFT required for TradeFi

[vagrant beacon]

We usually use VHDL for that (because it's strongly-typed as compared to Verilog... very useful in preventing bugs arising from silly mistakes from weak-typing)

[round jungle]

Excellent, similar to C

[patent basin]

GG @round jungle, you just advanced to level 3!

[round jungle]

@vagrant beaconThe TradeFi fee is very low, less than 0.01%, which is suitable for HFT. But cryptocurrency is not suitable for HFT becouse of high fees and low liquidity. What do you think

[vagrant beacon]

if you utilise the same amount of capital that tradfi shops use, in crypto... you can quite easily get some special deals with CEXes and/or upgrade your VIP level. VIP 9 in Binance gives you 0% fee for maker-side orders for futures, and 0.01% rebate for futures pairs with BUSD for VIP 5 and above

I think the main problem why HFT (in the traditional sense) and crypto do not play well together is mainly because each DEX need their own liquidity pools (hence the liquidity problem you mentioned, ... and the decentralised nature of the blockchain requiring confirmation of different nodes physically separate from each other mean that it can never really (for now) offer nano-second latency

At the same time, most CEXes do not even have proper infra to support HFT... it's very common to see CEXes' matching engine or even their streaming engines suffer from hiccups when there is a surge in the number of transactions

But it just means that there are now newer (and different sets of) opportunities for other traders... MEVs for one

so it's just a different way of playing the HFT game

[round jungle]

Totally agree. MEV similar to DEX？

[vagrant beacon]

https://www.coindesk.com/learn/what-is-mev-aka-maximal-extractable-value/

[placid gyro]

@vagrant beacon @round jungle @opal hare let me create an example to show you why I think that we can't override the top of the OB.
convention: (a|b)price:amount
1st message: Snapshot

b97:1 | b98:1 | b99:0.5 || a101:1 | a102:1 | a103:0.5

2nd message: Diff

b98:2 and b99:0 --> OB now: b97:1 | b98:2 || a101:1 | a102:1 | a103:0.5

3rd message: realtime top bid and ask

b102:0.5 | a103:1 --> OB now: should we delete a101:1 and a102:1 (yes because the top bid is crossing them)? what about the volume of the levels at the left of b102:0.5 now?

Also this computations are expensive since you will need to compare in which part of the book you are

that's why I think that the best solution is to create another source that is just getting that information

[vagrant beacon]

I think you got me wrong. The top of book stream CANNOT override OB... but OB delta updates MUST override top of book if the timestamp of that message is more recent than the timestamp of the top of book message

[round jungle]

@placid gyroI'm not a developer. That's also what I mean. But Lagamail didnot believe what I said.

[placid gyro]

yes that is how is working right now

but at first they wanted to override the ob with this new stream

and for mi is not a good idea

[vagrant beacon]

it is better to for each asset pair to have its orderbook, and best bid and best ask property

[placid gyro]

exactly

[vagrant beacon]

if code is written to take advantage of new streams (for any exchanges or custom source) then the stream with most recent timestamp updates the best bid/ask

[placid gyro]

but at the same time, idk if all the exchanges are offering that stream

in real time

[vagrant beacon]

that's why I said this

[placid gyro]

what I mean, is not a change that we can say, okey let's plug and play this new stream

[vagrant beacon]

if there is no such stream then the only source of update for best bid/ask property is whenever OB is updated

[placid gyro]

mainly because we have to modify the OB class

[vagrant beacon]

that way, we can plug and play

[vagrant beacon]

Oh but my brain agrees with my heart... they are both saying... for Federico this is a piece of cake... just a few lines and before the next cup of coffee is done brewing the OB class will already be successfully modified... AND tested 😄

[placid gyro]

haha 😉

but now I'm working in the candlesticks module

[vagrant beacon]

It's alright... my 6th sense is telling me that you need to take a quick break and distract your mind a bit after working on candlestick module for a while

[vagrant beacon]

how about OB? 🤣

[placid gyro]

almost ready

[vagrant beacon]

don't lie... the source code says otherwise

SEE? PROOF! 🤣

self is not ready

lol

[placid gyro]

🤣

you catch me

[vagrant beacon]

off topic... when do you decide to use snake case and when not to use them? e.g. extendleft vs extend_left

[placid gyro]

historical vs realtime

[vagrant beacon]

as in... self._candles.extendleft vs self._candles.extend_left?

[vagrant beacon]

I don't code in Python so I'm just asking out of curiosity

[vagrant beacon]

https://tenor.com/view/nosy-chisme-stay-away-you-are-nosy-rumors-gif-21887495

[placid gyro]

ahh haha

is a data structure

[vagrant beacon]

Ah I just realised that it's because it's most likely because _candles is a deque and that's how the original author named his function... extendleft 😄

[placid gyro]

yes

[hexed eagle]

Hello 🙂
I've one doubt and I see that you know very much about this subject, so maybe you could help me, please? 🙂
I'm working with a exchange that when there is any update in the order book then the exchange always send me the new complete order book (no diffs messages). So, in the method _parse_order_book_diff_message of my connector I'm always returning this:
OrderBookMessage(OrderBookMessageType.SNAPSHOT, order_book_message_content, snapshot_timestamp)
I supposed that working in this way Hummingbot would update the complete order book.
The problem is that when in Hummingbot user interface input panel I enter order_book the order book is never updated.
Do you know how I could handle this situation?
Thank you very much in advance!! 🙂

[placid gyro]

hi! you should apply the snapshot only and you are not going to use the _parse_order_book_diff_message since you are not receiving diffs