hardy dune
import asyncio
loop = asyncio.get_event_loop()
server = await loop.create_server(...)
try:
async with sevrer:
await server.serve_forever()
except asyncio.CancelledError: # Client still alive
print(server.sockets) # ()
Seem I cannot get all alive connections or kill them by public api 😦
The left connections keeps until python executable exits.
What show I do?
quartz coralBOT
```python
import asyncio
loop = asyncio.get_event_loop()
server = await loop.cr...
@hardy dune
Python help channel opened
hardy dune
Note that
async with server:
...
equals to
server.close()
await server.wait_closed()
brittle trout
perhaps you're running into this?
since wait_closed should actually wait until all clients connections are closed
and that is the behavior that I'm observing from my testing
manic sun
supposedly server.close() should close its sockets as well, but curiously when i wrote my own test, it hanged on 3.12 only, not 3.11
https://paste.pythondiscord.com/JUVA ```py
py -3.11 combined.py
Server established connection
Client established connection
Client sending hello world...
Server received b'Hello world!'
Client sending hello world...
Server received b'Hello world!'
Client closing connection
Cancelling server...
Server took 0.00s to exit
Server closing connection
py -3.12 combined.py
Server established connection
Client established connection
Client sending hello world...
Server received b'Hello world!'
Client sending hello world...
Server received b'Hello world!'
Client closing connection
Cancelling server...
Server closing connection
ah wait i forgot writer.close() in the server callback, if i add that and run it in 3.12, it closes after 2 seconds
brittle trout
yeah, but it's still weird that it just hangs forver for uncooperating clients
manic sun
oh wait, i misread the docs:
https://docs.python.org/3/library/asyncio-eventloop.html#asyncio.Server.close
Stop serving: close listening sockets and set the sockets attribute to None.
The sockets that represent existing incoming client connections are left open.
icy pond
I saw that the serve_forever already captures the CancelledError, and does a close as well. it's probably that server.close that removes the sockets.
manic sun
so its supposed to reject new connections but leave existing connections open
and in OP's example their "protocol" object has no way of calling transport.close(), so the server can never close it
icy pond
so you might be required to keep track of connections yourself somehow.
async def wait_closed(self):
Historical note: In 3.11 and before, this was broken, returning immediately if the server was already closed, even if there were still active connections. An attempted fix in 3.12.0 was still broken, returning immediately if the server was still open and there were no active connections. Hopefully in 3.12.1 we have it right.
brittle trout
yeah, but it's still weird that it just hangs forver for uncooperating clients
what I find weird is this, the server doesn't stop until the client does, even though it was cancelled way before
so, I suppose this is that wait_closed working as it should and blocking since there are active connections, but shouldn't the client reader get cancelled at this point? I would expect to see Server received cancellation there, but that's not the case, the server only ends once the client task ends, with that break stmt on EOF. Am I really expected to somehow handle closing all clients when my server task is cancelled?
icy pond
yea server_forever calls await self.wait_closed() on a cancelled error, and that's where it blocks. the cancellation error never gets propagated until that wait_closed realeases... so it's pretty much pointless around the server_closed() until all connections are closed.
brittle trout
that's so annoying though, that would mean I have to have another task that just waits for an event that I have to trigger just before cancelling server, which would go over the active connections and cancel them?
icy pond
so the only way I currently see around it might be.. whenever you cancel the server task, you should also go around and cancel or close all connected clients yourself.
brittle trout
yeah so basically what I mentioned, though it doesn't need to be a task waiting on an event I could just do it right before, that was just me overcomplicating things as usual lol, hmm still kinda annoying though
icy pond
that seems to work somewaht. the client now receives a ConnectionResetError on writer.drain
I wrapped the callback and run_server in a class, so I can keep track of all clients, and at the very bottom, I simply loop through the current clients after cancelling the server, and close them all.
(oh and never mind the write_eof, that was just me testing, doesn't really do anything useful here, I think - works the same without it)
brittle trout
yeah, but it's super annoying that you need a reference to the clients all the way down to main because of that internal try-execept in serve_forever
icy pond
well, you can keep it in the server class.. have a "shutdown" method or something like that, that does all that.
dunno
brittle trout
yeah definitely, it just surprised me that "blindly" cancelling the server will make it hang forever unless all clients disconnect
icy pond
I assume it's made that way, to give control of the termination to the dev... in case you want to make some final transmissions to all connected clients.
brittle trout
you could also like make your own version of serve_forver
so you don't have to deal with the internal try-except and first cancel all clients on your own
icy pond
that's what I temporarily did to figure out where it blocks.
also, server.sockets doesn't seem to contain the client sockets... but only the listening sockets (you can have more than one, e.g. by giving the host argument a list of ips instead).
it seems to me the asyncio.Server might not be tracking the connected clients at all.
yup, definitely only the server listening sockets
brittle trout
yeah, it just has _active_count to keep track of the amount of attached sockets
which is used for the wait_closed
icy pond
a new version with a custom (mostly copied) server_forever:
https://paste.pythondiscord.com/WNWQ
now the server.cancel() works as envisioned
brittle trout
yep, already did something similar too
honestly kinda annoying that this much work is necessary to achieve something like this though
it'd be great if serve_forver just had like a bool flag arg for this
naive arrow
I like using the with statement in the first place. Handles all that for you 🙂
brittle trout
I like using the with statement in the first place. Handles all that for you 🙂
well serve_forever has internal handling for closing
which means the with statement is actually useless, and that's the problem
since we need logic in between that err handling that it does
icy pond
the asyncio.Server doesn't seem to keep track of the connections, so it's probably difficult to achieve reasonably within that server. it seems the design was straight up the resposibility of the developer to keep track of all connections. which is fair, imho
brittle trout
oh yeah, that's true
naive arrow
well serve_forever has internal handling for closing
I see
icy pond
I like using the with statement in the first place. Handles all that for you 🙂
yea, the main problem is that the async with never exits, so never gets a chance to do clean-up, because the serve_forever waits for all connections to close before returning (or probably waits for all the client call backs to return - I doubt it actually checks the connections).
or at leas some kind of callback you could register would be great.
server.add_closing_callback(...) or something like that, that would get called, when the server is in the process of closing, so you can clean up. currently there doesn't seem to be a way around that except by overriding or implementing a custom serve_forever.... same for __aexit__ btw.
brittle trout
yeah, a custom callback would be a nice solution
Well, this was a lot of fun to look into and I was also able to find a pyright bug while working on this as I was awaiting a noreturn coroutine in that custom serve_forever impl, lol (https://github.com/microsoft/pyright/issues/8826)
Thanks for having someone to bounce ideas off of @icy pond!
icy pond
https://paste.pythondiscord.com/V6OA
here another one, without custom serve_forever... instead the asyncio.Server is stored on our own Server, and the client connected callback registers a simple done_callback onto the future that is used by the server_forever internally... it's a bit of a hack, but much shorter than the other solution.
quartz coralBOT
Python help channel closed