surreal rock
Hi. I have a problem with I/O for TCP sockets, mainly it either:
a. blocks, servers sends first respond then crash
b. it doesn't block, server doesn't get a send due to I/O buffer fill and back to crash.
I have tried threading to solve this, literally no change whatsoever. Can anyone help?
https://pastecord.com/ilopelodud
here is a code snippet
Beginner needs help with TCP in Python
winged walrus
I think you need to describe what you're trying to accomplish. Your code clearly wouldn't work but I can't tell what it's trying to accomplish.
surreal rock
So, I need to make a simple message client on TCP, that connects to a server. So far, I have managed to make it send the first HELLO message to the server, and get the OK response back. Basically, the handshake. IF I don't have socket.setblocking(False), the program quits immediately after the aforementioned server response
golden nebula
I think maybe rethink your design.
surreal rock
If I do put in the socket.setblocking(False), I get the buffer fill-up error, and all the methods online that i found to try and fix it have failed
surreal rock
Look into `select`
I have tried it and tbh not did i understand it much, nor did it work
golden nebula
I can code an example in a bit.
surreal rock
ok
winged walrus
You don't need a thread at all. Do the send then do a recv
golden nebula
For something as basic as sending and getting a response you don't need select also.
surreal rock
Yes, the threads thing has proved utterly useless for my use case, but i read online that it could stop I/O blocking.
also, in the very short guide we got for this uni assignment, there is mention of the thread lib
golden nebula
Is this for an assignment? Maybe you could list the requirements
winged walrus
Why are you worried about blocking?
golden nebula
So we can have a better idea what you're trying to accomplish
surreal rock
Sure
winged walrus
Recv is supposed to block while it waits for a response
surreal rock
But after it gets it, why does the whole thing quit/crash?
winged walrus
No idea
surreal rock
Since I can see the response is getting is the correct one, according to the protocol. I log that response in the terminal via print
winged walrus
The code you shared would crash for a lot of reasons.
surreal rock
Should I send the whole code here?
Since that was a snippet
it is 110ish lines iirc
winged walrus
You need to rewrite it I think. The code you shared was essentially nonsense haha
solar breachBOT
@surreal rock
winged walrus
You can only send or recv, you can't do both at the same time. So you send your message, then you recv the response after.
surreal rock
it is not complete, there is some code which will be used for the next part: aka the messaging which i have yet to implement, since i have tried to get the darn thing to work
winged walrus
Both block, but that's fine because you want to wait for them to finish/get a response
surreal rock
ok, then why is it broken? why does it quit after the right response is sent back?
i honestly dont know how to be more specific at this point
winged walrus
No idea. Your code is not logical and could be breaking for many reasons.
golden nebula
On Windows it looks like you'll want a single thread, on Linux or Mac you could do without one.
surreal rock
i am on mac
surreal rock
ok, so what should i do
golden nebula
If their code is for reading commands from stdin and sending them to a server they would on windows.
I think using the standard select loop on your socket is probably the easiest approach here.
surreal rock
this is the assignment criteria
appendix B, C is the socket/thread library with some minimal explanations
winged walrus
If their code is for reading commands from stdin and sending them to a server th...
Just do it sequentially.
surreal rock
Appendix A is the list of the commands the server expects aka the comm protocol
golden nebula
Hmm I think you're probably right if you had stdin in your select loop it might just always be open for writing.
I haven't tried it myself.
surreal rock
ok, i am very confused at this point..
golden nebula
You'll use your main process for user input and then a thread to interact with your server.
I think is probably the ideal way.
surreal rock
No idea. Your code is not logical and could be breaking for many reasons.
what part is illogical about it?
golden nebula
I'll write a small example.
surreal rock
I'll write a small example.
thank you
golden nebula
Are you familiar with classes?
surreal rock
in python - no. I did use classes in cpp
golden nebula
are you expected to use classes in your project?
surreal rock
i am not sure, doesn;t say anywhere. the "guidance" is literally that page i sent above
golden nebula
it makes things a little easier
but not necessary
i'll go ahead and not use them for the example
surreal rock
ok
golden nebula
import socket
import select
import threading
import queue
HOST_PORT = ("localhost", 5555)
g_message_queue = queue.LifoQueue()
g_stop = False
def read_user_input():
global g_stop
while not g_stop:
message = input("Enter a message: ")
g_message_queue.put(message)
if message == "quit":
g_stop = True
def process_data(data: bytes):
print(data.decode("ascii"))
def main():
global g_stop
read_user_input_thread = threading.Thread(target=read_user_input)
server_socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
server_socket.connect(HOST_PORT)
read_user_input_thread.start()
socket_list = [server_socket]
while not g_stop:
readable, writable, _ = select.select(socket_list, socket_list, [])
for fd in readable:
data = fd.recv(4096)
if data:
process_data(data)
else:
print("socket closed")
g_stop = True
for fd in writable:
try:
message = g_message_queue.get_nowait()
fd.send(message.encode("ascii"))
except queue.Empty:
pass
read_user_input_thread.join()
if __name__ == "__main__":
main()
@surreal rock
There is a obvious problem though here, and I'm not sure the best way to resolve it.
surreal rock
?
golden nebula
@winged walrus how would you handle the case where the server sends a response while you're waiting for input?
e.g "This message is from the server" is placed in a bad place.
You'll probably need to do some terminal magic to get it work nicely.
Here specifically.
surreal rock
```py
import socket
import select
import threading
import queue
HOST_PORT = ("l...
erm, can you explain the above?
golden nebula
which part?
golden nebula
We use it instead of mutexes and a list for message passing.
Between the thread and main thread.
surreal rock
socket_list = [server_socket], this is the select part, which i do not understand
golden nebula
select.select(rlist, wlist, xlist[, timeout])
This is a straightforward interface to the Unix select() system call. The first three arguments are iterables of ‘waitable objects’: either integers representing file descriptors or objects with a parameterless method named fileno() returning such an integer:
rlist: wait until ready for reading
wlist: wait until ready for writing
xlist: wait for an “exceptional condition” (see the manual page for what your system considers such a condition)
surreal rock
wait, but for the first one, the input, shouldnt that also be with the input from the user? like why just the socket
i understand that the output is like this, since only server sends response
but what about input
golden nebula
The reading is when we read from the server.
and the writing is when we can write to the server.
surreal rock
and that writing, shouldnt that include the user input? from the program
golden nebula
No, we want to wait for when the server itself is running recv
A file descriptor is ready for writing if a write operation will not block.
And when the server is ready to be written to, we will try and get a message from the queue and write to it.
If you have netcat, you can test this program pretty easily.
golden nebula
Well, there isn't a lot of information, but I don't think select is particularly advanced.
I have no context to what you've learned in this class.
I can only assume you've learned about the existence of select since it's such a useful operation.
If you want to instead, you could use multiple threads to recv and send I guess.
But this makes it nicer since you well, don't have to do that.
surreal rock
I have no context to what you've learned in this class.
python wise - nothing. We are supposed to just learn ourselves ig
golden nebula
Nothing in any lectures?
surreal rock
nope
golden nebula
surreal rock
no python content, only the layers part of networks
golden nebula
I see.
Typically you use select on multiple sockets in the server, this is probably what the server does.
But it's fine to use on a single socket as well.
surreal rock
we are gonna have to implement our own server as well a bit later
that has even less info provided
golden nebula
@winged walrus do you think select is advanced usage?
You could not use select and instead use timeouts I guess.
small timeouts should be fine.
winged walrus
!ban @sterile vessel scam