Help with setting up a TCP/IP Server or Suggestions for Alternatives

[whole jolt]

Hello, so I am attempting to create a TCP/IP server using my Raspberry Pi Pico 2w as the client that sends data to a server on the desktop. Creating the server in a language like Java will make things easier but maintain performance.

I can already successfully have the temperature probe detect the temperature and send it in a serial port on my PC where it is read. I have also managed to leverage the multicore tools provided by Raspberry Pi to be able to have one core run the temperature probe and the other the client. I did a quick run by creating a server in Python and had the pico connect to the server. However, this is where the problems begin, the server doesn't respond to every packet submitted and it actually just stops there until one packet is able to be read and the client connection ceases. The server is able to successfully decode the information.

I used the examples provided by the documentation but the multicore implementation as well as some of the LWIP stack had to be done through thorough(probably trashy on my part) reading of the documentation. (Don't worry none of it is AI generated so there won't be surprises for me)

I have provided the main code.

This is the output I get

Writing 64 bytes to client
        FROM CORE 0! The temperature is: 23.812500 °C
        0: 23.812500

This basically keeps happening until 

tcp_client_sent 1460
Sent more than buffer size
tcp_client_sent 1420
Sent more than buffer size

And then this keeps looping

Failed to write data -11

[final flint]

Detected code, here are some useful tools:

<@&987246924425994290> please have a look, thanks.

[final flint]

I uploaded your attachments as Gist. This makes them more accessible, for example to mobile users.

[blazing holly]

https://www.raspberrypi.com/documentation/pico-sdk/index_doxygen.html#raspberry-pi-pico-sdk

Do you use this doc?

[whole jolt]

Yes.

That's where I learned to use their multicore feature.

I was also able to set up an http server quite easily but that's just because they basically do all the work for you.

[blazing holly]

check this repo https://github.com/LiamBindle/MQTT-C

they made a full mqtt client to use

Like, do you actually want to do it all by hand to learn it? Or just do it?

[whole jolt]

I'd like to learn it at the cost of my mental health.

[blazing holly]

Ok, I just read the question from start.
So your last hurdle is python server disconnecting?

from what I understand, you send 1 data point to the server in tcp/ip, the server is a simple python tcp loop that get it and read it, then in the second loop it fail.

[whole jolt]

Right. Well the client keeps sending packets and yes once one is actually received and decoded, the connection ceases.

[blazing holly]

how is your python server writen?

[whole jolt]

Let me run it and I'll show u the code and output

[blazing holly]

Ok

The context is pretty important, knowing how far you are, I will not offer mqtt xD

[whole jolt]

Ok lol. I mean once I'm done with this project, I'm going straight to MQTT lol

Here is the python

import socket
import threading

import struct

bind_port = 4242

server = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
server.bind(("0.0.0.0", bind_port))

server.listen(1)

print(f"[+] Listening on port 0.0.0.0 : {bind_port}")

#client handling thread
def handle_client(client_socket): 

    request = client_socket.recv(64) 
    print(f"[+] Recieved: {request}")
    
    temperature_b = request[:4]

    temperature = struct.unpack("<I", temperature_b)
    
    print(f"The temperature is: {(temperature[0])/16.0} degrees Celsius!")
    #sending back the packet 
    #client_socket.send("Ping recevied".encode()) 
    #client_socket.close()

while True: 

    client, addr = server.accept() 
    print(f"[+] Accepted connection from: {addr[0]}:{addr[1]}")
 
    client_handler = threading.Thread(target=handle_client, args=(client,))
    client_handler.start()

The output is this

[+] Listening on port 0.0.0.0 : 4242
[+] Accepted connection from: 192.168.1.35:52414
[+] Recieved: b'\x81\x01\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00'
The temperature is: 24.0625 degrees Celsius!

[final flint]

Detected code, here are some useful tools:

[whole jolt]

And it just stops after that because the connection is gone.

That's it. It just receives one packet.

[blazing holly]

can you add a log in your c code

my bad...

[whole jolt]

Hmm well the log I added doesn't get called.

[blazing holly]

does printf("Writing %ld bytes to server\n", BUF_SIZE); continue?

so this is ```
Writing 64 bytes to client
FROM CORE 0! The temperature is: 23.812500 °C
0: 23.812500

it

Are you doing c or cpp?

[whole jolt]

C. It gives me the failed to write to server error.

"Failed to write data -11"

[blazing holly]

are you using this?
https://github.com/raspberrypi/pico-examples/blob/master/pico_w/wifi/tcp_client/picow_tcp_client.c

[whole jolt]

Absolutely

There's no way I figured this out on my own

When I use the example server and client as is, it works.

The two picos manage to send and receive data.

Right now I'm basically trying to make my own implementation for what I need.

[blazing holly]

Yeah I pretty much see that xD.

I think the problem come from a buffer you do not clear

I just do not see which one.

Is there a place you have an index on the data sent to the python server? an index to a position in a buffer?

Sent more than buffer size
Like that should mean you went out of your buffer

[whole jolt]

Yeah so in the tcp_client_send_data function, I reset all the buffer bytes to 0 and then put the temperature bytes.

[blazing holly]

at what position to you put that temperature bytes, is it always at 0?

[whole jolt]

0,1,2,3

I spread out the uint32_t to 4 uint8_t cells.

[blazing holly]

what line is that?

this?

[whole jolt]

Yes

[blazing holly]

why?

[whole jolt]

Because the buffer only holds uint8_t so I just spread it out to be unpacked by the python server.

[blazing holly]

where is that buffer defined?

I dont see

[whole jolt]

This is my header file

#ifndef SERVER_SETUP_H_ONEWIRE
#define SERVER_SETUP_H_ONEWIRE

#include "lwip/pbuf.h"
#include "lwip/tcp.h"

#define TEST_TCP_SERVER_IP "192.168.1.28"
#define BUF_SIZE 64
#define TCP_PORT 4242

typedef struct TCP_CLIENT_T_ {
    struct tcp_pcb *tcp_pcb;
    ip_addr_t remote_addr;
    uint8_t buffer[BUF_SIZE];
    int buffer_len;
    int sent_len;
    bool send_complete;
    int run_count;
    bool connected;
} TCP_CLIENT_T;

static err_t tcp_client_sent(void *arg, struct tcp_pcb *tpcb, u16_t len);
err_t tcp_client_recv(void *arg, struct tcp_pcb *tpcb, struct pbuf *p, err_t err);
static err_t tcp_client_poll(void *arg, struct tcp_pcb *tpcb);
static void tcp_client_err(void *arg, err_t err);
static err_t tcp_client_close(void *arg);
static err_t tcp_client_connected(void *arg, struct tcp_pcb *tpcb, err_t err);
static bool tcp_client_open(void *arg);

err_t tcp_client_send_data(void *arg, struct tcp_pcb *tpcb, uint32_t temp);

#endif
```c

I include this in my main file

My bad my words don't make sense.

What I mean is that this code is in a header file.

And I include that header file in the main file.

#include "server_setup.h"

[blazing holly]

How long have you done c?

[whole jolt]

Like 4 months.

[blazing holly]

do you know you can just do

[whole jolt]

Yes but is there an advantage to that?

[blazing holly]

yes, readability

[whole jolt]

That's definitely cleaner but why would the people making the examples write it like that.

[blazing holly]

like tcp_sent force you to use a certain method signature

but that is just for the callback

the rest of your code does not need to suffer for it.

They think everything need to go in the heap...

[whole jolt]

Heap lovers. I'm actually going to try what you suggested. I want to follow up on this.

[blazing holly]

like, this is actually evil

a pointer on void....

just an adress on anything

so unclear it hurt

Ok, I want you to do something else first

create a python client

than send the same data as that c client

to your python server

first check that this work as intended.

Like, we will be able to just remove that part of the problem.

[whole jolt]

I'm gonna go to class. I want to follow up on this. I'll ping u.

[blazing holly]

ok

[blazing holly]

I was misremembering from my c++ time, in c & parameter do not exist. only *,

#include <stdio.h>

struct Client
{
  char* name;
};

void printClient(struct Client* client){
  puts(client->name);
}

int main() {
  struct Client client = {"Bobby"};
  printClient(&client);
  return 0;
}

#include <stdio.h>

struct Client {
  char *name;
};

void printClient(struct Client *client) { puts(client->name); }

int main() {
  struct Client clients[] = {
      {"Bobby"},
      {"Sam"},
      {"Paul"},
  };

  int numberClient = sizeof(clients) / sizeof(struct Client);

  for (size_t i = 0; i < numberClient; i++) {
    struct Client client = clients[i];    
    printClient(&client);
  }

  return 0;
}```

[whole jolt]

Hey so I managed to get the Python client working. I had to include a while loop that continues sending receive requests.

[blazing holly]

Ok, so the python server should be good.

[whole jolt]

Now, when I connect it to my micro controller, it does receive more of the data but it backlogs for some reason and still crashes at around 1.4k bytes sending.

Sometimes it receives empty bits but that's to be expected.

[blazing holly]

It there just no more memory?

[whole jolt]

I'm not sure tbh.

[blazing holly]

do you create buffer without cleaning them?

what exactly is multicore_fifo_pop_blocking

when you compile it, what do you use?

[whole jolt]

So for multi-threading, it has this library package that enables a queue for communication between threads.

[blazing holly]

ok, so you create a queue to exchange data pio_can_add_program

does that queue have a max size?

are the item evicted if you push more the the max?

[whole jolt]

Yes so when I call multicore_fifo_push_blocking(uint32_t) it pushes the int to the queue of the other core.

[blazing holly]

where do you set the max size?

[whole jolt]

Each core has its own queue that obtains data from the other core that is pushes.

I don't know. I think it's just set in stone.

The thing is that everytime I call multicore_fifo_pop_blocking() it pops the queue so it shouldn't overflow.

[blazing holly]

the queue fill faster than you empty it, maybe

can you log the size of that queue

[whole jolt]

Lemme see

[blazing holly]

Just to reiterate, the c loop send multiple data point now, then stop after a while?

[whole jolt]

No. So I changed the python server by adding this

#client handling thread
def handle_client(client_socket): 

    while True:
        request = client_socket.recv(64) 
        print(f"[+] Recieved: {request}")

        temperature_b = request[:4]
        temperature = struct.unpack("<I", temperature_b)
        print(f"The temperature is: {(temperature[0])/16.0} degrees Celsius!", flush=True)
    #sending back the packet 
    #client_socket.send("Ping recevied".encode()) 
    #client_socket.close()

Now it's able to receive multiple packages and it runs it.

The FIFO is 32 bits wide.

Thing is that the queue is a blocking function.

It won't let it overload.

Nor will it let it continue running until the queue has been emptied.

Which is why I can run it forever if I take away the server stuff.

[blazing holly]

OK, so the queue is only 32 items

[whole jolt]

No. 32 bits which means only one item.

Since it stores a uint32_t

[blazing holly]

https://datasheets.raspberrypi.com/rp2350/rp2350-datasheet.pdf

[whole jolt]

Well this is what it says in the documentation

"Each FIFO is 32 bits wide, and 8 entries deep on the RP2040, and 4 entries deep on the RP2350. "

[whole jolt]

I'm on a rp2350

Oh

[whole jolt]

I actually can't interpret this.

Right right. It basically locks the program.

I don't think it's the multithreading FIFO Queue. Because I can make it bounce data between the two cores forever.

[blazing holly]

Yeah it is not it.

[whole jolt]

Do you know python like that?

[blazing holly]

I did more python than c

in micropython it is pretty easy

[whole jolt]

Because I want to see how it works if I basically add a condition that if it reads a byte array b' '

That it won't send a receive request

Or something

[whole jolt]

Yeah I know. I've seen so many tutorials about how to make a webserver for the application I'm making lol

[blazing holly]

what I like to do in c and c++ socket, is to write as the first byte the size of the rest of the data

then you can just read the first byte in any other lang

then read that size next

[whole jolt]

The size in bytes?

Or in numbers ?

That would make it easier lol

Now it doesn't crash at all

Only when it receives an empty data packet

So I should just put an if statement or something lol

Oh nevermind

It still crashes for some reason

Wait you said that I had to clean the buffer right?

Would I have to clean the buffer in python as well?

[blazing holly]

in bytes

[blazing holly]

no