networks

[wicked ocean]

how do i redirect incoming/outgoing data between users and django

[cedar forum]

you need to configure a reverse proxy rule in caddy

[wicked ocean]

        reverse_proxy * 192.168.54.51:8000
}```

this is on the caddy server, but requests doesn't seem to reach at all

from django to caddy using curl seems fine

[cedar forum]

hmmmmm

[wicked ocean]

but from caddy to global gives "too long" response

[cedar forum]

my prod caddyfile used to look like this in /etc/caddy/Caddyfile: ```caddyfile
*.jb3.dev {
tls {
dns cloudflare [...]
resolvers 1.1.1.1
}

    @site host site.jb3.dev
    handle @site {
            reverse_proxy localhost:8890
    }

    @anothersite host anothersite.jb3.dev
    handle @anothersite {
       reverse_proxy localhost:3000
    }

}

but i since moved away from caddy

[wicked ocean]

the issue is that its not localhost

its in the local network, on a different machine

[cedar forum]

that shouldn't affect things

if you are on the machine that caddy is on and curl to django does that go through?

[wicked ocean]

yes

[cedar forum]

hmmmmmm

anything in your caddy logs?

[wicked ocean]

its empty

it could be an issue between caddy and global then

[cedar forum]

hmmmm

maybe

[wicked ocean]

pinging the ip address from global gives no response if thats an indicator

[cedar forum]

that might be related

are you running on like a home network or something?

[wicked ocean]

trough ovpn, company network

they said they opened 80 and 443 only

[cedar forum]

ah, then it might make sense that ICMP doesn't go through

if you can tcpdump you could probably check if any traffic hits the caddy box at all

[wicked ocean]

whats tcpdump

[cedar forum]

also worth checking if the caddy box has a firewall up, ufw or whatever

https://www.tcpdump.org/

[wicked ocean]

ufw is disabled

[cedar forum]

it will dump all packets received by a machine to stdout

[wicked ocean]

should it be run from global

[cedar forum]

no, from caddy, and then try hit it from global

see if any traffic hits port 80

[wicked ocean]

what do i type

just tcpdump

[cedar forum]

you can filter it to a port

tcpdump -n port 80 probably

[wicked ocean]

nothing

[cedar forum]

hmmm

sounds like an issue between global and caddy then, all I can say is check firewall and check what your it says

[wicked ocean]

yes, ill talk to the server provider and see if they can find anything suspicious

thanks for your blazing fast response

[cedar forum]

👍

[amber garden]

Hello 👋

[prisma cobalt]

hi

time for some sockets help

[amber garden]

:)

Yeah

[prisma cobalt]

your interested in making a request to a website with sockets yes?

[amber garden]

Apparently their much faster than normal requests

[amber garden]

Post request if that says anything

[prisma cobalt]

well sockets are the lowest level of internet communication

a post request? okay sure that works

what data do you want to send?

[amber garden]

Json type stuff

Like

[prisma cobalt]

do you know sockets already yet?

[amber garden]

Not really

[amber garden]

{"Authorization": ""}

[cedar forum]

don't go down this route

[prisma cobalt]

import socket

client = socket.socket()
client.connect(("localhost", 8000))

client.send("POST example HTTP/1.1".encode())
client.close()

[cedar forum]

you are going to get minimal performance gains out of using sockets than using something like requests/httpx/aiohttp, if you want low level use urllib, do not implement your own HTTP

[amber garden]

How come?

[prisma cobalt]

me implementing my own HTTP right now: 😳

[cedar forum]

you are going to miss 99% of HTTP standard, you won't be using HTTP/2 which gives huge speed boosts

[amber garden]

Hm ok

the main thing i want is speed

no like reading the websites data n shit

[cedar forum]

aiohttp, requests, urllib, httpx are all your friends

[amber garden]

Their fast?

I heard requests is slow asf

[prisma cobalt]

good midnight btw Joe

[cedar forum]

requests does a lot of stuff for you, if you want low level then aiohttp or httpx are pretty speedy

[amber garden]

But would requests be faster 🤔

I mean

socks

[cedar forum]

not necessarily, no

[amber garden]

This chart saying requests is fastest?

[cedar forum]

no it's not

it's saying pycurl is fastest

the metric is requests/second, larger is better

[amber garden]

Oh i see now

[cedar forum]

if you want a lot of speed then pycurl is your best bet http://pycurl.io/

[amber garden]

Is it super difficult or something I could probably do

[cedar forum]

much much much easier than sockets

[amber garden]

oh nice ok

[cedar forum]

anyone that builds their own HTTP is going to have a very hard time supporting all the different standards, it's just not something you do

[prisma cobalt]

😳

[amber garden]

Is there any examples for pycurl i can take a look at

[cedar forum]

http://pycurl.io/docs/latest/quickstart.html

[amber garden]

oh yes i just came across that

[cedar forum]

what are you actually building here

how sure are you that you even need this performance

even with pycurl you're going to have a significantly more painful time than just using httpx/aiohttp, if you want concurrency then throw some async in there as well

[amber garden]

100%

speed is key

gotta be the first one there

[cedar forum]

are you going to be making more than 600 requests per second

[amber garden]

no

2 requests

but they need to be fast as possible

[cedar forum]

the speed of httpx, aiohttp, pycurl and sockets will be indistinguishable at 2 requests

[amber garden]

Ah ok

[cedar forum]

you don't need as much throughput as you think you do, and at your scale 2 requests is going to be bottlenecked at the server response time and your network, not at your http client

[amber garden]

I mean that makes things easier then ig lol

Yeah I just realised that I can just account for delays by changing the times i send the reqeusts

Thanks for the help anyway guys

Ill probably try a few options and see what is best for me :)

[short crescent]

Is there a framework like FastAPI but for processes on the same machine? Most of my experience is with “write a script to do X”, and I recently got into webdev with FastAPI. Let’s say I’m building a game where the ‘world’ is created like a back-end/database endpoint. Want to know what you can see? Query the ‘World’. But if I’m only playing locally, is there a more efficient way to pass messages between agents (the human players and AI players) and the World than sending messaged over localhost:6789 ?

Some google-fu turned up… Redis.

[steady sage]

How to run arp command through python and print output

[steady horizon]

If you have some kind of a terminal command for that then you can use subprocess

[ember ledge]

ok so how can i make in python voice text to speech with my own voice

[frigid ravine]

I need help with dns spoofing on macos

Can anyone help?

[opaque jacinth]

so i am making a discord bot without voice so i would like to ask other peoples opinion on which is currently the best discord.py based wrapper

[steady horizon]

py-cord

[thorny cobalt]

Guys can anyone help me to do a project on computer network pls

I have no idea what should I do pls guys help me

[thorn stratus]

that's very vague

but usual first project with networking is like a simple chat program

[twin heath]

is this where you would ask about things like udp

[thorn stratus]

well udp is a network protocol so that would fit the description of the channel yes

[twin heath]

fair

I am trying to implement congestion control into this project I am doing but cant seem to get close

Is there a resource you recommend?

its in python

specifically this

[thorny cobalt]

Can u give me that code @twin heath

[thorn stratus]

looking at how tcp does it would probably be your best bet

[heady orchid]

I have something stuck in my mind with the socket program, maybe it happened as a result of my carelessness, but I want to ask. socket training videos always use the threads on the back of the modem, such as 192.168.1.150 . How will someone on another network join my server?

[clear pasture]

yall im havin this problems for some time now, i installed scapy by following its document and did everything correctly, but it says that Ether or anything related to Scapy even IP is undefined, any fixes?

[steady horizon]

Read about port-forwarding

[ember ledge]

So, I have a small question regarding a scenario I have right now on a project I am working on.

I am using flask-socketio to create a connection between the user and the server, and if my understanding is correct, each user spawns a socket "worker" of their own

concurrently, the server is also connecting to a different server as well using sockets

in short, the connections basically go client <-> server1 <-> server2

What I'm worried about, is that where as client <-> server1 creates as many workers needed per client, server1 <-> server2 will create (I believe) only one connection, and could possibly "overload" the worker if too many clients send data to server1, which will then send it to server2 over a single worker

Would it be possible (and/or recommended) to create multiple workers per user between server1 and server2, or am I talking nonsense?

[thorn stratus]

Probably just your IDE not recognizing it properly, I bet it still runs

[thorn stratus]

Home routers generally use something called NAT to connect the LAN to the WAN. This means all the devices on the LAN are sharing the same public IP address. Therefore, for someone to be able to connect to you, you need to tell your router which device on the LAN to forward packets to specific ports to.

[clear pasture]

any solutions?

[thorn stratus]

idk, does it run?

You need to troubleshoot the issue

[heady orchid]

yes i understood it, i let the device working in local to broadcast with port forwarding and it worked!

[quiet pine]

i'm trying to get an image from bytes in pillow. the client takes a screenshot using pyautogui, which it then converts to bytes. it then gets the size of that string of bytes and sends both that and the actual image information through socket to the server. the server then takes the size that it just got and uses the int value of size as the buffer for receiving the image bytes (imgbytes as variable). then the server takes that information and executes Image.frombytes("RGB", (1920, 1080), imgbytes, "raw"). now, that line, it returns:

C:\Users\arvid\OneDrive\Skrivbord>python server.py
1702521203
Traceback (most recent call last):
File "C:\Users\arvid\OneDrive\Skrivbord\server.py", line 26, in <module>
displimg()
File "C:\Users\arvid\OneDrive\Skrivbord\server.py", line 16, in displimg
img = Image.frombytes("L", (1920, 1080), imgbytes, decoder_name="raw")
File "C:\Users\arvid\AppData\Local\Programs\Python\Python39\lib\site-packages\PIL\Image.py", line 2671, in frombytes
im.frombytes(data, decoder_name, args)
File "C:\Users\arvid\AppData\Local\Programs\Python\Python39\lib\site-packages\PIL\Image.py", line 800, in frombytes
raise ValueError("not enough image data")
ValueError: not enough image data

Why? Becuase i dont understand and i cant google my way to it.

[prisma cobalt]

Show all the code

From both the client and server

Well.. all the relavent code at least

[quiet pine]

well i just want to know what causes that error message to display.

[shut geyser]

I created a little multiplayer snake game with server/client. When i start the server locally everythings alright. But when i start the server on a VM get this error when i try to connect with the client: [Errno 10060] A Connection attempt failed because the connected party did not respond after a period of time, or established connection failed because connected host has failed to respnd.

[heady orchid]

My first online game

[hasty quest]

Your VM's network settings are probably improperly configured. Can you post the name of the hypervisor you use (e.g. virtualbox / vmware) and the name of the network type you have set up currently?

[crystal current]

https://sylvaindurand.org/use-tor-with-python/

[craggy hollow]

Hello, I want to get all the packets, a website sends me

print('Receiving')
print(connection.recv(1))
connection.close()```
But its just stuck at `connection.recv()`

how do I fix that?

I am talking about this stuffs

[candid wasp]

Hi, can someone help me with this: In flask you can return an image to a request with send_from_directory method, what is the equivalent of this in tornado?

[hasty quest]

TOR operates via the socks5 protocol

It functions with any client which supports socks5

[hasty quest]

Going from a pure TCP socket to a full HTTP exchange is a bit hard.

[errant bay]

Hey @hasty quest!

Uh-oh! It looks like your message got zapped by our spam filter. We currently don't allow .txt attachments, so here are some tips to help you travel safely:

• If you attempted to send a message longer than 2000 characters, try shortening your message to fit within the character limit or use a pasting service (see below)

• If you tried to show someone your code, you can use codeblocks
(run !code-blocks in #bot-commands for more information) or use a pasting service like:

https://paste.pythondiscord.com

[hasty quest]

https://www.cloudflare.com/img/learning/ddos/what-is-a-ddos-attack/osi-model-7-layers.svg

ffs discord

A TCP socket operates at layer 4 of this chart

HTTP is layer 7

A TCP socket just facilitates very basic data exchange

HTTP is the protocol which governs what that data exchange is used for in the context of a webpage

[craggy hollow]

so this socket library doesnt work?

[hasty quest]

It does, you just haven't communicated properly.

You've opened a TCP connection to the server. the server then expects an HTTP exchange to occur, which you haven't created.

Network protocols are built on top of eachother like matryoskha dolls.

HTTP is built on top of TCP. The socket only handles TCP.

If you're not using an HTTP library, then handling HTTP is up to you.

If you want to mess with raw HTTP on a TCP socket, that's fine ofc

But you will have to look into how HTTP works in order to do it.

If you connect to the server and send a properly formatted HTTP request, it will respond with an HTTP response.

This response might contain information about other pages for you to request.

[craggy hollow]

well, my website is https, does that work the same way?

and the website sends packets to me, once I connect, do I still need to request them?

[hasty quest]

HTTPS is HTTP on top of SSL or TLS on top of TCP.

You still need to establish a connection to the service over TCP, but with the added step of establishing a TLS session. The HTTP exchange then happens over this session, instead of over bare TCP.

"Packets" actually aren't even a thing at the TCP layer or higher

[craggy hollow]

how are they called instead?

[hasty quest]

Usually a "packet" refers to a piece of data at layer 3

Ethernet operates on "frames"
IP operates on "packets"
TCP operates on "segments"
HTTP operates on "requests" and "responses"

A single request could be broken up into multiple segments and packets

[crystal current]

Are you asking what these are called? Those are files

[hasty quest]

But HTTP doesn't need to know or care about that.

All HTTP needs to deal with is requests and responses. It doesn't need to know or care what the lower parts of the network stack are like

[hasty quest]

I think they're referring to the request chain

That's something they'll encounter as part of HTTP, but better to start with the basics before dealing with a complicated exchange like that

[craggy hollow]

so, the response is the thing, I'm looking for?

[hasty quest]

Requests and responses, yes

[craggy hollow]

but there's no request I have to send, for the server to send me a response

it just does that when connecting

[hasty quest]

Then it's not HTTP

that's not how HTTP works.

[craggy hollow]

its a normal website like youtube.com

[hasty quest]

When I open a TCP connection to youtube.com, nothing is sent from the server until I issue a request.

So if your server is sending something in response to the bare connection, it's not like youtube.com

[unborn gazelle]

source: my ip address
destination: web

this is an "outbound" connection right?

[hasty quest]

Correct

An inbound connection comes from another IP towards you

Or more accurately (since you can have inbound connections on loopback) an inbound connection has you listening, and the other end contacting you

An outbound connection has someone else listening, you contact them

Usually this is across two different IP addresses but not always

[nimble breach]

I will send code

[sinful peak]

So what's the issue here, @nimble breach

thank you

[nimble breach]

Server

import socket
import threading

s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.bind(("10.0.0.155", 1234))
s.listen(5)

clients = []
usernames = []

def broadcast(message):
    for client in clients:
        client.send(message.encode("utf-8"))

def handle(client):
    while True:
        try:
            message = client.recv(1024)
            broadcast(message)
        except:
            try:
                index = clients.index(str(client))
                clients.remove(str(client))
                usernames.remove(index)
                client.close()
                break
            except:
                pass

print("Server is listening...")

while True:
    client, addr = s.accept()
    print(f"{addr} connected to the server")

    client.send("USER".encode("utf-8"))
    username = client.recv(1024).decode("utf-8")
    usernames.append(username)
    clients.append(client)

    print(f"{username}")
    broadcast(f"{username} has entered the server.")
    client.send("Connected to server.".encode("utf-8"))

    thread = threading.Thread(target=handle, args=(client))
    thread.start()

Client

import socket
import threading

username = input("Enter username: ")

s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.connect(("10.0.0.155", 1234))

def recieve():
    while True:
        try:
            message = s.recv(1024).decode("utf-8")
            if message == "NICK":
                s.send(username.encode("utf-8"))
        except:
            print("Fatal ERROR ocurred. Exiting...")
            s.close()
            break

def write():
    while True:
        message = f"{username}: {input()}"
        s.send(message.encode("utf-8"))

thread1 = threading.Thread(target=recieve)
thread2 = threading.Thread(target=write)

thread1.start()
thread2.start()

Original ERROR:

y", line 1009, in _bootstrap_inner
    self.run()
  File "C:\Users\USER\AppData\Local\Programs\Python\Python310\lib\threading.py", line 946, in run
    self._target(*self._args, **self._kwargs)TypeError: __main__.handle() argument after * must be an iterable, not socket

@sinful peak not sure what is wrong, but the broadcasting won't work either

broadcast function I mean

@sinful peak ?

[sinful peak]

Sorry I was afk a minute

Looking now

[nimble breach]

ok

[sinful peak]

@nimble breach

[nimble breach]

Yeah?

That is just a print

[sinful peak]

?

It's a question, in a way

I see no issue so far

so I need clarification

I have no error on your server code

it starts and listens

@nimble breach

[nimble breach]

but have you tried typing on the client

when it recieves a message from the client then it all breaks apart

[sinful peak]

For that I'd have to look over the whole code, set some stuff up

If you have specific errors I can explain them

the rest would take some time

[nimble breach]

Just those first 2 i stated

[nimble breach]

^

[sinful peak]

@nimble breach I have to be honest with you. I could solve this, but it would take some thinking. If nobody else helps you by tomorrow I'll do it 100%

[nimble breach]

ok

[sinful peak]

I'm just really tired and gambled it was a simple error

[nimble breach]

thanks for at least looking

[sinful peak]

Hopefully someone sees it before tomorrow and helps you out

[sinful peak]

Here's the code for anyone following

[dreamy talon]

@nimble breach I got a simple server working although Im sure its riddled with concurrency issues that can only be uncovered through thinking and testing. As well as the fact that users will be bottlenecked by the slowest client but regardless.

If you use netcat like so: nc localhost 25565 you can type your username and hit enter once you connect. Then do it again in another terminal to show 2 clients connecting! Then start sending messages and the other users will see them!!

import socket
import logging
import threading
from contextlib import contextmanager

logging.basicConfig(format='%(levelname)s:%(message)s', level=logging.DEBUG)
logger = logging.getLogger(__name__)

client_lock = threading.Lock()

@contextmanager
def server(hostname="localhost", port=25565, backlog=5):
    sock = None
    try:
        sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
        # Reuse port/addr even on bad crashes without waiting
        sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
        sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEPORT, 1)
        sock.bind((hostname, port))
        logger.debug(f"Binding IPV4 TCP server to {(hostname, port)}")
        sock.listen(backlog)
        yield sock
    except Exception:
        logger.exception("BAD THINGS RUN AWAY")
    finally:
        if sock:
            sock.close()


def broadcast_message(connected_clients, message):
    # Only broadcast to clients that have joined
    # We could lock the connected clients here but that would add contention
    # And we want to only notify clients that exist anyways
    logger.debug(f"Broadcasting Message: {message}")
    for other_client in connected_clients:
        other_client.sendall(message.encode("utf-8"))


def handle_client(client, connected_clients):
    with client_lock:
        connected_clients.add(client)

    try:
        client.sendall(b"USERNAME: ")
        username = client.recv(1024).decode("utf-8").strip()
        # I couldnt do a set difference on a socket but I could do set construction I dont know
        broadcast_message({c for c in connected_clients if c != client}, f"A wild {username} has appeared\n")

        while True:
            message = client.recv(1024).decode("utf-8").strip()
            # Only broadcast to clients that have joined
            # We could lock the connected clients here but that would add contention
            # And we want to only notify clients that exist anyways
            for other_client in {c for c in connected_clients if c != client}:
                other_client.sendall(f"{message}\n".encode("utf-8"))
    except:
        with client_lock:
            # Might have gotten a bad message before they appeared
            if client in connected_clients:
                connected_clients.remove(client)
    finally:
        # Give me my port back dangnabit
        client.close()

if __name__ == "__main__":
    with server() as s:
        connected_clients = set()
        try:
            while True:
                client, addr = s.accept()
                logger.debug(f"New Client Connection: client={client} with addr={addr}")
                thread = threading.Thread(target=handle_client, args=(client, connected_clients))
                thread.start()
        except KeyboardInterrupt:
            # Just some cleanup on a SIGINT
            # I could potentially miss cleaning something up here if one is added while im doing this
            # but im lazy
            for client in connected_clients:
                try:
                    client.close()
                except:
                    pass

[nimble breach]

could you please explain what logging and contextlib do? I am not familiar with either of those modules. But thank you for demonstrating another way to make a server.

[nimble breach]

@dreamy talon Could you show me how you would use netcat and also please answer my questions that I asked before

[pearl anchor]

hey guys, how can i generate and utilise session ids

im using flask and react frameworks, so if i log in a user from the backend, generate a session id, how do i authenticate through the frontend, do i just store it on the frontend? also do i authenticate every request from the front to the back?

[steady sage]

How to get mac address from which my device is connected through python code, but doesn't display mac of every device which are connected like arp does

[prisma kiln]

i have a question do socket server run forever in replit? or is there some time it will be expired?

[steady horizon]

When you exit the tab it closes

After a few minutes or instantly

[prisma kiln]

oh ok

[dreamy talon]

Logging is well for logging. Which is a thing software developers use to see nice clean status logs from a program usually.

I wouldnt worry about contextlib but it allows you to make things thafs use that with syntax among other things.

[nimble breach]

oh ok, thank you

[dreamy talon]

I showed you the netcat command above
nc localhost 25565 for my example

[nimble breach]

I figured out to use and install netcat

[dreamy talon]

That opens a tcp connection to the server!!

[nimble breach]

yeah

[dreamy talon]

It helps a lot for testing

So you dont have to fiddle with your own client until the server is nice and done

[nimble breach]

Your code example actually made me laugh lmao

[dreamy talon]

Haha

As long as I bring some sort of cheer

If you have anymore networking questions just ask in the server and i can respond when I have time. I havent written multithreaded networking code in a while but i can sure as heck write some python

[nimble breach]

Thank you.

[limber meadow]

😩

[pearl solar]

Hello, i have a yaml file with this info , I need to grab the 1 and 3 line and print it on the screen, can someone please give me a hint? thanks.

 server1:
       host: test1
       ip: 192.168.10.10
 server2:
       host: test2
       ip: 192.168.10.15

[shut geyser]

I have made a little Multiplayer Snake Game where i convert the data to json and then to bytes which i send over a socket to the client or server but when i e.g. pick up an apple or die i get an jsonDecodeError from the line where i convert the bytes to json and then do the data. The Error Line: data = data.decode("utf-8") data = json.loads(data)

The Error:

    self._target(*self._args, **self._kwargs)
  File "C:\Programmieren\Python\Games\SnakeServerOneFile\Game.py", line 93, in run
    self.packetPool.handlePacket(data)
  File "C:\Programmieren\Python\Games\SnakeServerOneFile\Game.py", line 118, in handlePacket
    data = json.loads(data)
  File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.9_3.9.2544.0_x64__qbz5n2kfra8p0\lib\json\__init__.py", line 346, in loads
    return _default_decoder.decode(s)
  File "C:\Program Files\WindowsApps\PythonSoftwareFoundation.Python.3.9_3.9.2544.0_x64__qbz5n2kfra8p0\lib\json\decoder.py", line 340, in decode
    raise JSONDecodeError("Extra data", s, end)
json.decoder.JSONDecodeError: Extra data: line 1 column 54 (char 53)

[valid plaza]

Quick question, does anyone know of a module/ script which allows you to run a mini server broadcasting your status to the network so another computer knows if you are up (so they can make a sockets connection) , sorry I cant seem to articulate it well enough to look it up. Many Thanks

[gilded thistle]

have you tried the "socket" module ?

in the pins of this channel there are nice examples for that module

[dreamy talon]

You can just use port scanning. Netcat can do that really easily

[thorny cobalt]

How gps is related to networking ?

[ember ledge]

Anyone wanna help me with classb subnetting

[nimble breach]

@dreamy talon think you could tell me what is wrong with this? I am making an online game. It is recieving the same thing it is putting out

Server:

import socket
import threading

s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.bind(("127.0.0.1", 1234))
s.listen(5)

clients = []
nicknames = []

def broadcast(message, sender):
    for client in clients:
        if client is not sender:
            client.send(message.encode("utf-8"))

def recieve(client):
    while True:
        try:
            msg_recv = client.recv(2048).decode("utf-8")
            broadcast(msg_recv, client)
        except:
            try:
                index = clients.index(client)
                clients.remove(client)
            except:
                pass
            client.close()
            try:
                nickname = nicknames[index]
            except:
                pass
            #broadcast(f"({nickname}) left the server.", client)
            print(f"({nickname}) left the server.")
            try:
                nicknames.remove(nickname)
            except:
                pass
            break

def handle():
    while True:
        client, addr = s.accept()
        msg_recv = client.recv(2048).decode("utf-8")

        clients.append(client)

        # Append nicknames and clients, broadcast messages
        if msg_recv[0] == "N":
            nicknames.append(msg_recv[1:len(msg_recv)])
            #broadcast(f"({nicknames[len(nicknames)-1]}) joined", client)
            print(f"({nicknames[len(nicknames)-1]}) joined")
        else:
            broadcast("C" + msg_recv, client)

        thread = threading.Thread(target=recieve, args=(client,))
        thread.start()


thread1 = threading.Thread(target=handle)
thread1.start()

Client:

import socket
import threading

class Networking:
    def __init__(self, ip, port, nickname):
        self.ip = ip
        self.port = port
        self.nickname = "N" + nickname

        self.send_coords = ()
        self.rect_coords = ()

    def init(self):
            self.s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
            self.s.connect((self.ip, self.port))

            thread = threading.Thread(target=self.send)
            thread.start()

    def handle(self):
        while True:
            msg = self.s.recv(2048).decode("utf-8")
            if msg[0] == "C":
                self.rect_coords = tuple(f"({msg})")

    def send(self):
        while True:
            # Send messages
            try:
                self.s.send(f"C{str(self.send_coords)}".encode("utf-8"))
            except:
                print("Connection to server lost.")
                self.s.close()

            thread = threading.Thread(target=self.handle)
            thread.start()
    
    def send_nickname(self):
            self.s.send(f"{self.nickname}".encode("utf-8"))
            self.nickname = self.nickname[1:len(self.nickname)]

Actual game (you shouldn't need this)

from ursina import *
from Network_Files import game_client

nickname = input("Enter username: ")

app = Ursina()
window.borderless = False

# Networking
client = game_client.Networking("127.0.0.1", 1234, nickname)
client.init()

# Rect
rect1 = Entity(
    model = "cube",
    scale = (1, 1, 0.1)
)

rect = Entity(
    model = "cube",
    scale = (1, 1, 0.1)
)

speed = 0.01

# Gameloop
def update():
    # Keybinds
    if held_keys["w"]:
        rect1.y += speed
    if held_keys["a"]:
        rect1.x -= speed
    if held_keys["s"]:
        rect1.y -= speed
    if held_keys["d"]:
        rect1.x += speed
    
    # Send Coords
    client.send_coords = rect1.position

    # Recieve Coords
    rect.position = client.rect_coords
    
    print(f"Send: {client.send_coords} Recv: {client.send_coords}")

app.run()

[whole badger]

Hi i'm trying to use a dos script but it says that pcap/wincap isn't found any advice to install it and add it to path ?

[toxic mural]

@nimble breach wild guess, this line is wrong:

        if client is not sender:

it should be:

        if client != sender:

!identity

[errant bay]

Identity vs. Equality

Should I be using is or ==?

To check if two objects are equal, use the equality operator (==).

x = 5
if x == 5:
    print("x equals 5")
if x == 3:
    print("x equals 3")
# Prints 'x equals 5'

To check if two objects are actually the same thing in memory, use the identity comparison operator (is).

list_1 = [1, 2, 3]
list_2 = [1, 2, 3]
if list_1 is [1, 2, 3]:
    print("list_1 is list_2")
reference_to_list_1 = list_1
if list_1 is reference_to_list_1:
    print("list_1 is reference_to_list_1")
# Prints 'list_1 is reference_to_list_1'

[hasty quest]

Ye bruh

Just go to the terminal, and type in

"get tf on"

[clear pasture]

ive sent this here before but i really need help, i cant get scap to work in my IDE, works in terminal but not here. it says module missng (as u can see Ether() isnt found)

[sturdy hedge]

What distro r u using

[thorn stratus]

ARe you sure your IDE is using the right python install

[whole badger]

lol

[heady orchid]

Manjaro

[night ermine]

Hey, I'm trying to implement send logic over a slow lossy, constrained network.
For example some packets are lost during the sending, how are some ways I can resend or retrieve those lost packets?

[ember ledge]

TCP protocol knows about packet loss and resends them

[prisma cobalt]

TCP ensures packet loss
heh... what?
TCP ensures no data is lost

[ember ledge]

sorry, i was about to edit

not my main language xD

[night ermine]

sorry should have said it's simulate packet loss

[nimble breach]

For anyone who wants to help me ^^^^^^^^^^^^^^^^^^^^^^^^

[urban snow]

Might get some more responses if you take an open help channel btw

[thorn stratus]

Did you see the response below your post?

[clear pasture]

yes it is

but still wont detect scapy

[oak meteor]

Do you guys have any books about python networking?

[errant prism]

can you use python for cybersecurity (like linux)

[thorn stratus]

have you tried googling "<IDE> doesn't recognize python package"

[nimble breach]

nobody helps in help channels

[thorn stratus]

what

look below your post lmao

they even pinged you

[nimble breach]

how do i see my post

i dont remember it

[thorn stratus]

just scroll up

[nimble breach]

i dont know my post

[nimble breach]

which one is it

[thorn stratus]

the one where the guy pinged you.......

[nimble breach]

i dont have a noti

[nimble breach]

can you tell me which one, since you saw it, you should know

[gilded thistle]

If your bandwidth is like 30kbs, are you unable to receive network packets which are larger than that ? Or am I misunderstanding the whole concept of bandwidth

[deft sonnet]

Here is a quick picture of my code. Main Goals:
-Send 100 of the same file
-Receive it
-Rename it
-Calculate receive and send times
-Average the receive and send times
-Check transfer errors with reference file

Still working on changing it to work with any file as apposed to just a text file. I figure out how many... packets? frames?... to send is by diving the whole raw bites length by 1024 and then sending that many packet/frames. I have got to believe there is a better way. Lmk if anyone has a better idea! Thanks!

[ember ledge]

Packets are limited by MTU

Normally 1500 bytes

[ember ledge]

Skys the limit

[cedar forum]

yeah as Noobian says it just means that you can receive 30kb worth of packets in a second

1.5kb is the normal maximum unit, because an ethernet frame is 1500 bytes + 18 bytes which are overhead

but TCP starts out by sending 10 packets, so 14kb, and then it sees how fast they get accepted, after a few get accepted it'll start increasing the rate that it sends packets

then, once it starts reaching the cap and hitting congestion packets will get lost, so it slows down the rate that packets are being sent at

it increases the number of "in flight" packets, which are the number of packets currently going down the wire to the other machine

so transmission actually looks like this

[faint cairn]

im a little confused about how websockets work in quart. I want to make a chatting application that uses websockets where my frontend made with svelte will connect to a python backend made with quart

how would i go about sending data to a particular client using websockets

theres only a websocket submodule which i can use to websocket.send() or websocket.receive() to the client who sent the message but not to the recipient

[frozen drum]

you need to keep a list of recipients and their specific websockets so you can send to specific destinations

which also means your chat protocol needs the recipient (or some sort of channel) as part of it

[vagrant glade]

Hi any network administrators here?

Or anyone who's good at VLSM?

I just need somebody to check whether this VLSM subnetting is correcct?

With the main IP 40.30.20.0/22

[faint cairn]

i use websocket._get_current_object() to store the websocket when it connects but when i try to send data to it when another websocket connects it doesnt work

[faint cairn]

nvm im stupid

i didnt realise that when the websocket function finishes the websocket is closed

[plucky bough]

I have a question.
Can we do network tunneling in browser?
Like I have server running at port 5000 in local system. Can I tunnel it for making it accessible over the internet by only using browser?

[frozen drum]

technically possible by having a proxy application running in your browser yes, but there has to be a server on the other end accepting the incoming connections and proxying through your browser.

[torn river]

does anyone know how I would use an api that has 2 endpoints

[lusty ermine]

yes @torn river

Yes I know.

[ornate delta]

hi

[steady sage]

How can i extract dhcp data using python

What protocol should i follow

[heady belfry]

hello all

anyone want to work on a project?

[cedar forum]

scapy is your friend!!

https://scapy.readthedocs.io/en/latest/api/scapy.layers.dhcp.html

https://jcutrer.com/python/scapy-dhcp-listener this has a nice example

[hasty quest]

Dynamic Host Configuration Protocol

[exotic anchor]

Hey all..Complete python n00b here....trying to learn netmiko. I got netmiko to log into a switch and execute commands....but when giving it credentials I need it to read a CSV with 300 unique hostnames & passwords of cisco 9300's.....any advice? I've played with CSV reader I'm just not sure how to make the two talk to eachother

Something like this

import csv
from netmiko import ConnectHandler # Shouldn't be indented

hostnames = []
localpassword = []
platform = 'cisco_ios'
username = 'myusername'

password = 'password'

with open('creds.csv') as csvfile:
readCSV = csv.reader(csvfile, delimiter=',')

for row in readCSV:
    hostnames.append(row[1])
    localpassword.append(row[2])

for i in range(len(hostnames)):
    device = ConnectHandler(device_type=platform, hostname=hostnames[i], username=username, password=localpassword)

    config_commands = ['show ip int br'.format(hostnames[i])]
    print(config_commands) 
    print(device.send_config_set(config_commands))
    print()
    print(device.send_command('show switch'))

    device.disconnect()

That's what I pieced together after some googling but it's not happy

[steady sage]

I want to do it with pyshark

[lethal tusk]

it looks correct. is not connecting?

[ember ledge]

I wish to implement the TCP protocol myself for the sake of learning. Where can I get started?

[steady sage]

how to extract mac nad ip from dhcp packets using pyshark

[exotic anchor]

It connects to the switch fine if I specify a single set of credentials like this:
from netmiko import ConnectHandler
import csv

cisco_881 = {
'device_type': 'cisco_ios',
'host': 'myhostname.com',
'username': 'local-tech',
'password': 'mypassword',
'port': 22, # optional, defaults to 22
'secret': 'secret', # optional, defaults to ''
'session_log': 'netmiko_1.csv',
}

Establish an SSH connection to the device by passing in the device dictionary.

net_connect = ConnectHandler(**cisco_881)

Execute show commands.

output = net_connect.send_command('sh switch')
print(output)
output = net_connect.send_command('show mac address-table')
print(output)

But I need it to read a csv of hostnames and passwords line by line

[pliant fjord]

Hello, can we help me, plz) I have csv file, and I need parse him and than make class, based on the first argument of the file. For example: first args: 'car', that is mine, the class name is Car. Thank You for the reading!)

def get_car_list(a):
result = []
if a and a != []:
with open(a) as csv_fd:
reader = csv.reader(csv_fd, delimiter=';')
next(reader) # пропускаем заголовок
for row in reader:
c = 0
ocl = []
for r in row:
if r != '':
ocl.append(r)
c += 1
print(ocl)
if c == 5:
if len(ocl) == 5 and len(ocl[2]) > 2:
if ocl[0] == 'car': # brand, photo_file_name, carrying, passenger_seats_count
print(f"fff {ocl}")
car = Car(ocl[1], ocl[3], float(ocl[4]), ocl[2])
result.append(car)
print(f"fff {car}")

                    elif ocl[0] == 'truck' and len(ocl) == 5 and (('f4' in ocl[2] and len(ocl[2]) > 2) or ('png' in ocl[2] and len(ocl[2]) > 4) or (ocl[2] in 'jpeg' and len(ocl[2]) > 5)):  # brand,

photo_file_name, carrying, body_whl
try:
car = Truck(ocl[1], ocl[2], ocl[4], ocl[3])
result.append(car)
print(car)
except ValueError or IndexError:
pass

                    elif ocl[0] == 'spec_machine' and len(ocl) >= 5 and ocl[2][-2] == 'f4':
                        try:
                            car = SpecMachine(ocl[1], ocl[2], float(ocl[3]), ocl[4])
                            result.append(car)
                        except ValueError or IndexError:
                            pass
            elif c == 4 and len(ocl) == 4:
                if ocl[0] == 'truck' and len(ocl) == 4 and (('f4' in ocl[2] and len(ocl[2]) > 2) or ('png' in ocl[2] and len(ocl[2]) > 4) or (ocl[2] in 'jpeg' and len(ocl[2]) > 5)):  # brand, photo_file_name, carrying, body_whl
                    try:
                        car = Truck(ocl[1], ocl[2], float(ocl[3]))
                        result.append(car)

                    except ValueError or IndexError:
                        pass
else:

    return []
return result

The csv file:

car_type;brand;passenger_seats_count;photo_file_name;body_whl;carrying;extra
car;Nissan xTtrail;4;f1.jpeg;;2.5;
truck;Man;;f2.png;8x3x2.5;20;
truck;Man;;f2.png;;20;
car;Mazda 6;4;f3.jpeg;;2.5;
;;;
spec_machine;Hitachi;;f4;;1.2;Легкая техника для уборки снега

I am sorry, file have Russian words because I from Russia ( UTF-8)

And I have a problem, class instances for Car are not created.

I do not, why

[lusty field]

@exotic anchor you can use the csv dictreader function to get a line by line pull

[frozen drum]

you cannot use ```python
except ValueError or IndexError:

you have to change it to
```python
 except (ValueError, IndexError):

[steady sage]

What filter should i use to capture dhcp packets , bpf filter or display filter in pyshark

[lethal tusk]

bpf is faster. display is slower but less limited

[steady sage]

Idk why but when i capturing packet using bpf filter for dhcp or udp port 67,68. I'm not getting any packet. It shows 0 live packet

But when i entering arp in filter it is capturing packets

Display filter didn't tried yet

I tried too many times but not getting any packet

But if I'm using wireshark , that is capturing packets for dhcp

[gilded thistle]

i forgot to thank you for this, sorry

[cedar forum]

all good! glad it helped 👍

[steady sage]

How can i filter incoming dhcp packets so that it doesn't contain any duplicate entry

[ember ledge]

hi, is there a way to apply filter in wireshark based on the Info of a packet?

[wintry compass]

Hi, I have a project where I have to do network analysis using pcap, log, text files using python I didn't have done anything like this before I mean in networking can you suggest some packages , blogs ,articals and some guide to do this. kind of stuck

[cedar forum]

depends on the type of packet, but generally yes, you can filter down to application layer packet details

[cedar forum]

Scapy is pretty much the goto tool in this sort of thing. https://scapy.readthedocs.io/en/latest/usage.html#reading-pcap-files

You can load in a pcap and iterate through all packets, accessing the details in a Pythonic way.

[rain geyser]

hey

i want to connect to this wss://streamer.cryptocompare.com/v2 in python
and it connects in postman
but when i write my code it says getaddrinfo failed

HOST = 'wss://streamer.cryptocompare.com/v2
socket_client = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
socket_client.connect((HOST, 80))

[acoustic mural]

should it be port 443 instead of 80?

[cedar forum]

this isn't how websockets work

you need a websocket library, not raw sockets

https://websockets.readthedocs.io/en/stable/ or https://docs.aiohttp.org/en/stable/client_quickstart.html#websockets

[rain geyser]

wow thanks
can u give me an example of using this "wss://streamer.cryptocompare.com/v2?api_key={your_api_key}"
using https://websockets.readthedocs.io/en/stable/
i dont understand how i can set the header

[cedar forum]

what header are you referring to?

the api key?

[rain geyser]

only apikey

before you guide me, i didnt know websocket and socket are diffrent

[cedar forum]

should be just as you've got it there ```py
import asyncio
import websockets

api_key = "......"

async def connect():
async with websockets.connect(f"wss://streamer.cryptocompare.com/v2?api_key={api_key}") as websocket:
while True:
msg = await websocket.recv()
print(msg)

asyncio.run(connect())

[rain geyser]

ty so much

[cedar forum]

👍

[ripe coyote]

i'm scraping urban dictionary

here's my code :

query=input("Enter A Word : ")
data=requests.post(f"https://urbandictionary.com/define.php?term={query}").content
soup=BeautifulSoup(data,"html.parser")
for i in soup.find_all("div",class_="meaning"):
   print(i)
and i'm getting :

<div class="meaning">A person who writes <a class="autolink" href="/define.php?term=funny" onclick="ga('send', 'event', 'Autolink', 'Click', &quot;funny&quot;);">funny</a> and motivational <a class="autolink" href="/define.php?term=posts" onclick="ga('send', 'event', 'Autolink', 'Click', &quot;posts&quot;);">posts</a> on <a class="autolink" href="/define.php?term=Facebook" onclick="ga('send', 'event', 'Autolink', 'Click', &quot;Facebook&quot;);">Facebook</a></div>

how to get inner html of div and anchor tags?

[thorny talon]

Does while True: data = requests.get(...) create new connection for every request?

[unreal sail]

Hi everyone. I’m trying to solve with the best solution to this problem. I want to check running status of a service in three networked windows 10 machine. This three machine like this. (a,b,c).
Computer a is master and service at first running only at this machine and b, c computer are slave that only use when machine a service was down or stopped. I develop a python program that had multiple socket connection to the machine but that wasn’t good and optimal. We want all machines can see and check the service if one of them stopped the service other only one machine starting that service.
Who can help me to solve this? Thanks a lot.

Anyone have idea?

[ember ledge]

how i update the GET string in my web server?

[toxic mural]

parse it (see urllib.parse), change the parsed data structure, serialize it again

[ember ledge]

like this?

[toxic mural]

well in the linux world you would use something like corosync & pacemaker for this. I'm not aware of software like that being available for windows... what was wrong with your python program?

[toxic mural]

yes

[ember ledge]

ok

thx

[ember ledge]

Could you explain better, I didn't quite understand how I'm going to update

[toxic mural]

for example:

>>> import urllib.parse
>>> query = 'search=true&language=en_GB'
>>> urllib.parse.parse_qs(query)
{'search': ['true'], 'language': ['en_GB']}
>>> parsed = _
>>> parsed['language'] = ['en_US']
>>> urllib.parse.urlencode(parsed, doseq=True)
'search=true&language=en_US'

see the docs on parse_qs:

https://docs.python.org/3/library/urllib.parse.html#urllib.parse.parse_qs

[ember ledge]

thankyou for this

[ember ledge]

how do I add a proxy to aiohttp?

[cedar forum]

https://docs.aiohttp.org/en/stable/client_advanced.html#proxy-support

[ember ledge]

thanks

[vale tapir]

np

[elder cobalt]

Having a sort of confusion here, is there anyway to implement scapy along with multiprocessing? 🤔 been figuring out for a day but couldn't get any lead, basically I am trying to build packets and assign it to a variable and then using sendp(var,loop=1) but I want to run this in 5 different instances of the same script instead of manually opening command prompt 5 times

I tried making a loop instead without using scapy's implementation but then for some reason when I loop it for maybe 10-20 times for testing it hampers my PC a-lot with multiprocessing at the start but then slowly restores back to normal which I am not sure why but with scapy's implementation of the loop if I use it once it's fine but it's not what I wanted when I run the same loop 5 times together but that'd be mundane opening 5 different or maybe X amount of command prompts just to create and send a packet on loop...

Any help regarding this would be great as I've tried various different approaches but can't seem to figure it correctly on how to open let's say any user defined amount of command prompts running the same script instead of manually opening each of them and then running them one by one

[ember ledge]

I tried it and I get an exception with this:

except aiohttp.client_exceptions.ClientProxyConnectionError as e:

and I get:

405, message='Method Not Allowed', url=URL('http://ip:port')

am I doing something wrong or is it just the proxy?

[upbeat cedar]

hey fellas

im trying to send http requests

in an optimized fashion

using http.client

but im havin issues

[rugged plinth]

bruh i got a bsod smh...

[wind oriole]

Multiprocessing a crawler creates the question of how the individual crawlers talk to each other about things like which pages they've already visited. There are solutions, redis is a popular choice. But really if you're to the point where you need to multiprocess Scrapy, you might want to consider building your own. I don't know your use case but if you're after speed or scale, you'll be able to do more with 1 asynchronous crawler than with 4 Scrapy ones. Aiohttp will get you into the millions of pages/day on a single thread.

[elder cobalt]

Does aiohttp work with sending udp based packets? 🤔

Since my use case is along sending udp/tcp packets to test it out on several other devices on my network

With different layers of packet building with them but I can't really find a way to automate this whole process 🙁

[cedar forum]

It does not, HTTP is always on TCP

well, sometimes you do it over QUIC which is UDP, but HTTP is far too high for UDP

you need sockets for UDP

[elder cobalt]

Hmm I see, I tried looking over sendpfast functions which are there in scapy to maybe speed up the process of sending packets in a loop but it looks like you need tcpreplay preinstalled as well for it to actually work and I tried installing the win32 version which was very old but for some reason the libnet-latest.dll which comes along with it does not work properly

[ember ledge]

?

[fallen canyon]

can someone help me why am I getting this error?

[wind oriole]

well you call data a string on line 95

and then try to treat it as a dictionary object

[sacred berry]

error says what is wrong

you are mutating str

[frozen iris]

If I have received or sent one (the closure is not finished yet) closure frame to a websocket, am I allowed to send a normal frame between that and closing the TCP socket?

I have a task running in the background and there is a very small chance that this might happen so I am mostly wondering whether I should take precautions to prevent this.

[steady sage]

#help-avocado

[hasty zodiac]

can i know what device is behind a NAT, or am i "invisible" behind a NAT

[cedar forum]

invisible

[hasty zodiac]

ok

thanks

[cedar forum]

well

kind of

depending on the protocol you can sometimes work things out from things like TCP time stamps, or if it's a protocol like HTTP then the user agent gives away different devices too

[hasty zodiac]

ok but that was the reason of the question marks

[cedar forum]

so you can sometimes tell when two connections from a NAT'd IP are different devices, but it's generally harder to distinguish much about the device

[hasty zodiac]

i mean if i only make a tcp connection or smt like this

[ember ledge]

from aiohttp import ClientSession
import asyncio


async def headers(token):
    async with ClientSession() as client:
        async with client.get("link") as response:
            cookies = str(response.cookies)

    return {
        "User-Agent":
        "Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) discord/1.0.9001 Chrome/83.0.4103.122 Electron/9.3.5 Safari/537.36"
    }


async def start():
    headerss = await headers()
    async with ClientSession(headers=headerss) as client:
        async with client.get(
                "link"
        ) as response:
            print(response.text())


asyncio.get_event_loop().run_until_complete(start())

error:

RuntimeWarning: coroutine 'ClientResponse.text' was never awaited
  print(response.text())
RuntimeWarning: Enable tracemalloc to get the object allocation traceback

[frozen drum]

in case you are asking why this happens, you need

print(await response.text())

[ember ledge]

oh alright thanks

[tough sentinel]

I don't know where to put this question so I will put it here.

Does anyone here know how to setup Myrtille in docker properly?

I did research on it, though I didn't understand quite well.

I used these instructions in the Docker section. https://github.com/cedrozor/myrtille/blob/master/DOCUMENTATION.md#docker

[flint timber]

Is there a discussion here about C++ programming? I cant figure this out:

1. Create a program that can make choice’s base on MySQL Database Topic.
2. Make a database of your compound neighbor . manimum of 25 maximum of 30.

[lusty gazelle]

I don’t think so

[flint timber]

What about five examples of proper codings in Python ??

[lusty gazelle]

?

What do you mean

[lusty gazelle]

?

[flint timber]

Not really sure tbh, maybe any examples that are considered proper codings

I am legit confused

[scarlet jewel]

@flint timber This is a Python networking chatroom. You'll have better luck looking into a C++ specific server/forum.

[flint timber]

Ahh thank you my brother

[ember ledge]

lol i need 50 messages

[twin vault]

Hi, there...anybody familiar with matlab? I have few questions regarding mathematical expression to plotting in matlab for network protocol performance analysis.

[ember ledge]

Hi, this doesn't seem to have anything to do with python or networking, maybe check out our off-topic channels

[twin vault]

Okey. Thanks

[ember ledge]

does anybody know how to handle these exceptions? (proxy - aiohttp and asyncio)

[thorn stratus]

send a valid request lmao

[hasty quest]

400 - request wasn't formatted the way server expects
405 - request method is not valid for that page

[ember ledge]

But I just added proxies ._.

[tough sentinel]

Does anyone know how to use Redirections on Cloudron?

I am running Apache Guacamole and I need to be able to access it outside of my home network, I did the port foward but when I try to access it outside of my home network it says this "You are seeing this page because this domain points to Cloudron server IP but no app is installed on that domain."
Which then I found this forum which someone fixed - Here: https://forum.cloudron.io/topic/6023/you-are-seeing-this-page-because-this-domain-points-to-cloudron-server-ip-but-no-app-is-installed-on-that-domain

The person who gives the fix tells everyone to go to: https://docs.cloudron.io/apps/#redirections - I did the exact directions but it won't work for some reason.

[solemn mist]

looking for project ideas using sockets apart from chat application. any leads ?

[broken crypt]

I want to ask about Flask. I'm using Flask for REST API, can i run with https?

[steady sage]

#help-pineapple

[clever gazelle]

how can i connect to a socket i created with socket.create_server?

[ember ledge]

import socket
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)

# Connect to an IP with Port, could be a URL
sock.connect(('0.0.0.0', 8080)) # second argument should be whatever port your server is listening on 
# Send some data, this method can be called multiple times
sock.send("Twenty-five bytes to send")
# Receive up to 4096 bytes from a peer
sock.recv(4096)
# Close the socket connection, no more data transmission
sock.close()

[clever gazelle]

thanks

[proud swift]

You'll need a b" on that sock.send

[ember ledge]

i just copy pasted the first example on google cause i didn't want to write the comments myself

but yes @clever gazelle send() accepts object of type bytes

as an argument

[cedar dust]

Hello guys I have a network topology question, can anyone help me understand it?

How can I calculate theoretical bandwidth, latency and loss between two nodes?

[lusty field]

Programmatically? You could model this as a graph or tree. Then you would need something that could trace between two points in the model and sum the values.

Quick and dirty would be nested dictionaries. You could also do it a tabulated format and track connections explicitly.

[raw snow]

if anyone knows about combining a udp broadcast and the select function, i have a scenario in #help-dumpling

[frozen drum]

if you had a bigger topology, one option would be to model it as a graph and use dijkstra to compute shortest path between any two nodes you want to compute the figures for.

Then you can sum the latency from the source to the destination, multiply loss values for every subsequent point on the path, and use the minimum bandwidth measurement (accounting for latency)

[lusty field]

And youve invented ospf/ is-is :)

[cedar dust]

What does it mean to use the minimum bandwidth measurement accounting for latency?

[frozen drum]

If you need to calculate raw bandwidth of the underlying path, it does not matter. If you need to calculate bandwidth for tcp, latency matters due to having syn/ack

[misty zephyr]

https://youtu.be/nU5ZUNB7l9E

check

[dark trench]

Looking for some help on networking with sockets between a website and a robot on a raspberry pi, if you can help me out please check out the full question on https://stackoverflow.com/questions/70227410/getting-brokenpipeerror-errno-32-broken-pipe-when-sending-second-socket-msg

[frozen drum]

you are trying to send on a socket from a different thread, which is a no go.

I'm not sure if it would work, but you could try calling socket.accept() within the handle_client function to create the new socket from the new thread (instead of passing it as an argument)

[undone flint]

does anybody know of a simple way to chain proxies with requests?, ideally a library

so i wanna it to go like server > proxy1 > proxy2 > web

i have found this, but im a bit confused of how to implement it with requests, i havent dealt with raw sockets b4

https://github.com/nighthawkk/pyChainedProxy

[undone flint]

nvm i got it

[quartz tundra]

def create_socket():
    s = socket.socket()       
    print("Socket successfully created") 
    port = 1234
    s.bind(('localhost', port))          
    s.listen(1)   
    print('Socket is listening')

    # Establish connection with client.  
    c, addr = s.accept()      
    print ('Got connection from', addr ) 
    return s, c

def close_connection(s, c):
    s.close()
    c.close()

Is this the correct way to close a socket connection ?

[floral horizon]

https://github.com/tuxlovesyou/snapcam-python

Still working on the C extension that handles RTP headers, but the current proof-of-concept "works"

Had to send RTSP commands myself with sockets, as the RTSP server on the SnapCam can't be streamed with FFmpeg, Mpv, or VLC directly

[placid solstice]

https://usemynotes.com/what-is-intrusion-detection-system/?discord=python

[ember ledge]

I made an account checker friends

[sterile spoke]

  File "/usr/lib/python3.9/site-packages/urllib3/connectionpool.py", line 688, in urlopen
    conn = self._get_conn(timeout=pool_timeout)                                 
  File "/usr/lib/python3.9/site-packages/urllib3/connectionpool.py", line 273, in _get_conn
    conn.close()                                                                
  File "/usr/lib/python3.9/http/client.py", line 970, in close                  
    response.close()                                                            
  File "/usr/lib/python3.9/http/client.py", line 421, in close                  
    super().close() # set "closed" flag                                         
  File "/usr/lib/python3.9/http/client.py", line 432, in flush                  
    super().flush()                                                             
ValueError: I/O operation on closed file.

_io._IOBase.close and _io._IOBase.flush decided not to agree on whether an HTTP response was closed, and I don't know whether to blame a race condition (there is only one network thread) or a cosmic ray (despite having ECC memory)... hoping this never happens again.

[steady sage]

import time  # importing time module

capture = pyshark.LiveCapture(interface='wlo2', bpf_filter='arp')

capture.sniff(timeout=5)

keys = {}
e_mac = '00:00:00:00:00:00'
count = 1
for packet in capture:
    keys['ip'] = packet.arp.dst_proto_ipv4
    keys['mac'] = packet.arp.dst_hw_mac

    if keys['mac'] != e_mac:
        count = count + 1
        print(time.strftime('%b %d %H:%M:%S '), count, keys['mac'], keys['ip'])```

Dec 06 16:07:45 2 28:d1:27:1a:12:c0 192.168.1.3
Dec 06 16:08:01 3 ff:ff:ff:ff:ff:ff 192.168.1.1
Dec 06 16:08:14 4 f8:c4:f3:56:a3:70 192.168.1.1
Dec 06 16:08:14 5 f8:c4:f3:56:a3:70 192.168.1.1
Dec 06 16:08:49 6 28:d1:27:1a:12:c0 192.168.1.3
Dec 06 16:09:01 7 ff:ff:ff:ff:ff:ff 192.168.1.1
Dec 06 16:09:54 8 28:d1:27:1a:12:c0 192.168.1.3```

this the o/p
but i want if same packet arrives then update its count otherwise not
for e.g,

Dec 06 16:07:45  2 28:d1:27:1a:12:c0 192.168.1.3
Dec 06 16:08:01  4 f8:c4:f3:56:a3:70 192.168.1.1```

 
like 1.3 i received 2 times so it update only it's value to 2 and 1.1 i received is 4 times so it update only 4
it don't create duplicates only update counter

[topaz dune]

Hey, this is my first job and I will try to build a soap server-client in php from scratch(never done anything like this before). Any general advice how to start/ what to be careful?

1. Should I use for example 1 repo for client and 1 repo for server, and implement both of them in the same time

[lost dagger]

how do I find my sevres hostname?
I used the ipconfig function but I couldnt find it

[cedar forum]

what OS are you on?

[lost dagger]

windows*

[cedar forum]

you can use hostname to find it

[lost dagger]

sorry Im setting up kali

mb

I need to find the host name to set up

[cedar forum]

you can run hostname on windows as well

[lost dagger]

on command promt?

prompt*

[cedar forum]

yep

[lost dagger]

it gives my device host name

not my server thou

[cedar forum]

to find the hostname of a server, you need to run the command on the server

I'm not sure what you are trying to find or why

are you trying to find the hostname, domain or IP address?

[lost dagger]

to set up kali lnux, it asks you to get your server hostname as part of the procedure

[cedar forum]

you can put in anything

it's asking you to pick a hostname

[cedar forum]

it's just a device nickname or identifiable string

pick anything you want

[lost dagger]

oh okay ty

[short jackal]

when using DHKE to create a symmetric AES key on both sides, can you predefine the iv?

[dark trench]

Where within the handle_client function?

[short jackal]

does anyone know the reason you can't both encrypt and decrypt with the same AES object?

[hasty quest]

It probably uses a nonce

if you used the same nonce twice, it wouldn't be a nonce anymore.

A number used once is used to ensure unique crypttexts.

If I encrypt some data D with a key K, then I send that to you, an observer can't see the message, but they can see the crypttext.

If I then later encrypt the same data with the same key and send it to you or someone else, that observer knows that I replayed a message.

That's an information leak. So a unique number is added to the plain text to ensure that each crypttext made from that plaintext is unique.

[short jackal]

oh I finally got nonce

I was always confused about how you can decrypt a message without knowing the nonce, I didn't know it was just in plain text

[hasty quest]

ye

you do need to know the nonce to decrypt the message - but the nonce doesn't need to be a secret, so it can be sent in plain form.

In other words it only needs to be unique, not secret. But it probably won't let you use the same object twice, if that object has a nonce associated with it

[short jackal]

another thing, if I use DHKE, can I use a predefined iv (initial value)?

[hasty quest]

unsure, maybe

I would guess you probably can but that's really a matter of whether the library supports it or not.

[short jackal]

you can always derive the iv from the shared secret, but is it a bad idea to have it predefined?

[hasty quest]

I would guess it probably doesn't matter

[short jackal]

alright, thanks!

[hasty quest]

Though you ideally shouldn't re use it.

[short jackal]

but is it fine to use it for a single conversation?

[hasty quest]

yes

[brittle gazelle]

does someone know about Socket Programming here? can someone help me?

[crystal trellis]

guys someone tell me is there any way i can bind to a url instead of ip using socket lib?

[frozen drum]

no, DNS exists on a completely separate layer.

[topaz dune]

Hey, does anyone knows how can i test this SOAP server-client ?

https://gist.github.com/umidjons/f3de2533c51495a9c557

I don't understand what commands do I need to run

[compact widget]

Hello, my goal to use python to interpret the behavior of the Windows & Linux DNS Stub Resolver. In Linux I've created a python script

import socket
for x in range(5):
    print socket.getaddrinfo('example.com')

I then execute that script using 'strace -e trace=connect python dns.py 2>&1 | grep 53'. I then receive output that I can use to determine the order of operations of the DNS Stub Resolver and my use case is fulfilled for Linux. However I have failed to find a method of performing the same function on a windows machine. Is there anyone here smarter than me that could point me to a resource to help accomplish this?

[cedar forum]

you're looking for a strace alternative for windows?

https://drmemory.org/page_drstrace.html this kinda does that

[compact widget]

Thank you @cedar forum , I'll check that out 😀

[prisma cobalt]

Yes people do, what’s your query?

[brittle gazelle]

@prisma cobalt I'm developing one chat platform and i'm using Apache Cassandra with tenant architecture with python. I've implemented Django Channel + Redis server for web socketing. The problem is socket taking too much CPU utilisation even on a normal user flow. So, my messages are delivering late to the end users. Is there any idea OR any point where I can improve to get the smooth message transactions?
Thank You.

[steady sage]

#help-bagel check it

[visual lance]

Does anyone have a link where I can learn networking on a higher level?

that isn't outdated

[fair flax]

You mean like CCNA level? messing with router configs etc?

[visual lance]

yea

[crystal trellis]

Guys what are the things in networking that are necessary to learn

[fair flax]

Hm not sure if there is links here, but i can point you in the right direction since that networking is my career, https://www.youtube.com/watch?v=S7MNX_UD7vY&list=PLIhvC56v63IJVXv0GJcl9vO5Z6znCVb1P this is probably the best you can find for free

[visual lance]

thanks!

[ember ledge]

how do I setup a static ipv4 wireless on ubuntu server 21.10(ik shouldn't use wireless on server)

[steady horizon]

The concept of protocols and a few basic protocols

[crystal trellis]

@steady horizon any documentation or blog??

[steady horizon]

I don't know of a resource

Sorry

[hasty quest]

Start with the OSI model and these protocols:
Ethernet
IPv4
TCP
UDP

The TCP/IP guide is a good resource.

[lusty field]

@crystal trellis 'networking' is big. Are you interested in it as it relates to applications, or interested in the systems that connect other systems.

[crystal trellis]

Yep I am too much interested in it

[ember ledge]

hes asking

what part of "networking" do you want to learn

for example, how application, say youtube app on your phone, communicates with google servers
or for example, accessing your own server running in the cloud, the networking behind it

[ember ledge]

what is the best module for making a socket ui in python?

[wind oriole]

If I wanted to build a local proxy to preserve perfect versions (ie. same as they looked and functioned in a browser) of my web history, essentially by saving the data that's incoming and headed for the browser like a WARC file.... where would I start?

A library, a name, something to Google? Anything, I'm well out of my depth here

[wheat hatch]

So I am trying to connect to my deep learning vm by using ssh port forwarding
ERROR: (gcloud.compute.ssh) argument [USER@]INSTANCE: Must be specified.
I get this error. How do I fix it

[wind oriole]

For anyone else looking, I used mitmproxy, super simple

[wooden karma]

requests.get keeps throwing when requesting https://www.google.com using fiddler as a proxy

does anyone know why?

[wind oriole]

throwing what?

[wooden karma]

unable to connect to proxy

            "http": "http://127.0.0.1:8888",
            "https": "http://127.0.0.1:8888"
        }```

this is how i define the proxies that i pass

r = requests.session()
r.proxies.update( proxies)

[wind oriole]

get rid of http://

response = r.get('http://www.abcd1234.xyz', proxies=proxies)

[wooden karma]

get rid of it where?

ok i see

i think its specific to sessions

proxies = {
    "http": "http://127.0.0.1:8888",
    "https": "http://127.0.0.1:8888"
}
r = requests.Session()
r.proxies.update(proxies)
r.verify = False
r.get("https://www.google.com")

this fails to connect to the proxy, so clearly its not the correct way to pass them i guess?

proxies = {
    "http": "http://127.0.0.1:8888",
    "https": "http://127.0.0.1:8888"
}
# r = requests.session()
# r.proxies.update(proxies)
# r.verify = False
requests.get("https://www.google.com", proxies = proxies, verify = False)

this works without issue

and goes through fiddler

yea it seems like r.proxies.update(proxies) does not work

if i manually pass it in with the session request it works

otherwise, it doesnt

[hollow lark]

Hi, why is using 8192 bytes for a buffer so popular? Or why do u see many ppl use that amount of bytes for their buffer?

[cedar forum]

Great question! 8192 bytes is the default msgmax value for the Linux Kernel (see image). That's the maximum size of a message queue. There is also msgmnb which is the maximum message size in bytes, and that used to be set to the same value, although in more recent times it's now set to double (on my system it's currently 16384 bytes.

[hollow lark]

Oh okay what would happen if u put a bigger amount of bytes that in the msgmnb?

[cedar forum]

That's a good question. I suspect you'd either have it be truncated or the Kernel wouldn't send the packet.

[hollow lark]

That makes sense, why is there even such a buffer limit?

[cedar forum]

there is a limit of how much data should be sent across a network in one packet

say you're sending a PDF document to a friend

you'll be sending that with TCP

if you split it into smaller TCP packets and send that, then if some fail it's cheap to send them again

TCP will keep sending the smaller packets until the recipient gets all the data they need

if you didn't split it up as much and had packets that were large, then if one failed and you had to retransmit it's an expensive operation which takes longer

so it's all rooted in how data transmission works, and having smaller packets ends up being considerably more efficient

[hollow lark]

Oh yeah that makes sense

When something gets sent over udp tho is there is limit too?

[cedar forum]

there is a limit

of course all these are configurable, so the limit isn't concrete

most typically you'll see both TCP and UDP capped to 1,400 or 1,500 bytes on a network, known as the MTU (maximum transmission unit)

but for example, on a private network at work where we have machines that have good connection to one another and do a lot of file transfers we up that MTU to 9,000 bytes

[hollow lark]

Whats the difference between the MTU and the msgmng defined in the kernel?

[cedar forum]

msgmnb/msgmax is a linux kernel thing only, MTU has general definitions in all of networking

[hollow lark]

So defining the MTU would work over the msgmnb when using the linux kernel?

[cedar forum]

you generally need them to align

most people set a MTU lower than the linux msgmax so it's kinda irrelevant

[hollow lark]

Where do you set the MTU?

[cedar forum]

using the ip command on Linux, like:

$ ip link set dev eth0 mtu 1500

[hollow lark]

Hmm okay to be honest I dont think I really understood the difference between the mtu and the msgmnb if someone would be using the linux kernel so having the msgmnb whats the difference between those two?

[cedar forum]

msgmax is a linux limit that is for the whole system, MTU is per interface (per network card, vpn tunnel, etc.)

[hollow lark]

Ohhh okay now I got it

Thank you very much for your help!

[glacial burrow]

Hey guys I'm pretty new to networking and I'm giving it a crack but I'm having a few issues. So I'm trying to use UDP to transfer some time sensitive but loss tolerant data similar to voice but not. I'm sending the data from a C program to a Python program. Using netstat and wireshark I can see that there are no send buffer issues but I'm not receiving most of the packets that are sent. I tried to increase the SO_RCVBUF but it didn't seem to help much and netstat is still showing a lot of receive buffer errors. Even with a 200MB receive buffer I'm still getting 90%+ packet loss on the loopback interface. Does anyone have a clue as to why it's happening?

I'm trying to transfer much less than 200MB btw

This is how I'm setting up and using the socket

    sock.setsockopt(socket.SOL_SOCKET, socket.SO_RCVBUF, buffer_size)
    sock.bind((bind_ip, bind_port))

    window_packets = []

    print('Awaiting packet')
    while True:
        readable, writable, exceptional = select.select([sock], [], [])
        for s in readable:
            message, client = s.recvfrom(buffer_size)
            server_message_header = struct.unpack('=IIIQ', message[:struct.calcsize('=IIIQ')])```

[last sapphire]

Hi guys, I am rebuilding a backend with fastApi for a chat app. I need a high performant async non blocking websocket library for python. Any suggestions?

This app has hundreads of thousands of users, so the socket should be quite fast and stable.

I have been told to use node js for socket part, but I really wanna stick with python. So any suggestions would be really helpful. Thank you.

[gloomy root]

client or server?

if the latter, fastapi already has ws support

[undone gust]

about the socket module:
what's the difference between a ConnectionAbortedError and A ConnectionResetError?

[last sapphire]

Yes, but would it be a reliable choice lets say against node js socketio

[steady horizon]

I know one is raised by PyCharm when closing an end of a socket and the other is raised normally in the same case

[gloomy root]

Well alot of people prefer socketIO over raw ws but you can still do that with python if you want.

If you absolutely need performance though generally i avoid writing ws and continuous connection servers in python or in that fact, JS

Just because load balancing websockets are a pain so having to try balance them across each core manually is wayyh more work than say tokio and Rust where its basically just run it as normal and it'll use all cores and balance perfectly

[last sapphire]

If you had two choices between python and node js for socket, would it make a big difference in performance choosing node js?

I already started the backend in fastapi, so now i am thinking if i need to switch to node js.

I would prefer to stay on python, but am not sure of the performance of the python sockets. Wheres as node js socket seems to have good reputatuon.

Because this is basically a chat and video app for mobile, sockets will be used heavily

And with large number of users, i need the most stable performance i can get from the sockets

[gloomy root]

I wouldn't do it in nodejs

Realistically the perf gain isn't going to change much

The issue is more with balancing connections across cores which node isn't going to help you with

If performance is that tight i wouldnt use python or node for the websocket handlers, id use either Rust or c#

Ultimately the backend you choose isn't going to be as detrimental to performance compared to how well you load balance and how easy it is to scale up and down deployment wise

[last sapphire]

do you think GO would also be a good option?

https://matttomasetti.medium.com/websocket-performance-comparison-10dc89367055

this was quite astonishing to read actually

[gloomy root]

I guews, generally i dislike go for that

[sterile umbra]

Hi. Could you help me in #help-mango ? I'm trying to implement some API and I don't know how do I implement logging in with API key and API secret

[last sapphire]

would it be hard to implement backend in python and use c# for websockets. It will be deployed on AWS. Do I use extra EC2 instance for websocket handler?

[sterile umbra]

there's some hashing function, but I can't manage it to work

[prisma cobalt]

whats the difference between a message queue and the message size?

[cursive robin]

can someone help me

when I run it , it works for the first time

then it doesn't work in the second time

msgrev="message recived\nnumber of letters: "
serverPort = 12000
serverSocket = socket(AF_INET,SOCK_STREAM)
serverSocket.bind(('',serverPort))
serverSocket.listen(1)
print ('The server is ready to receive')
while True:
     connectionSocket, addr = serverSocket.accept()
     sentence = connectionSocket.recv(1024).decode()
     test=str(len(sentence))
     if (sentence=="exit"):
          print ("connection closed")
          connectionSocket.close()
     print (sentence)
     capitalizedSentence = sentence.upper()
     connectionSocket.send(capitalizedSentence.encode())
     connectionSocket.send(msgrev.encode())
     connectionSocket.send (test.encode())
     ```
tcpserver

serverIP = '127.0.0.1'
serverPort = 12000
clientSocket = socket(AF_INET, SOCK_STREAM)
clientSocket.connect((serverIP,serverPort))

while True:
    sentence = input('Input lowercase sentence:')
    clientSocket.send(sentence.encode())
    modifiedSentence = clientSocket.recv(1024)
    if (sentence=="exit"):
        print ("connection closed ")
        clientSocket.close()
        break
    print ('From Server:', modifiedSentence.decode())
    modifiedSentence=clientSocket.recv(1024)
    print (modifiedSentence.decode())
    
    ``` tcp client

it doesn't reach even the print statement in the server

[thorn stratus]

it looks like you send 3 messages on the server side and only recieve 2 on the client side

so the server is gonna be waiting on the client and will never get to serverSocket.accept()

this is generally why you want threading when dealing with network applications

[cursive robin]

ohhh

will do thanks

threading it's like making multiple processes run in parallel

right ?

[undone gust]

i'm not using pycharm, and pycharm doesn't change the behaviour of your script

[ember ledge]

I tried adding a proxy to aiohttp through proxiware and that's what it does, why?

[cobalt onyx]

Hi, how can i generate all ipv4 adresses from 0.0.0.0 to 255.255.255.255?

[cedar forum]

ipaddress is your friend

import ipaddress

whole_net = ipaddress.IPv4Network("0.0.0.0/0")
hosts = whole_net.hosts()

for host in hosts:
    # this will loop through all possible IPv4 hosts

as a smaller example that will run in Discord:

!e ```py
import ipaddress

whole_net = ipaddress.IPv4Network("192.168.10.0/28")
hosts = whole_net.hosts()

for host in hosts:
print(host)

[errant bay]

@cedar forum :white_check_mark: Your eval job has completed with return code 0.

001 | 192.168.10.1
002 | 192.168.10.2
003 | 192.168.10.3
004 | 192.168.10.4
005 | 192.168.10.5
006 | 192.168.10.6
007 | 192.168.10.7
008 | 192.168.10.8
009 | 192.168.10.9
010 | 192.168.10.10
011 | 192.168.10.11
... (truncated - too many lines)

Full output: https://paste.pythondiscord.com/padapozixe.txt?noredirect

[cobalt onyx]

appreicate it!

@cedar forumwould there be anyway to speed up the process? this would surely take hours to complete

[steady sage]

Can i use two bpf filter simultaneously in pyshark?

[cedar forum]

it would, yeah, but I mean there are 4 billion IPv4 addresses and you asked to generate all of them hahaha

let's approach it a different way: what are you trying to do with all IPv4 addresses?

[cobalt onyx]

pinging certain ports on them

[steady sage]

Is there anyway?

[cedar forum]

right, if you want to portscan the entire internet then it is going to take a lot of time

[cobalt onyx]

im aware

[cedar forum]

the best thing you can do is have several machines pinging

instead of generating for "0.0.0.0/0" generate for "1.0.0.0/8" and "2.0.0.0/8" and so on

but be warned, your ISP will likely cut service long before you can ping scan the entire internet

[cobalt onyx]

alright

ill try run it on heroku

[cedar forum]

yeah heroku will ban you pretty quickly

so, I wouldn't do that

[cobalt onyx]

even if i used an api to scan?

[cedar forum]

eh, probably not, but they'd still be curious why there is a stupidly large amount of traffic to and from one endpoint

[cobalt onyx]

well most of my heroku apps are just sending multiple requests, seems to be running good. but thanks for the help, will try

[steady sage]

pyshark.LiveCapture(interface='wlo2', bpf_filter ='arp and udp port 67')

Does this work?

[cedar forum]

yes, you can do that

[steady sage]

I think that's not working, coz after doing I'm unable to capture any packet

[cedar forum]

it's definitely supported

but your filter makes no sense

ARP is not a UDP protocol

[steady sage]

But using single filter like arp or udp port 67 it's working

[cedar forum]

right, but you're saying arp and udp port 67

and arp is not UDP

so you have defined an expression that can never be satisfied

if you want ARP and DHCP at the same time you need arp or udp port 67

[steady sage]

Oh kk

[restive pebble]

how do I port forward my socks5 server through my router? I realize I can do ssh -D <port> <server> everytime, but I really do not want to do have to open a new connection everytime I want to use the proxy, how can I port forward my socks5 server through it's router so it is accesible at any time?

[steady sage]

By using or it's raising Attribute Error : No attribute named dhcp

[cedar forum]

can you share some code?

[steady sage]

capture = pyshark.LiveCapture(interface='wlo2', bpf_filter='udp port 68 or arp')
capture.sniff_continuously()

for packet in capture:
    print(packet.dhcp.option_hw_mac_addr, packet.dhcp.option_hostname, packet.dhcp.option_vendor_class_id, packet.arp.dst_proto_ipv4, packet.arp.dst_hw_mac)

[cedar forum]

yeah so the problem is you're probably capturing an ARP packet and then trying to print out the DHCP properties on it

you need to add an if statement to see if you're processing an ARP packet or a DHCP packet, they're not the same thing

[steady sage]

share code for arp

[cedar forum]

huh?

[steady sage]

like if arp packet received

[cedar forum]

you need to check if the packet is of type DHCP or ARP

I'm unsure of the syntax to do so

[steady sage]

ok

[fierce valley]

I am having a doubt so I made a little multicast sender and reciever and the sender is sending readings of a water heater temperature.

So here is the problem I also want to send data back to the sender like telling it to turn of the heater, how can I do this. Is a basic http server the only way?

Also ping me :]

[lusty field]

I snorted some coffee

[prisma cobalt]

pinging certain ports... on all the possible IP addresses? 😳
see you in a few years

[cobalt onyx]

using an API lol

[prisma cobalt]

that wont do anything lol

if anything it would slow it down probably

[cobalt onyx]

true

we'll meet again in 2025 ⛵

[prisma cobalt]

lmao

[fair kindle]

you know, it's been done

It does take a long time though, and probably best done with a whole farm of machines

[slender steeple]

https://github.com/robertdavidgraham/masscan go brrr

[fair kindle]

wtf, that can't be for real

people would be scanning the whole Internet all the time if you could do it in 5 minutes :p

[slender steeple]

obviously it depends on your network speed

so its just a theoretical max 😛

[foggy basalt]

!rule 5 @lapis lichen

[errant bay]

Rules

5. Do not provide or request help on projects that may break laws, breach terms of services, or are malicious or inappropriate.

[foggy basalt]

sorry, but we cannot allow you to share that video- even if it's for educational purposes.

[prisma cobalt]

how do sites like youtube keep sending video data on demand without reloading the page or anything (trying to create an online multiplayer game in browser)

[cedar forum]

chunks and buffers

so youtube takes every video and splits it into chunks, and then the browser requests them on demand so instead of streaming a 300MB video you break it into chunks of 2MB and deliver them when the client starts running out (see the buffer health in that screenshot)

you can see when the buffer health gets low, the network then requests the next chunk (hence why network activity goes green), and then the available buffer goes up in size

crucially, it is not a live stream, and the video is not delivered all at once

[errant bay]

Hey @lone leaf!

It looks like you tried to attach a Python file - please use a code-pasting service such as https://paste.pythondiscord.com

[lone leaf]

https://paste.pythondiscord.com/fagikobome.kotlin

The output is my ip address. Why is the proxy not connecting?

[cedar forum]

you are specifying the proxy for https, you need an entry for http as well

[lone leaf]

so from me the website needs to be https. Did I get right?

[cedar forum]

yeah or just add a http entry to the proxies dictionary

[lone leaf]

proxies = {'https': '165.0.50.110:8080',
'http': '165.0.50.110:8080'

}

this is how i did it.it gives error

requests.exceptions.InvalidURL: Proxy URL had no scheme, should start with http:// or https://

[cedar forum]

you need to add a scheme onto the proxy, yeah

[lone leaf]

okay thank you

[iron fjord]

Is there any guides/articles/videos about buffering? What excacly is a buffer size and how does it work? How should i be implementing it?

[cedar forum]

many, depends on what you're doing

scapy, pyshark, higher level stuff like websockets & httpx

[hushed pine]

I've been trying to get a threaded ssl socket working in python but I've been having weird issues, it gets hung up on requestline = str(self.rflie.readlne(65537) when i try to connect with this python code, but if i use the openssl client in the command line it works perfectly

this is the client code, it gets hung up on response = self.socket.recv(1)

same thing with openssl, you can see it returned "test command" which is what it was supposed to do

omfg

it was the line part of readline

thank you

[fair flax]

Assuming i got no cisco/huawei router that i could try this. I would like build a script to use on those routes, where can i simulate this?

[delicate viper]

can someone help in #help-cheese ? it just takes some time to read but you can know my problem if you read from first message

#help-cheese message
first read my problem then start reading from here when i forwarded a port

[rotund wigeon]

^ tldr: cannot connect to Python application through public IP address, even with port forwarding

packet is probably being dropped somewhere because connections keep getting timed out

port forwarding is probably done correctly, because without port forwarding the connection is close unexpectedly

with port forwarding it's connection timed out

[ember ledge]

I used a proxy to request with aiohttp client, is ServerDIsconnectedError from my side (proxy) or the website's side?

[wooden karma]

is it possible to turn off remote degugging in pycharm

[hollow path]

so i'm using the requests library right

or package

whatever the hell

anyway

is there a way to return the text of the website when you use returned.text

instead of the html

[ember ledge]

what do you mean by the "text"?

text of what

[storm saffron]

would have to parse the html and extract text tags

could use beautifulsoup

the simple answer is there's no magic way to get 'just the text' since that's defined by how a browser renders the html

[hollow path]

ok thank you

[ember ledge]

?

[plucky oar]

Lol.

[plucky oar]

Whats networking.

[plucky oar]

Nvm

[heavy quarry]

hi folks

is this a place to ask how to capture network traffic with python?

something like this

really simple

but no matter how much forums I visit I can't seem to find an answer

[fierce pollen]

Hello! Is there anybody very educated in Python sockets (select, treading, etc.)?

[cedar forum]

a lot of variables are at play here

what sort of traffic are you trying to capture? HTTP, TCP, lower level?

If you're capturing HTTP, you need to find some way to decrypt it if it's HTTPS

take a look at https://mitmproxy.org/

for something lower level look at scapy/pyshark

[heavy quarry]

thanks for answering !

[cedar forum]

👍

[cedar forum]

what question do you have?

[heavy quarry]

I'm currently stucked here

raise Exception("Could not read Browsermob-Proxy json\n"
Exception: Could not read Browsermob-Proxy json
Another server running on this port?

[heavy quarry]

my final goal is to parse those xhr files

I need that info

[heavy quarry]

just in case, ofc I know how to parse that json by copy pasting

the problem is that I want to automate that task

cause there are a lot of those

and

the thing is that I can't just access the url of that request cause it's on a private server

[fierce pollen]

Just some suggestions. I am seeking for a high level Python lib, that allows me to create Socket server that can handle multiple clients (non-blocking) and do some threading, as for every received message I will need to do some calculations.

[frozen drum]

Twisted

[fierce pollen]

I was looking into it, bet when it gets to Threading > https://twistedmatrix.com/documents/current/core/howto/threading.html I am lost. The examples fully lose connection to socket server.

[frozen drum]

do you want to open new sockets in the spawned threads, that connect to another server..?

[fierce pollen]

No. Its just some middle-heavy python calculations,

[frozen drum]

I don't understand what you mean by The examples fully lose connection to socket server. then.

If you are talking about the clients, depending on the protocol you may have to implement some sort of keep-alive

[fierce pollen]

The examples serve only one thing.. to run some threads.. I don't see any socket server related code and how it would mix with it

from twisted.internet import reactor, protocol

class Echo(protocol.Protocol):
    def dataReceived(self, data):
        # Do some bad ass calculations on a seperate thread, send back message when done
        pass

def main():
    factory = protocol.ServerFactory()
    factory.protocol = Echo
    reactor.listenTCP(8000, factory)
    reactor.run()

if __name__ == "__main__":
    main()

The commented line, is what I can't figure out

[frozen drum]

something like this:

from twisted.internet import reactor, threads

... # your socket code
def long_calc():
   do_stuff_here()

def print_result(result):
   print(result)

# put this in whatever place you need to call from your socket code
d = threads.deferToThread(long_calc)
d.addCallback(print_result)

[fierce pollen]

Mkay will try

[frozen drum]

if you need to give arguments to long_calc, deferToThread can supply args/kwargs.
You likely want your callback to be some function that creates and/or sends the response after the calculation is done

[fierce pollen]

Ok Ok

[past raptor]

hi guys

[icy spoke]

hi, I'm looking to see if something like this is practical to write.

I'm looking to create a forward proxy that forwards https requests to a remote proxy server. Is this something I would do in sockets?

Kind of like proxy chaining

[graceful oriole]

sounds like something you'd do with a proxy

[brazen badge]

guys, i need a lil help, a main function in my program is supposed to do 10 post requests, to make the program faster i used threading "threading.Thread(target=themainFunction).start()" and i am using 10 threads as well now i see that my program's main function is doing 10 requests per thread so it means 10*10 = 100 requests are being done, how do i avoid those 100 requests and do only 10 requests but in parallel ?

[heavy quarry]

Idk 😦

sorry

btw does anyone know how to capture this network traffic that the browser records?

the json on the right

I tried everything more or less

[copper adder]

you would need to write a proxy server and direct your browser to use the proxy

[heavy quarry]

thx!

[heavy quarry]

I'm having a problem using that solution 😦

cause all ports are being used

in the company I work for

I did find a solution but it's not working as I intended

It's retrieving me this

[copper adder]

okay, since it's a work environment, you will have to connect to the management port where all traffic gets sent to

[heavy quarry]

oh

thx

I'll first try to find a solution without using a proxy but if there's no option I'll try that

[copper adder]

goodluck and be sure to share here incase you find another way to go about it

[heavy quarry]

thx!

import json
import pprint

from selenium import webdriver
from selenium.webdriver.common.desired_capabilities import DesiredCapabilities
capabilities = DesiredCapabilities.CHROME
# capabilities["loggingPrefs"] = {"performance": "ALL"}  # chromedriver < ~75
capabilities["goog:loggingPrefs"] = {"performance": "ALL"}  # chromedriver 75+
driver = webdriver.Chrome( desired_capabilities=capabilities)
def process_browser_logs_for_network_events(logs):
    """
    Return only logs which have a method that start with "Network.response", "Network.request", or "Network.webSocket"
    since we're interested in the network events specifically.
    """
    for entry in logs:
        log = json.loads(entry["message"])["message"]
        if (
                "Network.response" in log["method"]
                or "Network.request" in log["method"]
                or "Network.webSocket" in log["method"]
        ):
            yield log
driver.get("https://app.nubimetrics.com/account/login?")
username = driver.find_element_by_name("email")
username.clear()
username.send_keys("MY_EMAIL")
password = driver.find_element_by_name("password")
password.clear()
password.send_keys("MY_PASSWORD")
driver.find_element_by_xpath("//button[@type='submit']").click()
resultado = driver.get("https://app.nubimetrics.com/market/bycategory#?range=2021-11-01&category=MLA1051&order=Gmv")
logs = driver.get_log("performance")
events = process_browser_logs_for_network_events(logs)
with open("log_entries.txt", "wt") as out:
    for event in events:
        pprint.pprint(event, stream=out)

this is as far as I got

[heavy quarry]

only to get this part of the requests

[copper adder]

ohhh

so you want to read network traffic for just one browser?

[heavy quarry]

yes

I mean

[heavy quarry]

Im trying to scrape this json responses

[copper adder]

can't you just request it directly or the URL changes?

[heavy quarry]

the problem is that the url redirects me to a private api

https://app.nubimetrics.com/api/Market/CategoryData?category=MLA1051&currency=ARS&date=2021-11-01&language=es&seller_id=185702920&site_id=MLA

[heavy quarry]

so I can't access the json file, the response

[copper adder]

does this require cookie?

[heavy quarry]

ikd

I mean

[heavy quarry]

that's what I find here

but besides that ikd

[copper adder]

so on the browser, you get the json but if you make the request other than from the browser, it doesn't return the json and that is why you want a way to get access to that response while using selenium.
Is the above statement true?

[heavy quarry]

the problem is that Idk how to make that request

[heavy quarry]

I copied a script that retrieves me this headers

not the json

[copper adder]

it a bit difficult to understand, you want to scrap a website right???

[heavy quarry]

oh sorry

yes

What I'm trying to do is

Website > Inspect element > Network tab > (click on a file) Response tab

I want that json

I already know the name of the "file

[crystal current]

Is this all you need? If not, what exactly is the issue? https://stackoverflow.com/questions/39596826/how-to-download-text-as-a-json-file-in-python-using-the-requests-library

[clever gazelle]

this keeps happening when i run my code, i tried it with socket.send but it still doesnt work

[cedar forum]

that's a method of your open socket, you're calling it on the socket module

[clever gazelle]

Right so when I call socket.socket I jave to make it a variable

[frigid vault]

Try with only 'send'

[clever gazelle]

can i use the ip address of a linode server to open a socket/port?

[clever gazelle]

Yeah ik

I'm asking if I can use the ip on a Linode server to open a socket/port

[ember ledge]

whats a good library to use for running code on a "server" that does commands like generate files using adobe and upload them to google cloud from a client machine (aka my laptop or a coworkers laptop)

i can do this all easily using sockets but im guessing thats not "secure" ?

[frigid vault]

Try with scapy.

[ember ledge]

wait is scapy for hacking lol?

[frigid vault]

It depends on how you use it and for what purpose.

[frozen drum]

what you are asking makes no sense. You can't "use an ip to open a socket/port", you connect to an ip with a socket on a given port.

If you are asking whether you can connect to a linode server externally with sockets, the answer is yes, barring firewall rules etc. If you are asking whether you can open a socket ON the linode server to listen for incoming connections, the answer is also yes (again depending on firewall setup)

[clever gazelle]

Yeah I meant if I can use a linode server to open a socket

Thanks

[ember ledge]

can anyone refer me to a source to learn networking in python?

[night perch]

!resources

[errant bay]

Resources

The Resources page on our website contains a list of hand-selected learning resources that we regularly recommend to both beginners and experts.

[ember ledge]

theres nothing about networking there

[narrow sequoia]

https://pythongeeks.org/networking-in-python/

[hollow lark]

Many ppl say it dangerous to be connected to an open wifi, bc everyone could sniff the data u are sending around. Why does that not work when u are using a private wireless connection? I mean the data is still "flying" around thorugh the air.

[thorn stratus]

@hollow lark Because private wifi connections are encrypted

it's still less secure than a wired connection and there are attacks to figure out the key, but still a ton better than an open wifi connection

although hopefully all the apps you use now a days should be end to end encrypted anyways

[tepid helm]

Does anyone here know if it's possible to use Spark Streaming with websockets? I'm trying to capture a ton of data from a websocket connection and write it to parquet files

[frozen sleet]

Help

Hi guys

I need help

With something

So baicalky I am working on a machine learning assignment in creating an algorithm that will detect brain tumors based on brain data tumor data sets. This is an assignment that is asked by my professor but I don’t have experience with coding.

These are the steps I already complete which is

(1) Import the dataset into a fresh Google Collab project
(2) Split the dataset into training / testing / validation sets (thursday)

But i still need help with
(3) Defining my classification model(s). You would probably want to try a few different models here.  You can either build your own convolution neural network (layer by layer) in tensorflow and train it from scratch, or you can modify an existing pre-trained network like VGG19, alter it to better suit our binary classification needs, and retrain it on the dataset.
(4) Train you model(s) and evaluate

[steady sage]

can i run filecapture and livecapture concurrently in pyshark

will they work?

[carmine wolf]

you have no experience with coding but your professor assigns you a coding assignment? has he done any tutorials or demos?

[steady sage]

from collections import Counter

# creating a file capture, give path of your captured file
capture = pyshark.FileCapture('/home/sipl/Downloads/DHCP.cap', display_filter='udp.port eq 67')

count = Counter()
d = dict()  # declaring an empty dictoinary

# running for loop for extracting fields from captured packets
for packet in capture:
    try:  # using try block to check for error
        d['Mac'] = packet.dhcp.hw_mac_addr  # defining key and assigning values to empty dictionary
        d['Host_Name'] = packet.dhcp.option_hostname

        # d['Vend_Class'] = packet.dhcp.option_vendor_class_id
    except AttributeError:  # excepting attribute error
        pass
    try:
        d['Server_id'] = packet.dhcp.option_dhcp_server_id
        d['Subnet_mask'] = packet.dhcp.option_subnet_mask
        d['DNS'] = packet.dhcp.option_domain_name_server
        # d['Domain_name'] = packet.dhcp.option_domain_name

    except AttributeError:
        pass
    try:
        d['IP'] = packet.dhcp.option_requested_ip_address
        # Assigning counter on the fields
        count.update([(d['Mac'], d['IP'])])
        # Creating a text file in append mode
        with open('cap.txt', 'a') as op:

            # printing the output in text file
            print(packet.sniff_time, count[(d['Mac'], d['IP'], d['Host_Name'])], d['Mac'], d['IP'], d['Host_Name'],
                  d['Server_id'], d['Subnet_mask'], d['DNS'], file=op)
    except AttributeError:
        pass

what to do so it doesn't throw key error when i get missing field, instead of that it print nothing

2009-09-29 22:26:22.602625 0 cc:00:0a:c4:00:00 192.168.0.3 R0 192.168.0.1 255.255.255.0 192.168.0.1above code output

if i remove commented 'vend class' and 'domain name' it throw keyerror on printing these field

[heavy quarry]

howdy folks

would anybody help me with this?
I'm trying to scrape the response of a request that a platform with which I work does

r = requests.get('https://app.nubimetrics.com/api/Shared/getAccoounts?seller_id=185702920&site_id=MLA', auth = HTTPBasicAuth('SAME_USERNAME_FOR_LOGIN??', 'SAME_PASSW_FOR_LOGIN??'))

this doesn't work

print(r.content)

b' '

print(r.status_code)

401

I guess another way could be grabbing the json file response that my browser records but I can't find a way to do that 😦

[still loom]

You need to log in into the page

[heavy quarry]

Hi!

I already did that 😦

let me show you

def find_CategoryAndBrand():
    cj = http.cookiejar.CookieJar()
    br = mechanize.Browser()
    br.set_cookiejar(cj)
    br.open("https://app.nubimetrics.com/account/login?")

    br.select_form(nr=0)
    br.form['email'] = 'my email'
    br.form['password'] = 'my password'
    br.submit()]
    r = requests.get('https://app.nubimetrics.com/api/Shared/getAccoounts?seller_id=185702920&site_id=MLA', auth = HTTPBasicAuth('SAME_USERNAME_FOR_LOGIN??', 'SAME_PASSW_FOR_LOGIN??'))
    print(r.content)

the login works

[still loom]

It seems that the site is not working

[heavy quarry]

cause it captures the correct html

[still loom]

When I try to load it, it is throwing error

[heavy quarry]

it does show me some of the responses of the requests when I type those urls into tthe searhc bar of the browser

[heavy quarry]

yep

idk

[still loom]

<p>
<font face='Arial'>
<b>Text 1 </b>
Text 2
</font>
</p>

I want to scrape the 'Text 2' using beautifulsoup

Can you help with this?

[heavy quarry]

what should work is to capture what my browser already recordes

[heavy quarry]

yeah sure

let me think

[still loom]

I tried going to the network tab and reloading the page, it is throwing 404 error

You sure the website exists?

[heavy quarry]

could you try using

data = soup.find('p')
scrap_data = data.find('b')
result = data - scrap_data

[heavy quarry]

yep

[heavy quarry]

did it work?

[still loom]

what you mean by

data- scrap_data?

[heavy quarry]

minus

fr lol

[still loom]

You can't do that

[heavy quarry]

what are the results of the first 2 searches?

strings? lists?