networks

[balmy axle]

is it to be expected that a selectors event loop, when registering EVENT_WRITE, keeps one CPU core at very high utilization if at least one client is connected, regardless of network traffic?
the same doesn't happen if i only register EVENT_READ and write to the socket directly with .send().
example code with EVENT_WRITE that keeps one core busy is here, if that is of any relevance:
https://ahuemer.xx.vu/volatile/2023-07-14-RIpVXTpDZs4/main.py
strace says epoll_wait(3, [{events=EPOLLOUT, data={u32=5, u64=5}}], 2, -1) = 1 is executed over and over again.
it's understandable that the socket is writable virtually all of the time.
question is, is there anything that can be done to reduce CPU utilization in idle situations when clients are connected or do i have to accept that? (well, or reside to send without the loop again)

[cunning garden]

!rule 9

[errant bay]

Rules

9. Do not offer or ask for paid work of any kind.

[frozen drum]

are you on MacOS?

[balmy axle]

🐧

[frozen drum]

ok, but you want to update the selector depending on what you want to do

if you don't have anything to write, don't set the EVENT_WRITE selector

[balmy axle]

well, 'anything to write' depends on the point of view, right? i do have to send data, but we are not talking about excessive amounts

[frozen drum]

yes but you're basically emptying your output buffer

when it's empty, you should clear the write flag

[balmy axle]

i wonder if there are objective decision criteria, you know, something based on a formula that gives you an indication if EVENT_WRITE or direct send() is the better approach for a given situation

[balmy axle]

hmm, i might have a gap in understanding there. how do i do that?

[frozen drum]

you can modify a selector

e.g. sel.modify

and then only set the new event to EVENT_READ

if you know you have no data to send

[balmy axle]

oh... i get the idea

interesting

thanks for the suggestion, i'll give that a try

related topic... is my understanding correct that if i use EVENT_READ and EVENT_WRITE it is virtually not possible to run in an EWOULDBLOCK situation, but if i send out of sync with the event loop that only handles EVENT_READ than there is a non-zero possibility?
my tests indicate exactly that under high pressure situations, but i am not sure how how-suited my tests are

implicit -> explicit: this is a non-blocking socket

[frozen drum]

well

technically if you have control over the entire network path you are right

I wouldn't count on it though

e.g. if the other side of your pipe (or even a broken intermediary) reads way slower than you write, you can still get a full write buffer

[balmy axle]

okay, so i'll have to account for the possibility of that situation / exception regardless. got it

[frozen drum]

yes

99.99% of cases you probably don't

..but it's not guaranteed you won't hit that situation

[balmy axle]

you know, i find it relaxing to understand fully what i am working on. accounting for the 0.01% case isn't a mistake as far as i am concerned 🙂

well, thanks! very helpful information. i'll go hit the keyboard again

[frozen drum]

it's been a while since I've done any low level socket implementations, but I work with networking every day and can tell you weird shit happens at least every day..

have fun !

[balmy axle]

my dayjob isn't coding anymore since more than 10 years so i am pretty rusty. but yeah, can confirm, weird shit happens every single day 😄

[final meteor]

What does it mean when theres a double backslash in the middle of a url infront of an ip?
example: http://google.com/?page=//10.10.14.25/somefile

[inland rampart]

I don't think it means anything in particular; it's a pretty weird format but you can have all sorts of things in the query string

[languid umbra]

Any suggestions about how to get usage metrics for my flask API?

[prisma cobalt]

this might serve your needs: https://flask-track-usage.readthedocs.io/en/latest/

[languid umbra]

Eh, not really what I need. Is there a way to just have a universal route handler? Like any route requested will ping that hook and then go to the route normally?

[balmy axle]

wanted to let you know that this works beautifully. after accept()ing a connection i only subscribe to EVENT_READ. once i am done processing input, i temporarily subscribe to EVENT_WRITE as well. that then sends the data until the output buffer is empty. when that is achieved, i remove the subscription to EVENT_WRITE.
this seems to work entirely reliably. i can process around 15k EVENT_READ triggers per second without any issue.
when clients are connected without doing anything, CPU utilization is nada.

[frozen drum]

sounds great!

[shadow swift]

Hi there, I'm currently stuck at a point where I want to establish connection between host and guest (VM) machine.

I have a process running on Windows Host machine on a specific port. I want to access It via Linux VM.

[quasi widget]

what process

[shadow swift]

what process?

[quasi widget]

I have a process running

[shadow swift]

I have a process running on windows machine, that uses specific port. Now, I want to access it via VM.

[quasi widget]

What process is it

provide more context

[shadow swift]

Does it matter to create connection between host and guest?

I can share my entire system via teamviewer. If you can help.

[quasi widget]

you're avoiding my question

[shadow swift]

No, I'm not..I'm just tired. Since last 18hr I'm trying to search the solution but I'm not at all around the solution.
Everyone are just curious about the process.

Even I'm curious about setting up the server.

[quasi widget]

ill go sleep now, please do not mention me while i'm away

just say what process it is

[shadow swift]

cool,

you've no benefit from it dear..

😄

It's some Daemon running on my window host machine.

[coarse summit]

what issue are you having?

[shadow swift]

I've no clue whether I should be configuring the VM network settings or Is it because of firewall.

I'm able to connect to port that is runnnig on windows host from my linux(in vm).

When I try to ping: It say's connection refused,

[coarse summit]

from which os to which os are you trying to access

[shadow swift]

I checked the ipconfig and if config. Both are on different network and my virtualbox adapter shows some other IP which is not at all used in linux(VM)

[coarse summit]

.

[shadow swift]

Windows (HOST) <=> Kali Linux (VM)

[quasi widget]

Maybe try another distro?

Or try to Install NetworkManager and nmcli and Set it up

[coarse summit]

from which pc are you sending the ping request

[shadow swift]

Vice versa

[coarse summit]

wdym viceversa

[shadow swift]

linux to windows

[coarse summit]

how are you listening in linux

[quasi widget]

Can you Ping 1.1.1.1 or something else

In linux

[shadow swift]

windows ipv4 start with 192.168 and Linux ith 172.2.

[quasi widget]

.

[shadow swift]

turning on vm hold up

$ ping 1.1.1.1
ping connect: Network is unreachable

[quasi widget]

You have No internet

[shadow swift]

$ ping HOST_IP
Destination host unreachable

[shadow swift]

I'm using internet.. 😒

[quasi widget]

Not in your gm

Vm

Show your vm configuration

[shadow swift]

At both place

[quasi widget]

You'd be able to Connect to 1.1.1.1 ifnyou Had internet

[quasi widget]

See

You dont

[quasi widget]

Change Network adapter

[shadow swift]

sry you were right

[quasi widget]

To bridged

[shadow swift]

are you sure?

[quasi widget]

Yes

[shadow swift]

I shall change adapter 2 to bridged?

[quasi widget]

Try 1

Not sure i use VMware

[shadow swift]

hm

[quasi widget]

Try now

[shadow swift]

try what? pinging host from vm?

[quasi widget]

Yes or Just 1.1.1.1 to Check if you are conencted

[shadow swift]

nope

If i use bridged, I lose my internet connection.

[quasi widget]

And 1.1.1.1?

[shadow swift]

same

[quasi widget]

Not Sure then, maybe Just don't use Kali/virtualbox

[shadow swift]

do you think that is this something related to firewall or iptables?

[quasi widget]

I dont think so

Connect to eth0 instead of eth1

[quasi widget]

Ther

[shadow swift]

So, now do you have any sugessstion regarding I should be discussing this somewhere else or a specific community?

[quasi widget]

.

[shadow swift]

If i connect t eth0, I'll be able to ping 1.1.1.1 but that would be NAT not bridged..

[quasi widget]

Oh

Stay at NAT then

Try pinging Windows with it's Public IPv4

[shadow swift]

I'm able to ping in that way

[quasi widget]

So Just do that

[shadow swift]

I was able to do that, Now I tried pinging the linux machinne ipv4 from windows and got request timed out

[quasi widget]

Also the Public IPv4?

[shadow swift]

yes I was successfully able to ping public ipv4 of windows from Linux

[quasi widget]

And Ping Linux from Windows?

[shadow swift]

nope that din't worked

[quasi widget]

What happend

@shadow swift

[shadow swift]

Request timed out

@quasi widget tysm for your time.. It's late already 3.14am here.. going to crash on bed now. See you again! Thanks Again 😄

[quasi widget]

Public or private ip

I'll also Go sleep now

Answer later

[shadow swift]

one i found in ipconfig

Wireless LAN adapter Wi-Fi:

   Connection-specific DNS Suffix  . :
   Link-local IPv6 Address . . . . . : **************************
   IPv4 Address. . . . . . . . . . . : 192.168.1.122

[quasi widget]

Go to http://api.ipify.org/ instead

Use that ip

[shadow swift]

cool, you sleep I'll give it a try and sleep

[quasi widget]

Alright

[shadow swift]

I found the ip (this ip http://api.ipify.org/) to be same on both system.

I'm still stuck in same state.

[shadow swift]

Progressed, I'm able to ping vice versa.

But still can't access the port

[quasi widget]

is the port open in windows? also check firewall

[shadow swift]

Hey I just checked that I'm only able to connect thorugh lookpack

Currently, I'm on windows,

ADB interface is running on 127.0.0.1:5555
My ipv4 on windows is 192.168.56.104

When I'm trying to connect to adb interface using this command "adb -s emulator-5554 connect 127.0.0.1:5555" then the connnection is success.

But whenI'm trying to connect to adb interface using this command "adb -s emulator-5554 connect 192.168.56.104:5555" then I'm getting error "cannot connect to 192.168.1.122:5555: No connection could be made because the target machine actively refused it. (10061)"

[quasi widget]

use 127.0.0.1 as your ip on both linux and windows

[shadow swift]

This work on windows but not on linux

On linux I get this error:

└─$ adb -s emulator-5554 connect 127.0.0.1:5555     
failed to connect to '127.0.0.1:5555': Connection refused

[quasi widget]

can you nmap -p- 127.0.0.1

on linux

to see what is open

[shadow swift]

Starting Nmap 7.93 ( https://nmap.org ) at 2023-07-16 09:30 BST
Nmap scan report for localhost (127.0.0.1)
Host is up (0.000053s latency).
Not shown: 65532 closed tcp ports (conn-refused)
PORT      STATE SERVICE
1716/tcp  open  xmsg
5037/tcp  open  unknown
46331/tcp open  unknown

Nmap done: 1 IP address (1 host up) scanned in 0.80 seconds

[quasi widget]

okay

not sure

[coarse summit]

Has someone made a mail server with https://twisted.org ?

I am unable to understand how to make one

[cerulean onyx]

Hey everyone I'm having a really hard time with threads and sockets, here's my code:

import socket
import time
from random import randint,random

HOST = "127.0.0.1"


def serve(port):

    with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as s:
        s.bind((HOST, port))
        s.listen(1000)

        while True:
            conn, addr = s.accept()
            with conn:
                print(f"Connected by {addr}")
                try:
                    request = conn.recv(1024)

                    if request == b"r":
                        print(f"Request received {request}")
                        # time.sleep(random())
                        time.sleep(0.01)
                        value = randint(0, 2048)
                        conn.sendall(value.to_bytes(2, byteorder='big'))
                        print(f"{value} were sent")

                except socket.error as e:
                    if isinstance(e.args, tuple):
                        print("Detected remote disconnect")
                    else:
                        print("socket error ", e)
                        s.close()
                        break

this one is server as you can tell

then i have 3 different scripts which work like this : ```py
from serve import serve

PORT = 65432

serve(PORT)```

ports are different for each of them obviously

and i want to asynchronously connect with server and get the data

i wrote this code and it works

async def get_value_async(host, port,i):
    try:
        reader, writer = await asyncio.open_connection(host, port)
        writer.write(b"r")
        data = await reader.read(1024)
        writer.close()
        data_int = int.from_bytes(data, 'big')
        print(f"{i} - Received {data_int}")
        return data_int
    except Exception as e:
        print(f"Error while connecting to {host}:{port}: {e}")
        return 0

async def get_values_async():
    tasks = [get_value_async(HOST, port,i) for port in PORTS for i in range(1000)]
    results = await asyncio.gather(*tasks)
    total_values = sum(results)
    return total_values / (len(PORTS) * 1000)

async def runAsyncronousProcess():
    print("Start of asynchronous calls")
    start_time = time.time()
    result = await get_values_async()
    print(f"Average value retrieved asynchronously: {result}")
    end_time = time.time()
    elapsed_time = end_time - start_time
    print(f"Asynchronous calls took {elapsed_time:.4f} seconds to complete.")


async def main():
    # runSyncronousProcess()
    await runAsyncronousProcess()

if __name__ == "__main__":
    asyncio.run(main())```

buuuut

works without errors if i create 100 tasks and not 1000

otherwise gives error : Error while connecting to 127.0.0.1:65430: [WinError 1225] The remote computer refused the network connection

Can someone tell me why this happens?

[cerulean onyx]

update: i identified why this error happens

when the server is sleeping and tasks try to read data from sleeping server, error happens

but i cant fix it

any ideas?

[modern ravine]

Pretty sure you may have asked yourself those Qs already but just a dumb Q: why does the server sleep and is there a timeout on it? What can you do to keep it from sleeping?

Or maybe the code needs a trigger to wake it up

[cerulean onyx]

Server should sleep as it's given in task description, I guess whole problem was to deal with sleeping servers with asynchronous tasks somehow

Bh the way i fixed it

Took a while but came up with xD solution

If tasks don't connect with server, then it means server is sleeping so, i make tasks sleep to and retry connection after some intervals over and over again until they connect with servers xD

Dunno if it's the best solution but it works xD

[ember ledge]

¯_(ツ)_/¯

[wicked salmon]

I recently made a simple TCP chatroom, that you can message each other in the terminal. My question is, how can I have other users use my tcp chatroom from a different network?
I was originally thinking of hosting my tcp chatroom on some cloud provider but am completely lost on how another user can use my chatroom from a different network.

[open isle]

The cloud provider machine will need to have a fixed public IP that your clients can connect to

Most hosting cloud services should come with one, sometimes for free, sometimes for extra

[weary prairie]

Can someone help in Ubuntu ? I am operating 20.04 version, and I bought a package with 4 IPs. However, I don't know how to use them since they give me access to only 1 IP and the rest is unmanaged. Someone care to help here to activate the other 4 IPs ?

[wind radish]

Why do you need 4 IPs for one machine?

[weary prairie]

It was a sweet package and I bought it. I need for instance 2 IPs for sending mails so that I don't get blacklisted

[humble wing]

does anyone know why my rdns address gives no response?

do i need ports 25 and 465 open?

the mail ports

background info, im trying to install wazuh on a hetzner cloud instance

[humble wing]

kms

[wind radish]

If you will be blacklisted on one IP, you could be blacklisted on 2nd as well

You probably can configure one IP on a single network interface, then switch upon need.

You also could configure multiple IPs on a single network interface. It is very possible. However, you will also need to read LARTC (https://lartc.org/), configure policy-based routing, and become Linux expert admin to use such configuration.

[quaint token]

        except CannotSendRequest as e:
            logger.exception("Couldn't connect to server")

            if len(e.args) > 0 and e.args[0] == "Request-sent":
                # try reconnect
                logger.info("Reconnecting...")
                self.connect(self.hostname, self.port)

            return None

I'm trying to catch the CannotSendRequest exception and stop it from printing a stacktrace. Unfortnately, it still prints a stacktrace even though I'm catching it. Does anyone know how what to do?

[round parrot]

Any chance I could find a CFM packet implementation for scapy, so I don't have to write it from scratch?

[round parrot]

I assume not, so I have some questions

I'm trying to add padding while crafting a custom layer packet based on the length of multiple other fields
e.g.

field2 = "bbb"```

in this case, I want add padding so the total number of bytes adds up to 10, so ```aaabbb0000```

It looks so easy, but I can't really figure it out without rewriting a bunch of stuff

[inland rampart]

Add up the fields and then .ljust(10,b"0")?

[empty iron]

i can't get socskets to work for some reason don't know why #networks

[quaint token]

I'm not really sure what you're trying to do. If you want to send a packet with a fixed size over tcp sockets just create a byte buffer with that size, put your data in and send it. the unused bytes wll be zeros

you didn't mean this: https://en.wikipedia.org/wiki/IEEE_802.1ag with CFM, right?

[round parrot]

I did mean that CFM, yes

I don't just want to send the packets, but to describe the layer as a new scapy packet, so I can dissect/craft the packets

this is below TCP, there's no TCP at this level, only ethernet 🙂

[quaint token]

geez you are doing something this low level in python?

[quaint token]

hmm I'll give it a read since Im writing an exam about computer networking in like two weeks anyways, but this is not something I'm familiar with unfortunately

[round parrot]

yes, it's quite a niche topic. But the issue I described is a more generic scapy thing, not CFM only. So it would work with higher level packets as well

[quaint token]

Maybe ask in a more specialized place, I'm sure there is somne kind of networking subreddit/forum where you might get an answer

[coarse summit]

you will need to elaborate

[ember ledge]

Does anyone know how I can create a wifi hotspot in Python? Sort of like those wifi networks were you watch adverts in order to get more mb to browse more?

[mint tinsel]

Hello bosses 👋🏾

Please has any one implemented OSPF routing for an SDN with mininet and ryu?

I want to learn how to do so

I'd appreciate if you could share useful resources (articles, sample implementations etc)

🤧✨

[scenic flower]

 
from selenium import webdriver
import os
os.environ['PATH'] += r"/home/needjobcoder/Documents/Development/Web/webPythonProject/selenium/selProject/chromeDriver
"
driver = webdriver.Chrome() ```

does this add to path each time when it is run ?

[wind radish]

it doesn't do export, so it will affect current process only

[inland rampart]

(export isn't global either though, it only affects the process and its children. but yes, this doesn't do lasting changes.)

[umbral steeple]

import socket
s = socket.socket()
s.connect(('127.0.0.1', 9001))
s.send(b'Hey!\n')
data = s.recv(4096)

Can anyone explain what this does? I'm new to socketing
Specifically the s.connect() line

[wind radish]

@umbral steeple it is explained here:
https://docs.python.org/3/library/socket.html#socket.socket.connect

[prisma cobalt]

when you call socket.connect() you attempt to establish a bridge between your socket and a server socket on the other end (in this case, you have a local server running on port 9001). The call will block until it succeeds or errors (which you will need to catch).

Overall, the script will create a socket, connect it to your local server, send "Hey!\n" and then store whatever the server sends in the data variable

[umbral steeple]

thank you

[umbral steeple]

thank you

[sullen sphinx]

Hi everyone I live in souther California and am new to learning to code. Working through Angela Yu and freecodecamps python courses to get started. Can anyone recommend any meetups or conventions that would be cool for me to attend?

[zealous flint]

Hi! This channel is for computer networking rather than social networking - the mixup happens a lot so not your fault 😛 (we should probably change the channel name haha)
Python had a fair bit of conferences and meetups, the last one was EuroPython for the European folks. #python-events will generally keep you up to date on these

[molten abyss]

Does anyone know what versions of socketio, engineio, and flask-socketio I need to make 'em compatible? I have tried:

pip install --upgrade python-socketio==4.6.0

pip install --upgrade python-engineio==3.13.2

pip install --upgrade Flask-SocketIO==4.3.1

and

pip install Flask-SocketIO==5.1.1 python-socketio==5.4.0 python-engineio==4.2.1

The client is using an unsupported version of the Socket.IO or Engine.IO protocols (further occurrences of this error will be logged with level INFO)

[wind radish]

Check nearest cities here
https://www.meetup.com/topics/python/
https://wiki.python.org/moin/LocalUserGroups
https://datalab.ucdavis.edu/davis-python-users-group/

[sullen sphinx]

@zealous flint lmao shit.

@wind radish thank you for the recommends I’ll check it out

[limber stump]

hello all, have anyone used nornir ?

[radiant vortex]

Hi Guys! I need some advice doing basic network diagnostics on my PC with a python program. here is what i'd like to be able to find out:

No internet connection already able to do with by loading a website (try and except) but need to know why I cant load website:
No wireless network connected - Already able to do with with OS and CMD.
No ethernet cable connected to PC?
No ethernet cable connected to router?
**Router offline? **
PC in aeroplane mode?
Any other reasons there is no Wifi?

If anyone could point me in the right direction if these are all possible and what libraries or how to do that, I'd appreciate it greatly!
Thank you!

[ember ledge]

https://denizhalil.com/2023/07/28/privacynet-anonymization-tool/

[ember ledge]

Hey there! I am currently pursuing a degree in bioinformatics..I was looking for someone in the same field or in the data science domain in order to participate with..to solve an extremely challenging and interesting problem statement available on Kaggle..would love if anyone would want to collaborate..looking forward to the same!!! Do DM if u r an enthusiast too!

[arctic galleon]

Hello there, i have a problem with my domain got a domain with arecord set on noip with the duc now i got my minecraft server with port 25565 open and if i try to join the server with the domain on my network it works. but if someone else wants to they cant. im pretty sure i configured everything correctly in noip but could be wrong. i also diasabled firewall and put the server in dmz nothing seems to work. if anoyne has time would be nice if i can recieve help been strugling with this for 2 days and its pain

[arctic galleon]

alr all good

my friend was just fucking with me

[glossy walrus]

Hello, Bony from Kenya. I’m looking for anyone new to Python that has probably started #100DayOfCodeChallenge so we can monitor each other’s progress. Thank you.

[lucid widget]

how to send response to previously opened socketio connection?

can socketio connection opened on server side

for example server sends me something without i requesting anything from it

[edgy pelican]

example?

[lucid widget]

what example

[edgy pelican]

what do you mean

what od you want it to do

[lucid widget]

socketio.open("ws://ip")

[edgy pelican]

how should it work

[lucid widget]

server connects to me

not me

to server

[lapis badger]

Hi guys sorry if this is the wrong channel, but i have a question.
I want to make a python script which downloads music from SoundCloud and stream it to a plain website. The download part is done, now i just need to stream the audio. But i don't really know how to approach it
Thanks guys for the help! 😄

[errant bay]

:incoming_envelope: :ok_hand: applied timeout to @real compass until <t:1690639432:f> (10 minutes) (reason: duplicates spam - sent 4 duplicate messages).

The <@&831776746206265384> have been alerted for review.

[frigid osprey]

Guys does anyone know if replit still support the pinging thing to keep a host 24/7 online?

[quasi widget]

!rule 5 thats Not allowed

[errant bay]

Rules

5. Do not provide or request help on projects that may violate terms of service, or that may be deemed inappropriate, malicious, or illegal.

[frigid osprey]

My question is nowhere malicious or illegal

nor inapporopriate

[quasi widget]

Doing so is though

[coarse summit]

"How do I rob a bank"

[ember ledge]

idk but u can try and see

[dull mist]

How can I connect to my raspberry pi outside of my home network without port forwarding?

[coarse summit]

Why can't you open ports

?

[dull mist]

Parents haven't given me wifi admin

But I asked gpt and it suggests CloudFlare

[coarse summit]

You could searche the documentation on your router

And see if the password is still the default

It also may be on the back of the router

[dull mist]

No like my parents don't want me on admin page

But I found solution, thx anyway

[coarse summit]

Np

[ember ledge]

https://denizhalil.com/2023/07/31/osi-model-computer-networks-data-communication/

[modern spear]

is anyone here familiar with http/requests/post and the requests module

[thick heart]

running into annoying issues with circular imports with flask, and i don't wanna structure around their blueprints. looks like my other alternatives are mostly Quart, FastAPI, and Sanic framework. can some folks with strong opinions share them?

(maybe this would be better in the #1035199133436354600 forum, idk. let me know if i'm in the right place)

[gloomy root]

you will likely have the same issue with all of those

switching framework doesnt really solve the whole cirular import issue

[pastel cedar]

wild https://www.reddit.com/r/ReverseEngineering/comments/it7yv7/http_toolkit_opensource_tool_with_oneclick_mitm/g5d3k38/

i came across this tool that's "100% open source" and got curious about their subscription model, and why people dont just bypass it. turns out the creator themselves pointed out the exact part of code to modify it to blank out the "pro" paywall. based

[crystal current]

Probably a lot of people... Don't ask to ask, just ask

[pastel cedar]

oh oops this is a python channel still

[modern spear]

thats the exact tool im using, where do i go to fork it so its working?

he said to put return true here but i dont know exactly where

[pastel cedar]

let me get back to you once i successfully build this

i got distracted

[pastel cedar]

ok tbh building this has revealed its like very very outdated and using a ton of depracated stuff. so in the end other stuff like fiddle or mitmproxy feels better

[modern spear]

thanks for telling me

you have to pay for fiddle though😂

[pastel cedar]

no fiddle classic is free

[modern spear]

well im using the http listiner for a mobile device

and httptolkit was teh only thing that had a aandroid option that ive found

do you know how i would connect the proxy to a android emulator

[pastel cedar]

see if this helps https://mitmproxy.org/

android emulator is on windows right?

idk how that works tbh

[modern spear]

yes

memu play it makes a virtual phone on your machine

[pastel cedar]

nice then just use that mitmproxy program. you dont even need to go as far as installing root certs just run mitmdump
and set your windows proxy to use localhost:8080

[kindred briar]

I wanna setup a testbed of wireless mesh networks, to test different routing protocols like BATMAN, AODV, OLSR babble. Currently I have a single router on which I flashed Openwrt and installed BATMAN. I want to create more mesh nodes. Does anybody have any cheap but reliable hardware in mind?

[fiery kernel]

has anyone worked with Qualys API? Im using a post request to download their whole KB but the response is really big and causing the script to crash...it works fine on my pc but hit and miss on this aws VM...wondeirng what i can do with requests to make it more reliable

[crystal current]

What script? Are you following their sample code? https://blog.qualys.com/product-tech/2021/03/02/qualys-api-best-practices-knowledgebase-api

[ember ledge]

can someone in here checkout #1136762867652239580 its related to networking

[molten prism]

hi, i'm asking this because i've seen answers but they don't explain enough

how do i make a wget-style progressbar with downloads

(using urllib.request)

[summer canyon]

If i want to learn networking and use it with python, where should i start?

[wind radish]

@summer canyon Depends on what exactly you want to learn...
If you want to understand networks in general, read Tanenbaum "Computer Networks"
If you want to deeply understand how TCP works, read e.g. O'Relly "TCP/IP Network Administration" book
If you are not interested in theory at all, and you just want to program in Python, do socket programming

[kindred briar]

I was using wireshark to mointor traffic, and Ive made a bunch of custom filters that I wanna look for during normal operation, I would like to process all the incoming packets in real time and use it in backend for my django based backend. How do I apply multiple filters at the same time on the incoming traffic and output the results in an organized manner.
you can apply filters in the command itself but I wanna capture all the traffic and then flag whatever strikes
Any thoughts ?

[kindred briar]

lol what haha

[silent imp]

am trying to send a post request with cookies, headers , etc using urllib.requst
but I'm not getting the same response when using requests

[ember ledge]

k

maybe u could give a code

[silent imp]

import json
import ssl
import http.cookiejar
from urllib.request import urlopen
from urllib.error import HTTPError, URLError
import urllib.request
from urllib.request import urlopen, Request

ssl._create_default_https_context = ssl._create_unverified_context

def make_request(url, headers=None, data=None, cookies=None):
    request = Request(url, headers=headers or {}, data=data or {}, method='POST')
    try:
        cookie_jar = http.cookiejar.CookieJar() 
        for cookie_name, cookie_value in cookies.items():
            cookie = http.cookiejar.Cookie(
                version=0,
                name=cookie_name,
                value=cookie_value,
                port=None,
                port_specified=False,
                domain='www.example.com', 
                domain_specified=True,
                domain_initial_dot=False,
                path='/',
                path_specified=True,
                secure=False,
                expires=None,
                discard=True,
                comment=None,
                comment_url=None,
                rest={'HttpOnly': None},
                rfc2109=False
            )
            cookie_jar.set_cookie(cookie)

        opener = urllib.request.build_opener(
            urllib.request.ProxyHandler(
                {'http': '',
                 'https': ''}),
            urllib.request.HTTPCookieProcessor(cookie_jar))
        resp = opener.open(request, timeout=10)
   except Exception as er:
        print(er)

[ember ledge]

u dont need to put auth :/

so we all can see 😛

[silent imp]

it's required argument

[ember ledge]

ikn

[silent imp]

lmao

[ember ledge]

and req

[silent imp]

?

[ember ledge]

and requests code

[silent imp]

it's like basic post request

import requests 
cookies = {}
headers = {}
json_data = {}
r = requests.post(url, headers=headers, cookies=cookies, data=json_data)

[ember ledge]

idk withput testing

but try verify=False in requests

[silent imp]

same thing

[silk plaza]

Help

[edgy pelican]

wrong channe

[frigid vault]

Does anyone know how to make a minecraft bot that joins servers in python?

Like how to construct the packets and send them.

[coarse summit]

wiki.vg

I used twisted for the client

[frigid vault]

I know the wiki, but I don't understand how to make the packets based on it.

[coarse summit]

Wdym by that

Try checking out twisted for the client, you don't need to worry about constructing packets

You could also use socket library for that

If you really want to construct them yourself check the scapy library

[frigid vault]

I mean I don't want to make a client, but just the network part.

[coarse summit]

It's also called a client

[frigid vault]

I mean I know what packets I have to send, but I don't know how to make those packets.

[coarse summit]

My recommendation is using socket or twisted for not constructing the packets yourself

You just send data though the socket and it will create a packet automatically

[frigid vault]

any twisted libraries?

[coarse summit]

Twisted is a library

https://twisted.org/

Very useful

[frigid vault]

But I think I still have to make the packets. I don't see how the libraries makes the packets.

[coarse summit]

https://docs.twisted.org/en/twisted-18.9.0/core/howto/clients.html

[frigid vault]

Isn't it the same as socket lib?

[coarse summit]

You create a Clas for your cliente extended from the protocol class

[coarse summit]

It is built on top of the socket lib

[frigid vault]

Right. I will send the packets with twisted, but how to make the actual packets, because I see that I still have to make them.

[coarse summit]

Are created and sensed automatically when you use the data

[coarse summit]

Check this

[fading ruin]

Hi, I have a quick question, would a python http.server work connecting my server with a vpn, like hamachi?

[frigid vault]

I think if you set it up. Yes.

[frigid vault]

I found quarry that does it for me.

[coarse summit]

It is very outdated though

[frigid vault]

I used it for 1.17.1 server and it worked.

[wild dove]

Hello

How are you @frigid vault

I am new member.

[ember ledge]

This channel is not for social networking, it's for

Discussion on network protocols, technologies, hardware with relation to Python

[lucid widget]

im using this library https://github.com/gabrielbull/php-websocket-client wss://site.com does going to this url sends cookies automatically how?

[red crow]

hi am having prpoblem with deploying a django to server, using nginx and gunicorn. i am getting bad request 400, i am trying a lot to solve this but it is not solved yet

[civic fog]

No clue where to put this but im trying to send an attachment (smsmessage.txt) through email with python but it sends a corrupted file that is not the one im trying to specify

import smtplib from email.mime.multipart import MIMEMultipart from email.mime.text import MIMEText from email.mime.base import MIMEBase from email import encoders mail_content = '''''' #The mail addresses and password sender_address = '[e-mail]' sender_pass = '(MY PASSWORD)' receiver_address = '(MY PHONE NUMBER)@vzwpix.com' #Setup the MIME message = MIMEMultipart() message['From'] = sender_address message['To'] = receiver_address message['Subject'] = '' #The subject line #The body and the attachments for the mail message.attach(MIMEText(text, "text" )) attach_file_name = 'smsmessage.txt' attach_file = open(attach_file_name, 'rb') # Open the file as binary mode payload = MIMEBase('application', 'octate-stream') payload.set_payload((attach_file).read()) #add payload header with filename payload.add_header('Content-Decomposition', 'attachment', filename=attach_file_name) message.attach(payload) #Create SMTP session for sending the mail session = smtplib.SMTP('box.courvix.com', 587) #use gmail with port session.starttls() #enable security session.login(sender_address, sender_pass) #login with mail_id and password text = message.as_string() session.sendmail(sender_address, receiver_address, text) session.quit() print('SMS Sent')

[crystal current]

#❓｜how-to-get-help

[ember ledge]

Hi does anyone offer 1 on 1 guidance ? If yes dms are open 💕

[fathom walrus]

Any reading suggestions if I want to understand the theory of how localhost works? i.e., localhost seems to make a computer behave like if it was itself a network, how exactly is this possible and how is it happening?

we can for instance host on one same computer multiple servers that communicate with one another using the same protocols they'd use to communicate with physically distinct machines

[shrewd sparrow]

re: pygen
So at the highest level, when an IP addressed packet reaches whatever packet routers are managed by my ISP, my current understanding says they would strip the Ethernet frame (which was addressed for the ISP), strip the IP frame to parse the destination IP, then wrap the bare TCP frame in a new IP frame and send it on its way?
What I'm confused about is what level the packet is at when it reaches the ISP? Does it have an Ethernet frame at that point or just an IP frame? What kind of IP mappings does the ISP maintain?

[hoary sorrel]

there's always a TCP frame inside an IP frame inside an ethernet frame

your router removes an ethernet frame that says that it is the destination, and replaces it with an ethernet frame that says that (a router at) your ISP is the destination.
The ISP's router removes the ethernet frame saying that it is the destination, and replaces it with an ethernet frame that says that the next hop is the destination.
And every hop along the way repeats this until it reaches the actual machine it's destined for

[shrewd sparrow]

I offer a link I have referenced in the past: https://tldp.org/LDP/nag/node65.html#SECTION007700000

Very old but the only real thing that's changed is the syntax to the ifconfig/ip command afaict

[fathom walrus]

will get on that, thank you!

[hoary sorrel]

there's no particular magic involved here. You can mail a letter with your own home address as the destination. A computer can send an IP packet with its own address as the destination.

there might be some optimizations in place, though. Your mail carrier might realize that you addressed a letter to yourself, and stick it right back in your mailbox without ever taking it back to the post office. Similarly, your OS might realize that you addressed an IP packet to yourself, and skip some of the work that it might need to do if it was sending that packet to a different computer.

and "localhost" is just a name that always means "this computer"

[shrewd sparrow]

I think this question indicates that what I really want to know is "a general overview of how packets move" so like if this is something I ought to find a textbook for instead lmk lol.
Is there any intuition for when a router will choose to route by IP vs routing by MAC address then? My previous understanding was that the IP frame would hold the ultimate packet destination (sans NAT): so does the ethernet frame hold the "next hop"?

[glossy hull]

I just made this real quick

[hoary sorrel]

it always does both (well, at least for IP packets). It looks at the destination IP address in the IP frame to decide what MAC address to send the packet to - which becomes the destination MAC address in the ethernet frame

[shrewd sparrow]

I see. The router does that IP lookup in the local ARP cache. And then it fills the returned MAC address into the dest field of the Ethernet frame. And then whatever hardware (driver) or switch the packet is currently in takes that MAC/IP pair and, well, routes it

[glossy hull]

exactly, well switches usually just work on the MAC

layer 2 vs layer 3 switches

[shrewd sparrow]

Right, I understand that jargon now ^^

Layer 2 switch would be something like my router/modem box, right? Because it doesn't have ARP cached MAC data for my ISP servers, it routes it by doing funky stuff over IP?

Or would the layer 2 switch be the one that the ISP is running, doing the top level IP translation?

[glossy hull]

layer 2 switches are usually used on LAN networks to get more ports, think of them like USB hubs

[shrewd sparrow]

So this guy's a layer 2? Grey cord goes into the router

[glossy hull]

most likely yes

[shrewd sparrow]

So that's why it's the case that devices plugged into the switch are actually assigned IP addresses and accessible over WLAN -- it's all about the layer the switch is operating at?

Damn maybe skipping the networking class in uni is coming back to hurt me 😭

[glossy hull]

so getting assigned IP addresses is done with DHCP (most likely a DHCP server on your router)
basically everything plugged into the network is shouting out "Hey can I get an IP please!" (even tho the system doesn't have an IP it does have an IP packet, setting the source to 0.0.0.0 and destination to 255.255.255.255, meaning "send this to everyone on the network"), and then the DHCP server replies by sending a packet to the clients MAC address.

When it comes to accessibility all the switch is really doing is giving you more ports (at least layer 2 switches), the router will keep track of a table like this to know where each device is (storing a mapping from port to MAC):
| Port | devices |
| WLAN | Device1 |
| Eth0 | Switch, Device2, Device3 |

and the switch will keep track of something similar:
| Port | devices |
| Eth0 | Router |
| Eth1 | Device2 |
| Eth2 | Device 3 |

[shrewd sparrow]

Wait... DHCP working through IP layer multicast is really cool. And I did not even consider that without the source MAC address the router has no way to respond to the DHCP request at all

[glossy hull]

well you could still respond in theory, include some ID code with the request, then the DHCP server also uses the broadcast adress and the ID in its response (althought MAC is ofc better)

[shrewd sparrow]

If that is all the case, how does DHCP over VPN work? Does a VPN just simulate the ethernet-level MAC addressing as if all the devices--

I see, I was in the middle of typing the message when you noted alternative IDs are possible

[glossy hull]

possible yes, implented? I dont think so

[glossy hull]

this is also why having multiple DHCP server can completely fuck over a network, because it is basically random which response you are gonna get first

[hoary sorrel]

moreover, they don't know what addresses have been assigned by the other one (assuming they're both assigning addresses in the same range)

[glossy hull]

exactly

[shrewd sparrow]

So if I wireshark'd my VPN connection as I was spinning up my conn, would I just see all the same ARP identification packets and DHCP multicast requests as I do on my regular old LAN?

(Assuming the VPN gateway is running a DHCP server)

[hoary sorrel]

I'm not sure - that's a bit past my depth. I'd expect not, since VPNs are - well, virtual, I'd expect them to be able to be set up in a higher level way

[glossy hull]

yhe I am not terribly familiar with VPNs, although they usually setup a virtual network card, so they can do a lot of that in software, and might use their own protocols for it

[shrewd sparrow]

At least in the case of the one I manage, that is indeed the case, it's registered as a physical iface

But it's standard dhcpd

[glossy hull]

also there are many types of VPNs

[shrewd sparrow]

That is fair. I'm sure at the very least a mesh network type VPN has its own protocols lest a connecting client just screams into the void asking its two other p2p clients for an IP address

[prisma cobalt]

I’ll just note, after a packets leaves your home network, arp is no longer used to route the packet. Imagine every packet requiring the router to ask all other routers that it’s connected to if they know where to send the packet lol, it’s not feasible. Each router maintains a mapping of ip addresses and destinations but it’s a particularly big list relative to the size of the internet. If the destination ip in not in the list, the router forwards the packet upstream, this goes on and on until it reaches your isp infrastructure which contains some of the biggest routing tables on the internet. The packet is then routed to the correct isp and then goes down the router chain to the target network.

[hoary sorrel]

(and ARP isn't used for discovering which what the best path from one network to another is - instead a protocol called BGP is used for that)

[shrewd sparrow]

Gotcha, that was part of what I was confused about. So in order for your home router to submit a packet to the ISP, it simply submits a packet to a known host that is "closer" to the ISP.
Is there a protocol to keep those known-good host lists in sync?

(Is that the aforementioned BGP?)

[hoary sorrel]

https://www.practicalnetworking.net/wp-content/uploads/2016/01/packtrav-l2-vs-l3.gif

[glossy hull]

side note, to try and get some more details on that VPN question I found a youtube video which went into it a bit techincally

your vpn does not keep logs on you or track you
dude, that is not a feature of a vpn, that is just companies telling you they dont

[shrewd sparrow]

Lolol. Well my VPN better be keeping logs, I told it to!

[hoary sorrel]

your router routes everything to a router owned by your ISP. That router knows what paths to take to reach other networks because of BGP

[prisma cobalt]

So when you send off a packet, it’ll go up the router chain and then down once it encounters a router high enough in the chain that it’s common to both the sender and target device

I’ll draw a diagram

[glossy hull]

30 seconds later:

if a vpn is keeping logs on you
I thought you said it didnt 🙃

[shrewd sparrow]

How does that top level router knows that the destination is in its list of known hosts? Doesn't the top level router "fan out", so this would have to be like a DFS BFS?

Unless it's as simple as "yeah the DFS BFS is just on a scheduled job" and it just periodically asks all its nodes what nodes they node, and it just aggregates all that data up

[prisma cobalt]

an ISP will purchase a block of addresses, they can then assign these addresses and keep track of how to route to these addresses I believe

since they provide the infrastructure, they can maintain their routing tables

[hoary sorrel]

that's BGP

[shrewd sparrow]

Ahh. So it all comes together

[prisma cobalt]

if a packet reaches the top of the chain (an NSP backbone server) then BGP or suchlike (there are alternatives) is used to route the packet

[glossy hull]

cant believe what I learnt in school is becoming useful.... is teaching others "useful"?

[prisma cobalt]

we picture packets as hop by hop which is correct but you also have to picture an "up and down chain"

[hoary sorrel]

teaching others is one of the best ways to realize gaps in your own knowledge!

they have a way of asking questions you struggle to answer, or to answer succinctly and coherently 😄

[shrewd sparrow]

I see from https://www.cloudflare.com/learning/security/glossary/what-is-bgp/ that IANA maintains a master list of ASNs corresponding to known-good BGP enabled systems

[glossy hull]

certainly! I realised I have no clue how vpns work on the low level

[hoary sorrel]

yep, ditto, hehe

[fathom walrus]

applications listen to IP/port pairs and not only to ports?

[hoary sorrel]

yes

[shrewd sparrow]

This answers my question of "well yeah, but how does the ISP know what to do if they don't own the block of the destination IP"

[prisma cobalt]

it depends on the application, some listen to addresses regardless of port but none can listen to just a port

[hoary sorrel]

the arguments that you provide when binding a socket are an IP address (or a wildcard meaning "all possible IP addresses") and a port.

[fathom walrus]

I see, ty!

[shrewd sparrow]

I mean, you can really muck up your iptables config to bind to a port regardless of source/destination IP address (source: oops! all my traffic is suddenly getting clobbered through my gateway)

[fathom walrus]

Suppose IP1, IP2 are two IP addresses. Is there a conflict if application1 is listening to the pair IP1 with port 50000 and application2 is listening to the pair IP2 with port 50000?

[prisma cobalt]

no conflict

those identify as two different endpoints

[fathom walrus]

I see, ty!

[quartz frigate]

Each host can only host one thing per port at a time.

[hoary sorrel]

not true

[quartz frigate]

Oh?

[hoary sorrel]

each host can host one thing per port per ip address at a time

[quartz frigate]

I....
Huh

[hoary sorrel]

each ip address has its own set of bindings (though one call to bind can bind to multiple ip addresses with a wildcard)

[shrewd sparrow]

is the bound IP not the "host" in the jargon

[coarse summit]

per socket?

[shrewd sparrow]

so you're both right ^^

[prisma cobalt]

cant raw sockets be bound multiple times for multiple applications to listen at the same time?

[quartz frigate]

Or 0.0.0.0, right?

[hoary sorrel]

a socket can be listening on multiple addresses

[hoary sorrel]

that's the wildcard that I meant

[quartz frigate]

ic

[shrewd sparrow]

afaict no.... unless you fork()

[prisma cobalt]

running wireshark (which listens for traffic accross every port) wont block the traffic from being delivered to a different program on the server

[hoary sorrel]

that's true, but I don't think wireshark works by binding sockets

[shrewd sparrow]

wireshark puts the actual network card into that weird sniffing mode though
promiscuous mode?

[hoary sorrel]

right

well - yes and no

[prisma cobalt]

this is because of the level of the binding, you dont need to bind a socket at the transport layer

[shrewd sparrow]

it doesn't have to do that but it can

[hoary sorrel]

promiscuous mode is about whether it receives packets that were intended for other hosts, not whether it receives packets that are intended for other ports

[fathom walrus]

I can have an application outputting to localhost:50000 and another listening to this pair without a conflict, right?

[hoary sorrel]

most people will only ever bind to 127.0.0.1 or to 0.0.0.0 when binding a TCP socket. They're by far the two most common options: listen on all of this machine's public addresses, or listen only on the loopback address.

[hoary sorrel]

what do you mean by "outputting to"?

[shrewd sparrow]

I see wym
I would be curious to see how libpcap works then, it's gotta be doing some magic on the TCP layer

[fathom walrus]

sending packets to? I'm not 100% sure if this is the main way applications send things through IP/port pairs

[prisma cobalt]

yes, that'll work

if you have a server listening on port 50000, aa different program can send traffic to that address

[hoary sorrel]

TCP sessions are identified by 4 numbers: source IP address, source port, destination IP address, destination port

a client on 127.0.0.1 can connect to a server bound to 127.0.0.1:50000 just fine, though there will be a different (randomly assigned) source port for that connection

[fathom walrus]

I see, thanks for the answers!

[hoary sorrel]

(just in general, outgoing connections are assigned random port numbers)

[shrewd sparrow]

Do incoming and outgoing ports conflict?

Err, ports bound for listening and ports bound for sending

[hoary sorrel]

they're drawn from the same pool

you can't have two things listening on 127.0.0.1:12345 (one server waiting for incoming connections, and one client waiting for a response to a message it sent)

[shrewd sparrow]

Yeah, but can you have one listening for conns w/ dest port :12345 and one sending with the src port :12345?

[hoary sorrel]

no, because responses come to the port you sent from

if I start a tcp connection to a server on 127.0.0.1:50000, and I'm assigned source port 12345 for that, then my client is the thing bound to 127.0.0.1:12345. I can't also have another server bound to 127.0.0.1:12345, because if I did the kernel wouldn't know whether to give an incoming packet addressed to 127.0.0.1:12345 to my client or to the other server

[prisma cobalt]

are you sure? the router should be able to distinguish the connections because the session as you said is based on these 4 numbers

[hoary sorrel]

I'm... fairly sure.

[shrewd sparrow]

!!! Found the real deets

https://stackoverflow.com/questions/3229860/what-is-the-meaning-of-so-reuseaddr-setsockopt-option-linux

It's a socket option

[prisma cobalt]

that option just allows the binding of a socket to a used address before its "cooldown" period

[shrewd sparrow]

At least, for allowing re-binding of sockets that still might have open TCP sessions. Close enough

Look sifting through man pages is hard 😭

[prisma cobalt]

lmao

very true

[fathom walrus]

so say I have a program that sends lowercase strings and a server that capitalizes them, then returns it to a program that can read it. the server is on the same physical machine. Then the structure of this whole apparatus is two connections as such:

localhost:port1 -(lowercase string)-> localhost:port2; localhost:port3 -(uppercase string)-> localhost:port4, where my server listens to localhost:port2, then establishes a connection to localhost:port4 with source localhost:port3? is this correct?

[hoary sorrel]

yep, that sounds right

[fathom walrus]

i see, thanks for the help!

[prisma cobalt]

just tested, no errors :)

[hoary sorrel]

interesting. huh. well, fair enough then

[shrewd sparrow]

https://lwn.net/Articles/542629/ There we go

SO_REUSEPORT allows ephemeral reuse of listening ports as long as the listening application starts first

Cool stuff.

I mean, afaict, it actually just straight up allows N listeners too

[hoary sorrel]

that was traditionally done by forking instead

one process would bind the socket, and then fork N times, and then you'd have N processes that could accept connections on that socket

[prisma cobalt]

oo interesting, multiple threads listening to the same port at the same time

[shrewd sparrow]

This is across process boundaries though

Err, so is forking

But this is across execution contexts

[hoary sorrel]

sure, I'm not saying this isn't a new capability

just giving some flavor about what you would do without it (or on platforms that don't have it)

[shrewd sparrow]

Fair fair

[hoary sorrel]

the model where you bind to the socket, call listen, then fork, then call accept in the (each) subprocess is called "pre-forking"

it's a common web server pattern, there are lots of pre-forking web servers

[shrewd sparrow]

I've heard that jargon, but whats the alternative? Accepting in a main process then forking per-request?

Sounds expensive

Or is pre-forking simply The One Way To Do It

[hoary sorrel]

yep

[hoary sorrel]

it's a pattern exactly because it has less per-request overhead

@shrewd sparrow You should check out http://www.practicalnetworking.net/series/packet-traveling/osi-model/#osi-layer-23 btw

[shrewd sparrow]

Bookmarked, thank ya!

[hoary sorrel]

👍

I think it'll cover a lot of what we talked about today, in a bit more depth

[scenic adder]

Could someone explain to me what a VPC does? I just learned of it and my web host wants me to improve my database security by implementing that. I have no idea what a VPC does though and how it would help my database security.

[frozen drum]

a VPC is a logically isolated section of resources only for your use - basically you don't share hardware with other people running other virtualized resources

it can be a separate network, completely separate hosts, or anything else really.. it's kind of a vague marketing term

[scenic adder]

So then is the process of accessing that like using an ssh client or something?

[frozen drum]

no, as I said it's really vague.. depends what their definition of a "private cloud" is

[scenic adder]

Ah okay. Thank you. This clears it a bit up for me

[wind radish]

What is VPC? It is a network without public addresses, but also in the cloud

[nocturne tulip]

👋 idk if i'm on the appropriate channel. I'm implementing a solution where multiple threads are writing to a unix socket then they wait for answer from server before continuing their process. What's your recommendation to do this? Is it a problem to create a socket object and call connect method on each thread?

I'm a bit confused with all solution possible (using EpollSelector, Streams with asyncio, etc)

[wild dove]

How to stop running python script using common javascript

[wind radish]

wat

[civic fog]

tried that

what do i put for format(attach_file_name))

and

attach_file = open(attach_file_name, 'rb')

[crystal current]

Doesn't seem possible unless your Python has an API that would allow this (and I've never seen such a thing in practice)

[ember ledge]

first answer yes. second depends, most home internets have dynamic ip, but it also dependes of ur isp how often they change it. Could be every 24 hours, or cople of days to weeks or after every reset of router

third answer also yes

[inland rampart]

usually a static IP is a toggle you can check in your account on your ISP's site, often one you have to pay extra for.

[wind radish]

not for cell internet though

[void raptor]

hi, @crystal current

[jovial gate]

I'm trying to fix a bug in a Scapy dissector, but I'm struggling to understand the deep internal structures of Scapy, even with reading the docs.

I've managed to get the packet layer to process properly in my print statement, but there is still a bunch of weirdness with the underlying payload

And also, somehow, I've made a len(variable) respond with an expected value of 99, but my debugger reports that it's size 132 (the original incorrect value)

Long story short, Scapy correctly identifies the ADU and PDU, but then incorrectly reads the length of the PDU, overlapping into the second ADU/PDU of the same layer.

I'm pretty sure I corrected this by assigning length_from=lambda pkt: pkt.byteCount,, and now the field is the correct length according to the len() function, and the print() function shows only the expected values

But the payload is still entirely consumed by the first PDU, and the variable prints a count of 99 words, but is still the incorrect size of 132 words in memory

....I think this is more complex to explain than I realized and I should start a thread for it.

[jovial gate]

A link to a more thorough summary of my problem: https://discord.com/channels/267624335836053506/1140747269751177347

[wild dove]

Hello

I built a flask server locally.
My project is Laravel + javascript + flask.
I am sending a request request in javascript.
I want to stop the execution of the currently running api on the server.
How can I implement this?

[jovial gate]

Thanks to locating an old bug report in how scapy handles BGP, I learned about the PacktListField: https://scapy.readthedocs.io/en/latest/api/scapy.fields.html#scapy.fields.PacketListField

This seems to be the exact solution I need. Now I just have to make it work...

[tiny hatch]

hit this interesting error trying to deploy an app using Google cloud and WSL.
I was planning to use ray to distribute a training workload to multiple computers on different networks, but due to the problems with internal and external IPs I have gotten very confused.
I am running my head node on Google Cloud, with most ports open (for testing). I installed python 3.10.6 and ray onto this server and started the head node with ray start --head --port=6379 --node-ip-address=34.71.183.XX. This server works fine, and I can run ray applications locally on the server.
the issue occurs when I try to connect to this ray head over a WSL instance running on a windows laptop: ray start --address=34.71.183.XX:6379 --node-ip-address=172.31.86.XXX
the node IP being the IP address that my WSL instance appears to be running (according to ip addr show eth0). this command 'works' but throws the warning:

details.2023-08-15 12:52:33,971 WARNING utils.py:1401 -- Unable to connect to GCS (ray head) at 34.71.183.58:6379. Check that (1) Ray with matching version started successfully at the specified address, (2) this node can reach the specified address, and (3) there is no firewall setting preventing access.

when I go to run ray.init() it throws this error:

RuntimeError: b"This node has an IP address of 172.31.86.217, and Ray expects this IP address to be either the GCS address or one of the Raylet addresses. Connected to GCS at 34.71.183.58 and found raylets at 10.128.0.2 but none of these match this node's IP 172.31.86.217. Are any of these actually a different IP address for the same node?You might need to provide --node-ip-address to specify the IP address that the head should use when sending to this node."

I have been stuck on this, some help would be nice.

[crystal current]

Question isn't that clear to me but if you know how implement API endpoints in Flask, you can make then do pretty much anything you want.

[wild dove]

What should I do if I want to monitor the flask server in real time and start it again when the server is stopped?

[crystal current]

What exactly is "stopped" and why? That will determine what exactly you're trying to monitor.

Is this a physical dedicated server that you want to restart or where is it hosted? You want to restart the whole server and not just your Flask app?

Is this related to your original question somehow?

[fathom walrus]

is it possible that my router doesn't support telnet? or ssh?

[coarse summit]

I doubt it doesn't support it

may be blocked

[fathom walrus]

argh

thanks

[coarse summit]

why do you thing it doesn't support it?

[fathom walrus]

telnet fails and ssh gets connection refused, but I see no options to change it on my browser

[coarse summit]

how are you using telnet and ssh

as server or client

[fathom walrus]

as client, running ssh admin@192.168.0.1 on cmd

[coarse summit]

can you send error message as text

[fathom walrus]

ssh: connect to host 192.168.0.1 port 22: Connection refused

[coarse summit]

which server are you trying to connect to

[fathom walrus]

the router's, seems like some routers have a ssh server enabled by default

[coarse summit]

oh

not all routers have ssh access

why are you trying to connect to it?

[fathom walrus]

i wanted to check if I could modify its files, or run wireshark on it

or otherwise flash it

trying to figure out how networks work

[coarse summit]

http://192.168.0.1

access that

do you see a login page?

[fathom walrus]

yeah, i already did, and i've been messing around with configs and such

couldn't find anything explicitly called ssh

tried forwarding 22, still refused

[coarse summit]

not all routers have ssh

[fathom walrus]

hm yeah that might be it

ty for help tho

[coarse summit]

hmm @fathom walrus

try using putty

[fathom walrus]

oh, ill try that

nope still refused

yeah i think it just doesn't support it

[crystal current]

Port forwarding will let you connect to a device on your network from outside, so not relevant. If you want to SSH or Telnet into your router, you need to look up the documentation for your router to find out if that's possible.

And flashing the firmware is another story but also depends on your specific router. Probably doesn't require SSH or Telnet

It's a common error you can find lots of possible solutions for if you search around. I don't know about Laravel but Flask has various settings you can use to allow CORS

[wind radish]

Most likely it can be disabled or enabled

[fathom walrus]

thanks for comments!

[shell skiff]

how should i store data like images that i can recreate at a later time. I have a system where it uses json but it proves to be inefficient when i try to introduce it to bytes that are for pictures, files.... And so i need a way to store these as a file that i can then encrypt(cryptography library fernet) and then give the file to someone which that someone can decrypt it and basically decrypt the entire data stored and process it, XML might come as a good option but idk if it will work and how will i work with it

(i use fernet cryptography to encrypt it and its user is assigned with a special key so no worries as the admins cannot understand that much the file contents but the bot can)

[crystal current]

Totally off topic in this channel or am I missing something?

Sounds like you either want something like a) links to encrypted files (in an S3 bucket or whatever), b) encrypted blobs in a DB (if the images are small) or c) encrypted base64 strings (probably the least attractive option in general but closest to what you describe doing currently)

[shell skiff]

well i was in #algos-and-data-structs

and i was redirected here

even tho both channels secuirity and networking sounded off topic

i tried on networking

so im just gonna open a question on #1035199133436354600

sorry for like the entire disturbence ig

[crystal current]

It's all good. Have I not answered your question already though?

[shell skiff]

well

kinda

i tried like using bytes

on json

which is not a good approach

and the file size from like some kilobytes to 2 mega bytes

which isn't ideal

and idk about base64

it might be the same too

[gloomy root]

converting binary data to text formats will pretty much always have an increase in size

[shell skiff]

yes

and so i need a effective way to store

all types of files

including images

[crystal current]

I gave you two different approaches to doing that

[gloomy root]

What dowcet mentioned is about what i'd expect in your situation

[shell skiff]

also i don't wanna store it in a db

like i already have a db

and i basically give a encrypted file to the user as sort of a snapshot

for the S3 bucket i don't really know what to do

[gloomy root]

how is the user decrypting this tho?

[shell skiff]

the bot does

its basically a middle man

[crystal current]

Already having a DB sounds like a great reason to use it?

[shell skiff]

well..... im sharing the file with the user

i don't wanna add too much memory for snapshots

[gloomy root]

Eh, I'm not normally a big fan of storing large blobs in DBs, particularly relational DBs

[shell skiff]

so the solution i came up with

is have a file encrypted

and give it to the user to store it

[crystal current]

For example, upload the file to the bucket with boto3 and store a URL in your DB. Retrieve/decrypt the image as needed

[shell skiff]

which later they can use to recreate the snapshot

[gloomy root]

wait? The user has to store the encrypted file?

[shell skiff]

well yup

[gloomy root]

What is the point in the middleman then?

As a user, it's just easier to manage them yourself

[shell skiff]

well its responsible for decryption and encryption

and processing

in between

[gloomy root]

but... Why? Since the only interaction where the file being encrypted is actually useful is the process of transporting the file, from the user, to the bot

Which realistically is a bit pointless most of the time

I'd argue it's easier for the file to just not be encrypted and just ask the user for the file in order to do what ever processing it needs at the time

[shell skiff]

let me backtrack a bit

[gloomy root]

it's going to be no less secure

[crystal current]

Better move it to a help channel

[shell skiff]

ok

again tho sorry for the disturbance

[simple locust]

Hello

[shell skiff]

a bit of a update

i encoded to base64 and not only it reduced the file size it also worked

but

i feel like it can scale even lower

but lets not clutter and move to https://discord.com/channels/267624335836053506/1141750632122945617

[short flume]

Need help intercepting http and ws requests being made by a desktop application

The application is using its own certificates for SSL

So using a proxy like mitm is not working

Any suggestions?

[jaunty oasis]

Hey all, could anyone help me? I'd like to know how to create a very basic python websocket which reads from a .pipe file and broadcasts it to all connections. I have a working websocket and a working pipe file, however I cannot seem to connect the two without 1 blocking the process of the other. Much appreciated!

[wind radish]

base64 can't reduce files size. base64 is literally encoding with 6-bit alphabet, while byte has 8-bits

[shell skiff]

oh

yeh i kinda was testing and saw that

[ember seal]

Wassup fellas any decent coders want to join FG

[coarse summit]

No

[ember seal]

Ahhh, figured. Thanks anyways.

[grave lily]

Patch it to replace the certs?

[ember ledge]

finally i was able to post it, the tool has some minor issues, but it works just fine
https://github.com/HalilDeniz/ICMPWatch

[errant bay]

Rules

5. Do not provide or request help on projects that may violate terms of service, or that may be deemed inappropriate, malicious, or illegal.

[ember ledge]

Good but don't use py Charm instead use visual code

[ember ledge]

I actually use pycharm all the time 😅😅

Yep a lot of people use it, once I used py Charm (just to try) and I just stick with visual code, it's py Charm but more

[ember ledge]

PyCharm* please

[edgy pelican]

good but dont use visual code use neovim

[coarse summit]

Use a hex editor instead

[crystal current]

Use butterflies. Just kidding, use Emacs. https://xkcd.com/378/

xkcd: Real Programmers

Real programmers set the universal constants at the start such that the universe evolves to contain the disk with the data they want.

[covert forge]

Hello, does anyone know, how to use a Rasperry Pi as an bluetooth speaker?

[ember ledge]

#microcontrollers

[covert forge]

It's networking too ._.

[coarse summit]

Yeah but you'll get better help there

[covert forge]

Ah okay

thanks

[coarse summit]

No

Np*

[ember ledge]

Hello

How does my router know which device to forward if I only give out my public ip of my router?

[zealous flint]

I believe that is configurable

At least on my home router I remember having to point the router at my server with the open port

[coarse summit]

in my router it's automatically done

[ember ledge]

Anyone write socket code?

[wind radish]

For the outgoing connection established from LAN it maintains nat table for each connection.
For the incoming connections - you configure Port Forwarding, or for some applications it is configured automatically via UPnP

[ember ledge]

Thanks

What is the point of NAT?

[wind radish]

To use single IP address in WAN, even if there are multiple devices behind the router.

[open isle]

NAT was primarily invented IIRC (this is wrong) is quite useful so IPv4 addresses can be used more efficiently. In the context of residential internet, instead of assigning a globally unique IPv4 address to each device, each customer gets only one IP address.

There are simply not enough IPv4 addresses to assign a globally unique address to each device.

IIUC there is also carrier-grade NAT, ie. NAT at the ISP level

[ember ledge]

So what is NAT?

🙃

Nevermind. You were helpfull. Think I just need to do more research.

[ember ledge]

I'm guessing it translate public to private ip and vice versa

[wind radish]

NAT is Network Address Translation. The name occurred because in a nutshell it is manipulations with IP addresses stored inside IP packets headers

[ember ledge]

I know that the tcp header contains a source and destination ip address. I believe they are both global.

[wind radish]

not necessarily

they are just source IP and destination IP

[ember ledge]

Well if you want to connect over the internet you'd pass the global ones correct?

But if it's local you may use the private ones

[wind radish]

as a destination IP - yes

but when the packet leaves your PC, it has local IP as source IP inside the header

[ember ledge]

Is the source used to send messages back?

[wind radish]

yes

[ember ledge]

Hmm I made a socket connection. Was able to send a message to myself with that special ip but not my global ip

Think I need port forwarding

[ember seal]

Is this the place to find a programming mentor?

[zealous flint]

No, this channel is for computer networking (sorry if the name is confusing)

We don't usually do mentoring here but you can ask questions as you have them

[ember ledge]

My sockets book is written by Warren W Gay

On the side it says Gay at the top

Gay Linux socket programming

By example

That's pretty funny to me

[ember ledge]

This is actually fucking hilarious.

[errant bay]

Rules

7. Keep discussions relevant to the channel topic. Each channel's description tells you the topic.

[ember ledge]

Got a problem?

[coarse summit]

?

[ember ledge]

I was talking about a sockets book. That is network related.

[odd token]

Hi. The SSLv3 protocol is outdated and not supported since python 3.6+. Is it possible to use this protocol in newer versions?

[fathom walrus]

when a process is listening to a port for a packet, what OS/kernel-level process is it interacting with?

[coarse summit]

Transport

https://www.sciencedirect.com/topics/computer-science/transport-layer#:~:text=Layer 4%3A The Transport Layer&text=The elements of the transport,which TCP%2FIP ports listen.

[fathom walrus]

nice, the transport layer has a kernel component?

[coarse summit]

U got me there

[fathom walrus]

nevertheless i'll probably get better search hits using the layer model, thanks!

[coarse summit]

What do you mean if the layer has a kernel @fathom walrus

[fathom walrus]

I mean if a OS' kernel contains a component that deals with the transport layer, or if whatever interacts with the transport layer is external to the kernel

[coarse summit]

"The kernel primarily deals with the lower layers of the OSI model, such as the network and data link layers. The transport layer is typically handled by the operating system's networking stack, which works closely with the kernel to manage data transmission and flow control."

[fathom walrus]

nice ok

now, need to figure out what exactly is the networking stack

ty for the help!

[coarse summit]

Np

[dusty notch]

hey guys hope yall doing well here. I was wondering if someone understands a lil bit trading, because i'm developing a neural network trading AI that is nearly finished: I have a float issue because i want to float my broker balance and that is perturbing me a lot and i am struggling to fix. If you want to help. Please ping me. PS: I'm 15 y/o I don't have as many experience as you here. Thanks for reading

[crystal current]

#❓｜how-to-get-help

[ember ledge]

Anymore make money from coding Projects?

[open isle]

This is the wrong channel to ask in. This channel is about computer networking, think the internet, IPv4, HTTP, routers/switches/other networking hardware.

[jovial gate]

Having a small problem with scapy layer binds. I have two layers that always come as a pair such as Base_Layer/Upper_Layer1 or Base_Layer/Upper_Layer3 The problem is they can come in multiple sets. I've fixed scapy so that it properly parses the lengths of each layer, but when I try to process the second/third/fourth batch, the only way I can figure out how to do it is with the bind_layer functions like so:

bind_layer(Upper_Layer2,Base_Layer,)
bind_layer(Upper_Layer3,Base_Layer,)```
This works, but I have something like 30 different upper layer packet types.

Is anybody aware of a better way to handle this?

I feel like there is going to be something smarter I can do in the Base_Layer guess_payload_class function but I just haven't found any clues yet how to do it

[grave lily]

If you have to have a lot of variable names which are the same except for numeric suffixes, that may be a job for an array.

[edgy pelican]

wrong channel?

[drowsy knot]

https://blog.sanctum.geek.nz/vim-koans/

[jovial gate]

No, he's trying to help me, and he's got a good general point that, unfortunately, doesn't work in this particular case.

Also, I think I'm going to try converting each ADU/PDU pair to instead be an ADU with a packetlistfield, so we'll see how that breaks the build/dissect functions already in there.

[restive rover]

Oh! Looks like this is the wrong type of networking channel 🤣

[ember ledge]

We won't join calls for things that are not worth posting in help channels, it's more useless to join the voice channel in that case

Small questions can very well go in text channels instead of making people join calls

[wind radish]

Take your time and came up with a question. If you don't have any questions it doesn't worth to communicate.

[ember ledge]

Considering your post in the help channel, you were here in the wrong channel

Yeah, and that is the wrong channel

This channel is for computer networking rather than social networking. The mixup happens a lot so not really your fault

[forest patrol]

hello is wifi speed is included in networking ?

[cunning garden]

What would be your question if it was?

[forest patrol]

how to make my internet speed better without paying any extra cost ?

if it is possible

[cunning garden]

In general, you can't

you could optimize some things, but can't stuff more than the pipe allows

[forest patrol]

why 😔

[cunning garden]

For instance having ad blockers

[forest patrol]

why cant i have good wifi for once

[cunning garden]

wifi != Internet

[forest patrol]

i have bad internet and bad wifi and bad 4G

[cunning garden]

You are gonna have to divide and conquer

[forest patrol]

like there isnt some tips and dialy hacks to make it better

[cunning garden]

there is no magic

[forest patrol]

and how to make a file that doenst want to delete delete

it says item not found

[cunning garden]

you check your logs/system to understand what it is saying when it refuses to do so

[forest patrol]

its an empty file

that cant be deleted

from the desktop

[cunning garden]

there must be a reason

[forest patrol]

is there a way to make a normal folder into a zip file without keeping the original folder ?

[cunning garden]

you make a zip file and then delete the original folder

or if you mean not having the folder in the zip file, then just zip the files inside it

[forest patrol]

that is the problem i cant delete the original folder

[edgy pelican]

me and da bois rewriting amazon in brainfuck

[wind radish]

I don't even understand what are you asking. Do you mean your wired connection has good speed, but your wireless (wifi) connection is much worse?

[small basin]

Would anyone know why it is preferable to have a IP range start at E.g. 192.168.1.10-192.168.1.255 rather then starting at 192.168.1.2... in essance why start at .10 rather then .2

[zealous flint]

Is it? I haven't heard of such a thing

[small basin]

yeah i have seen it on some systems the DHCP will start assigning from 192.168.1.10 - ...255 and leave .2-.9 unassigned i think its a type of reservation for static addresses but idk and trying to confirm

[wind radish]

it's up to you

[zealous flint]

that sounds like just leaving some space for statically assigned hosts

[long gazelle]

Good day guys it's my first time here!

My question is can someone suggest to me a video tutorial or a step by step guide on how to host twisted python to internet to work in real life not in local host

[long gazelle]

Please guys help

[ember ledge]

get a vps and configure it with nginx

or use dokku

or dont get a vps and put ur localhost on cloudflare tunnels or ngrok

[ember ledge]

or rent some of the app platforms that do all hostng for u

[long gazelle]

Please can you suggest any app thag does so??

Thanks in advance

[ember ledge]

digital ocean

[long gazelle]

Thank you very much I will surely look into it deeply

[inland tiger]

wait where is that minimal code link

i think there was a gist pinned

maybe im blind but i dont see any gists

[heady bane]

I am trying to build the following architecture https://imgur.com/a/uPWwN9W , where from our platform internal resources will be analyzed from it . what's the best way to achieve this ?

[umbral vector]

what

[heady bane]

How do you build a saas internet to reach internet resources?

[steady epoch]

hi

[wind radish]

I don't understand why you call it architecture. This aint architecture.

[robust badge]

Hi everyone, I recently completed an internship at an ISP company and learnt quite a lot about Networking that i didn’t know before but now I need ideas on projects I can build with python based on networking

[wind radish]

• Connecting to devices via pexpect / paramiko / netmiko
• Generating configs via jinja2 templating engine
• Parsing output with regexp's and TextFSM
• Working with csv (maybe with sqlite as well)
• Learning what Ansible is and how to use it
• Learning what Fabric (fabfile.org) is and how to use it

here is good resource: https://pyneng.readthedocs.io/en/latest/
also you could like the "Automate the Boring Stuff with Python" book

[robust badge]

Thank you!

[urban river]

hello, I would like to know about a possible web system showing data to customers. I am redirected from #web-development , but I can synthesize: I need to build a web app which is made of a single page which displays data obtained from a remote server. Such remote server is running python endlessly, as it continuously generate data. The webpage and the server are located in a company network, with the server protected behind a router and it being accessible through vpn only, while the webpage server is available for customers without authentication and can access to the python server. Do you think it is possible to adopt obfuscation strategies such that the server URL is not visible from the webpage?

or should I ignore that and adopt authentication strategies?

[edgy pelican]

1: no, 2: #web-development

although i think the best idea would be to use a reverse proxy

---------------------
| proxy & webserver | <----------------- server
---------------------```

[lilac mantle]

Does anyone have a good resource for learning scapy?

[urban river]

in absence of alternatives I will take that into consideration, thanks

[edgy pelican]

#cybersecurity might be more suitable

[fathom walrus]

is there any way to track wifi strength of a device receiving wifi from the router itself?

or does this necessarily have to be device-side

[zealous flint]

Not without said device telling the router about the signal strength, no

[fathom walrus]

was hoping there was some sort of handshake going on, oh well. thanks!

[lilac mantle]

Is there a way to run python code on a router?

[ocean bay]

idts but you could point the router to a proxy server that runs python

[zealous flint]

Depends on the router

[wind radish]

My Mikrotik shows signal strength for each connected device

[edgy pelican]

wait so if 2 ppl have the same web browser version, hardware, and OS version, they'd have the same fingerprint

[jagged copper]

they wouldnt

[edgy pelican]

JA3 gathers the decimal values of the bytes for the following fields in the Client Hello packet; SSL Version, Accepted Ciphers, List of Extensions, Elliptic Curves, and Elliptic Curve Formats. It then concatenates those values together in order, using a "," to delimit each field and a "-" to delimit each value in each field.

and extensions

like, if you have the same hardware & OS & web browser, it'd be the same

[jagged copper]

and depending on the site, adds in ip, location, cookies, headers etc

[edgy pelican]

ah

gtg

[jagged copper]

https://engineering.salesforce.com/open-sourcing-ja3-92c9e53c3c41/

[ember ledge]

Does anyone has any project on networking using sockets and so?

[edgy pelican]

chat app

bonus points if you make it encrypted :) (kinda hard tho)

[ember ledge]

Who is making it?

[edgy pelican]

i am

[ember ledge]

How can I help?

[edgy pelican]

mostly working on https://github.com/SomeHybrid/crypto

(im making it from scratch scratch, so cant really help)

[cyan plume]

Hi, is anyone interested in becoming programming buddies ?

[ember ledge]

I want programming buddies

always

xD

but if u use diapers and don't sleep until 4 am due to an overdose of caffeine

[ember ledge]

So it is an encryption library?

[cyan plume]

if someone is interested please pm me

[ember ledge]

i will

[edgy pelican]

yeaaahhhhhhhhh

[ember ledge]

and you are coding an encryption algorithm from sratch because there is no hyper-optimzed version of it yet?

[edgy pelican]

no

its because i can

[ember ledge]

Ah, okay

So no need of it?

xDDDDDDDDD

[edgy pelican]

(dont do this at home kids, something something dont roll your own crypto something something schneier's law, something something timing attacks, something something side-channel vulnerabilities, something something just dont waste your time)

tl;dr: dont

[ember ledge]

Yeah, good luck.

It is gonna be fun though

[edgy pelican]

but this is more #cybersecurity land

[edgy pelican]

no its not

im translating literal garbage C code

where they dont use loops for the sake of not using loops

uGH

[ember ledge]

That's my definition of fun

[edgy pelican]

gtg

[ember ledge]

That and diapers

So I don't waste time going to the WC

[edgy pelican]

here's a taste
(not even the worst of it)

        d0 = ((unsigned long long)h0 * r0) + ((unsigned long long)h1 * s4) + ((unsigned long long)h2 * s3) + ((unsigned long long)h3 * s2) + ((unsigned long long)h4 * s1);
        d1 = ((unsigned long long)h0 * r1) + ((unsigned long long)h1 * r0) + ((unsigned long long)h2 * s4) + ((unsigned long long)h3 * s3) + ((unsigned long long)h4 * s2);
        d2 = ((unsigned long long)h0 * r2) + ((unsigned long long)h1 * r1) + ((unsigned long long)h2 * r0) + ((unsigned long long)h3 * s4) + ((unsigned long long)h4 * s3);
        d3 = ((unsigned long long)h0 * r3) + ((unsigned long long)h1 * r2) + ((unsigned long long)h2 * r1) + ((unsigned long long)h3 * r0) + ((unsigned long long)h4 * s4);
        d4 = ((unsigned long long)h0 * r4) + ((unsigned long long)h1 * r3) + ((unsigned long long)h2 * r2) + ((unsigned long long)h3 * r1) + ((unsigned long long)h4 * r0);```

[ember ledge]

It looks fun, if that's how you want to spend your time I see it as praiseworthy

[normal cape]

This is supposed to be a proxy that an application can connect to over http and have its connection be bounced to a specific url over https. It's a long story, but I have a piece of software that needs to talk to a database without encryption, and for a few different reasons the database connection needs some TLS.

import asyncio, sys, ssl

LISTEN_PORT = 7500
DST_PORT = 443
DST_HOST = "wikipedia.com"
#DST_HOST = "neverssl.com"

async def pipe(reader, writer):
    try:
        while not reader.at_eof():
            data = await reader.read(2048)
            print(f"Received {data!r}")
            writer.write(data)
    finally:
        writer.close()

async def handle_client(client_reader, client_writer):
    try:
        # connect to database server
        ssl_context = ssl.create_default_context(ssl.Purpose.SERVER_AUTH)
        ssl_context.check_hostname = True
        server_reader, server_writer = await asyncio.open_connection(
            DST_HOST, DST_PORT, ssl=ssl_context)
        print("Connected to server")
        pipe1 = pipe(client_reader, server_writer)
        pipe2 = pipe(server_reader, client_writer)
        await asyncio.gather(pipe1, pipe2)
    except Exception as e:
        # printing the error message
        print(f"Error connecting to server: {sys.exc_info()[0]}")
        print(f"{e}")
    finally:
        print("Closing connection")
        client_writer.close()

async def main():
    server = await asyncio.start_server(handle_client, '127.0.0.1', LISTEN_PORT)

    print('Serving on {}'.format(server.sockets[0].getsockname()))
    try:
        await server.serve_forever()
    except KeyboardInterrupt:
        pass

    server.close()
    await server.wait_closed()

asyncio.run(main())

I've tested a very slightly modified version of this (just no https) locally with regular old http over netcat and everything works fine, and I've tried connecting to neverssl.com over this using a web browser and that works fine as well.

When I change the code to use https and try connecting to 127.0.0.1:7500 using a web browser I get this in the console:
Error connecting to server: <class 'ssl.SSLCertVerificationError'>
[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:992)
Closing connection

I've probably forgotten to mention something important, but thanks for any input in advance!

[normal cape]

Oh, one thing I forgot to mention is that the answers to this that I'm seeing online have to do with pip installing certifi (already installed) and running a Certificates.command executable (I'm on a mac), but my python install doesn't include that anywhere for me to run it.

[next marsh]

Did you install python via an installer or homebrew?

[normal cape]

homebrew

[next marsh]

Certificates should have been handled for you then. The installer lacks permissions for both actions and, as you found online, gives the user an additional command to run.

[normal cape]

Ok, that makes sense. I was hoping that was the case.

[next marsh]

Aside from ensuring that certifi is up-to-date, I'm not sure myself. The SSL error is not the one I'd expect if the certs are bad.

[normal cape]

Ok, just checked and certifi is definitely up to date. Thanks for trying, I appreciate the effort.

[next marsh]

Things I'd doublecheck: Make sure you are running python with the same homebrew python version you installed. Make sure that homebrew version included ssl (it feels like it didn't). Might even check if you have openssl installed (via homebrew). Total spitballing here. As others check in here maybe more options will be forthcoming.

[normal cape]

ok, just checked all three of those things and they all look correct

[normal cape]

It was, in fact, just an issue with my local dev environment. I tried it on a different machine and there were no problems.

I spent a long time on that for it to end up being nothing. Ouch.

[weary coyote]

Сloud Builders: Python Conf is already today 🐍
Are you with us? Just save your place 👉 https://www.cloud-builders.tech/en/python-conf

Today's evening, we will start with the cosmic journey with a fireside chat about Python's role in unlocking the secrets of the Universe with the JamesWebbSpaceTelescope 🌌

Then we join 4 deep-dive Python talks: #codeimprovement with #formatters like Black PyBetter; AIspace, Open-Source models, Caikit, OpenShift, API; synergy between OpenSearch, Python, and serverlessarchitecture; and speedrun Pythondeployments with azd.

👀 See you soon!

[cedar forum]

the joy of certificates

[edgy pelican]

which sort

the ssl type?

[grand kayak]

damn. that's why I use a clean docker container for my tests every time

[edgy pelican]

!rule ads

[errant bay]

Rules

6. Do not post unapproved advertising.

[edgy pelican]

@queen moth also wrong channel, this is about network protocols, not ads

[queen moth]

Ok, will change. Thanks.

[cedar forum]

anything x509

[edgy pelican]

oh that sort

never had to deal with them, never want to deal with them

https://tls12.xargs.org already makes it look painful

[cedar forum]

you know the crazier thing

[edgy pelican]

what

[cedar forum]

TLS v1.2, v1.3 etc etc

their actual protocol header does not identify them as that

they all "pretend" to be TLS v1.0 for back compatibility

[edgy pelican]

protocol engineering is ???

[cedar forum]

even when they started saying that connections were TLS v1.2 when you use 1.3 it still pretends to be 1.2

spec literally says "hardcode this value to tls v1.2"

[edgy pelican]

lmao

[cedar forum]

and even better, it doesn't use version numbers like 1.2

[edgy pelican]

that kinda boosts my self-esteem

knowing that its not only my code that's held together with tape and glue

[cedar forum]

lol

[edgy pelican]

lol indeed

[cedar forum]

but yeah all TLS versions are technically "extended versions" of SSL

SSL was 3,0, TLS 1.0 was 3,1 because it was a minor revision to SSL, so TLS 1.3 is 3,4

confusingly

[edgy pelican]

oh wait

so

how does the client know what CA is being used?

does the server just send the name

or does it just go through its root CAs and check

[cedar forum]

certificates contain a "hierarchy"

so you have a certificate for pythondiscord.com which in turn gets signed by X numbers of intermediary CAs and then a trusted root CA

[edgy pelican]

ah

and the website sends over the hierarchy of certs?

[cedar forum]

yep

it's all contained within the certificate

[edgy pelican]

ic ic

[cedar forum]

https://paste.pythondiscord.com/4FXQ

you can see the certificate chain section there

[edgy pelican]

very cool

and then the client just iterates over the intermediary CAs and get their public keys?

[cedar forum]

yea you just need to validate that the topmost cert is a trusted root

https://paste.pythondiscord.com/R43Q

[edgy pelican]

turtles all the way down :)

[cedar forum]

and then that's the actual "cert data"

pulled from openssl x509 -text -noout

[normal cape]

I did initially, too. For some reason it wouldn't work in the container and I didn't get any errors anywhere, so I switched from docker thinking debugging it would be easier.

Famous last words.

[normal cape]

that looks awesome, thanks for sharing

[grand kayak]

that's strange, what image were you using? typically you can see the errors in docker logs

[cedar forum]

there is also a great one for QUIC https://quic.xargs.org/

[normal cape]

just the latest python image

[winter knot]

hello everyone! can someone help me with a socket code? I am trying to send a pickled object through socket. but whenever the client side receives the data, a '0' is automatically added in front of the serialized object which makes the object impossible to deserialize. do you have any idea why this might happen?

[cedar forum]

do you have any sample code?

and is it a 0 or is it a \0 (null byte)

[winter knot]

the starting should be b'\x80\x04... whereas it becomes b'0\x80\x04...

[cedar forum]

any example code would be great, it's almost impossible to diagnose this without that

upload some client and server code to https://paste.pythondiscord.com/

[winter knot]

have a look here: https://paste.pythondiscord.com/CF4Q

tell me if you need anything more

[cedar forum]

is there any chance that player_serial could be 0

i get the feeling that you are sending a 0 and then you are sending your pickle encoded data, not "a 0 is appearing in front of my pickle data"

[winter knot]

yes

[cedar forum]

yes

then that's it

you're not reading pickle data, you're reading the player_serial that you've sent and then the pickle data

[winter knot]

but the problem is, whenever I try to unpickle it gives me unpickling stack underflow error. in order to diagnose that problem, I unpickled the same pickled object on the server side and there it runs without a problem.

[cedar forum]

it's not at all related to the pickling

that works fine

okay so let's run through what data gets sent

the client connects, then client.sendall(str.encode(str(player_serial))) runs and sends the player_serial, that will be 0

now at the client end, the receive buffer will sit at 0

then the server will send client.sendall(pickle.dumps(game))

at the client end, the buffer will now sit at 0[ALL THE PICKLE DATA]

do you see where the 0 is coming from now?

[winter knot]

the 0 is inside the binary serialization: b'0\x80\x04...

or pickle serialization whatever you call it

[cedar forum]

on the client end you are running client.recv(1024)

that will recieve ALL data that has been sent to the client and is waiting to be read

you are sending two things, the 0 from player_serial and then the pickled game information

client.recv does not know those are two different things, it will just bundle them together as one binary blob, it has no idea what pickle is, it just knows you've sent a bunch of bytes and want to receive a bunch of bytes

[winter knot]

ohho now I get it

so how do I separate them? any suggestions?

[cedar forum]

well, you can use newlines (\n) to separate data

or maybe the smarter way would be to not use pickle for this and to use JSON and instead of just sending the player_serial as 0 send it as it's own JSON message format

that way you can parse all messages on the client the same and know "oh this JSON object is the player_serial", "oh this JSON object is the player state", etc.

(more secure this way as well, accepting/sending raw pickle over a network socket is a horrible idea as anyone that sends you data can execute code remotely on your system)

[winter knot]

by the way, there's another client.receive(1024) in another method before this particular one which sends it in an encoded string.

[cedar forum]

https://docs.python.org/3/library/pickle.html

[winter knot]

okay I get it

it works... thanks a lot. I was stuck here for almost a day.

[flint nexus]

i want to create a library/module for python to speak a specific header-based protocol. (i'm probably not using the correct jargon here). the protocol basically works like this:

• connect to a tcp server
• send a specifically crafted message to issue data
• start while loop to read data from the socket
• parse header for message type and message length
• parse message type, if message type data, then read data, if message type EndOfData, then terminate

i want to "hide" most of these details in a library/module.. so i can for instance just do frame = <library>.get_frame() or similar

[grand kayak]

could you please provide some info on how we can help you?

[turbid dock]

It'd probably be a lot easier if you had a library to run PHP

[jovial gate]

Scapy is a tool that is designed to do a lot of this.

[umbral vector]

scrapy is a pretty nice framework but with the things written as the only purpose, it might be overkill

[jovial gate]

Speaking of Scapy, can anybody offer guidance on the difference between using guess_payload_class vs packetfield?

[umbral vector]

it seems like a really niche problem to write a library for. maybe having it stuffed away in its own little module where you need it in your own project can work