networks

[modest brook]

you can't create private keys from pubkeys

[prisma flax]

Well but if we can't edit the pubkey or the coordinates we can't really do it

[modest brook]

ie, you can't create a pub key from some seed value, because knowing such pubkey won't be of any use given that its private key can't be derived

[prisma flax]

Its mathematically impossible

The solution would be deriving the privkey but you cant do that

Or the person who derives it form the username+coordinates would, well, have derived the privkey

[modest brook]

it calls for some form of centralization or geographic address mapping. ie, back to the internet

[prisma flax]

Wait I think I have to go for like 20 mins

[modest brook]

then anyone could derive the private key

[prisma flax]

Thats what Im saying

[modest brook]

cool, i'll head out to sleep now i think

[prisma flax]

Oh alright then

Hopefully we can continue this conversation tomorrow

cya

[modest brook]

surely

[sharp osprey]

@prisma flax

[prisma flax]

Here

The global rule is any client can create or join a network, which would usually just be the common open network, but still.
When you receive a join request you check if you have any connection slot open (in 2d you have 4 slots, 1 for each direction). If you do then you assign the joining user to one of them, otherwise you pass the connection request to your connections

A user can join through "entry points" which would probably be established naturally and are normal clients BUT theyre supposed to always be open and publicly known

Basically an easy gateway into the network

Now I said 2d because you can imagine this as a grid, but the real network would probably be more like 10d

You there?

[sharp osprey]

yeah

but why would there be dimensions in a network? usually its just a mesh of nodes

[prisma flax]

So that everyone has coordinates

The one who creates the network will have (0,0)

Then when someone connects, say he gets assigned to (1,0)

And so on

Which should naturally form a circle like shape

[sharp osprey]

how is the network dimension decided based on amount of people?

[prisma flax]

The dimension isnt based on the amount of people, its decided by the creator of the network

[sharp osprey]

this seems like a graph based network

[prisma flax]

Yeah i guess it is

[sharp osprey]

we are just naming nodes x,y or maybe higher dims

[prisma flax]

Yeah, as I said, I'm using 2d just to visualize and explain stuff faster, but it will not be in 2d or even 3d

[sharp osprey]

ok so say its a 2d network creator is 0,0 and there is x,y plane so u can still have a lot of combinations right or is it constrained to 1 step every direction from 0,0

[prisma flax]

Nope, it can stretch out to infinity

The creator will connect to the positions directly adjacent to 0

When there are no more open

He will start relaying the join requests to the connections

And whoever can connect the new person the fastest fills that slot

[sharp osprey]

so for 0, 0: 0, 1; 1, 1; 1, 0; 1, -1; 0, -1; ...?

[prisma flax]

(0,1);(1,0);(0,-1);(-1,0), almost what you said

1,1 and 1,-1 are diagonal to 0,0

[wild estuary]

i still fail to see how coordinates are mapped to IPs

[prisma flax]

They are not

If you are at 0,0 you are connected to the people around you

[sharp osprey]

so only orthogonal adjacency

[prisma flax]

And everyone is connected to the people around them

So if you wanna get your message 20 hops apart

[wild estuary]

then how would i send anything to my neighbour

[prisma flax]

You wouldnt know the ip of the receiving end

You would send the message to the one closest to him

And hed send it to his connection closest to him

Until it gets to him

[wild estuary]

right

so

is there no unique IDs?

[sharp osprey]

i am assuming ur coordinate is unique to u

[prisma flax]

Me and Tanner were trying to solve this

[sharp osprey]

in that network

[prisma flax]

Coordinates are

[wild estuary]

this feels like you're inventing IPFS but worse

[prisma flax]

We wanted to find a way to make it so that if you know the coordinates and the chosen username of someone you can get their pubkey and write to them

[wild estuary]

.

[prisma flax]

But this can't be done like this so we have to find a solution

[prisma flax]

How does IPFS work?

[sharp osprey]

its a file system

[wild estuary]

https://docs.ipfs.tech/concepts/dht/

[prisma flax]

Yes thats right

[wild estuary]

i suggest you take a read over that

[prisma flax]

I know what they are

[wild estuary]

specifically kademlia

[prisma flax]

Actually this idea is kind of derived from DHTs

[wild estuary]

then why not just

use one

sob

map a pseudorandom id to a pubkey

[prisma flax]

A DHT is like what I wanna do but 1d and circular

[sharp osprey]

its not exactly a dht is it thats more like a content hash table and the user directly looks up the node or peer responsible for the content

here he is talking about multiple hops to get to the peer instead of direct message to the peer (now that makes sense in bluetooth but idk about internet)

[prisma flax]

The connections between adjacent nodes would probably be P2P using webrtc

[wild estuary]

right but its the same fundamental problem being solved

doesnt webrtc require some form of server

[sharp osprey]

true its just how ur solving it in this case you would have to create a central look up table with pubkeys or a registry for look up

[wild estuary]

its not central tho

the entire point of a dht is that it isnt

[sharp osprey]

yeah distributed

mb

[prisma flax]

Not really. You can enter the network directly with someone or an "entry point", which would be a known, always open client (which you could argue is similar to a server, but the entry point has the exact same power as every other client)

And also anyone could create their own entry point

Its just a server running the client

[prisma flax]

So I could add a DHT to the system?

[wild estuary]

this really does sound like a dht tbf

[sharp osprey]

again i think some theoretical benchmarks, comparisons etc (basically a rfc doc would be nice to have)

[prisma flax]

Well until I actually create anything it's hard to get those

[sharp osprey]

u could technically but then it just becomes a direct lookup instead of ur orthogonal n dimensional grid system

[prisma flax]

Its a multidimensional DHT I guess

[prisma flax]

You use the DHT to look up the coordinates and pubkey maybe?

Idk I feel like Im just going in circles

[wild estuary]

i still fail to see why a coordinate system would be preferable to just directly sending

especially because a coordinate system is much more vulnerable to eclipse attacks

[prisma flax]

What are those?

[sharp osprey]

u create like artificial or fake nodes around the target

[wild estuary]

if all your neighbours are malicious then they can send you literally anything or deny service

[prisma flax]

In a 10d mesh it's impossible to coordinate such an attack

Also you can't really create fake nodes, I'd explained it above, let me explain it again

If someone wanted to claim that node (10,10) is now occupied he would have to tell all of (10,10)'s neighbours and make them connect to him. If he can't do this (which he can't if its a fake node) then he can't really fake anything. There's no such thing as claiming, everyone just acts based on what they know

If the people around the node don't have anyone connected there, to them there's no one

And they'll fill that spot up when they can

[wild estuary]

so once a spot is taken it can never be reclaimed?

that doesnt seem

[prisma flax]

When the webrtc connection with that person is ended, the people around it know that they now have a free slot available and will try to fill it

[sharp osprey]

whats stopping me from using like n devices and trying to be that persons neighbor

[prisma flax]

In a 10d network it would take a LOT of devices to have them fully surround even a single node

The person can just reconnect

[wild estuary]

"10d network" isnt very specific

i could have anywhere from 11 neighbours to 500000

[prisma flax]

In a 2d network there are 4 neighbours

4 directions

Plus all the two-hops neighbours, so double that and it 8

In a 10d network its 20, double that its 40

you would need 40 clients in that specific position to block one single node

[wild estuary]

40 is definitely possible what

spend 3 days

[prisma flax]

Not to have in that exact formation

[wild estuary]

waiting

[prisma flax]

Just for the person to disconnect and reconnect and have a new position...

[wild estuary]

the strength of S/Kademlia is you dont need to rely on "you need 40 people"

clients are distributed evenly

[prisma flax]

Thing is you dont need 40 people

[sharp osprey]

i could just be going rogue and blocking random nodes to disrupt the network

[prisma flax]

The way the relaying works is it forms a round-ish shape naturally

[wild estuary]

what about islands

[prisma flax]

In a high dimensionality network youd need hundreds if not thousands of nodes to block even a single connection between two users

Actually much more

[wild estuary]

if i only have 40 neighbours i only need 40 people to entirely block that out

[prisma flax]

Alright this is a good question and I thought of it. If you are at (10,4) you connect to all the people around you. But on your left, other than (9,4), you'll also connect to (8,4). And if (9,4) disconnects then you'll also connect to (7,4). You always have 2 connections in each direction

[sharp osprey]

what dimension are you talking about the formula is just 2 * that nth dimension so for thousands of nodes i would define like min a 500 dim network?

[prisma flax]

But you cant choose the position you get placed at

[wild estuary]

why?

[prisma flax]

You cant just put 40 clients in the right positions

[prisma flax]

You are placed where the fastest route from the origin you joined can place you

[wild estuary]

how is that verified?

what if you control the origin

[prisma flax]

The origin you joined, theres no single factual origin

[wild estuary]

.

[prisma flax]

Still, even if you control an origin thats used by many people you cant place yourself wherever you want

[sharp osprey]

u just mean that i get connected to whichever neighbor node accepts my request the fastest?

[prisma flax]

Yeah

And then he connects you to your neighbours

so you dont depend on him anymore

[sharp osprey]

also this

[prisma flax]

A 10000 people network would form a shape similar to a 100*100 square

So the most steps (in 2d, imagine in 10d) in this case is 200

[wild estuary]

i continue to fail to see how this is better than a DHT

[prisma flax]

In 10d even 10000000000 people (10 billion) would form a 10x10x10...x10 (10 times) shape

So at most 20 steps to get to anyone out of 10 billion people

[sharp osprey]

i am not asking no. of steps to get to someone u were mentioning how u would need a lot of devices to block out a person

[prisma flax]

Yes

[sharp osprey]

so technically for 10d u would need 20 to meet the number of thousands u would need a min 500d network

[prisma flax]

40, its two-hop so you need twice as many

[sharp osprey]

so a 500 int array for an address seems particularly to heavy compared to just a hash or some unique identifier

[prisma flax]

But the thing is that with 40 people, staying weeks to get the exact right position

Youd block a single node

[sharp osprey]

hmm for 2d u would need 4 ppl right to block a node? so isnt it just 2*n or am i missing something

[prisma flax]

Nope, 8 people. You connect to the second closest person in each direction too, so that if the closest disconnects you dont just not have a direction anymore

So you always have 2, not 1, connection in each direction

[sharp osprey]

oh i see

[prisma flax]

I wish I could make a visualization of why it's practically impossible to do any harm to a 10d network using this protocol but I wouldnt know how

Or some mathematical formula

[sharp osprey]

i think u should still include ur above thoughts and details in a rfc so u need not reexplain this to every next person you're talking with

[prisma flax]

Whats a rfc?

[sharp osprey]

request for comments

its basically how any idea or protocol gets introduced

https://www.rfc-editor.org/rfc/rfc5321.txt here is an example for smtp

[prisma flax]

Oh alright thanks

Ill go check it out

[sharp osprey]

its not code its just working theory

and proof how it negates all edge cases

[prisma flax]

Btw thanks @wild estuary and @sharp osprey for criticizing the idea. I have to think about how it's better (if it's better) than just DHT lookup. I don't know how to answer that now

[sharp osprey]

you should look at other decentralized algos or reasearch papers too

[prisma flax]

Ive looked at Matrix for now

[sharp osprey]

there might some other similar stuff or alternatives than dhts as well

[prisma flax]

Also https://github.com/StraReal/Cryptic is kind of decentralized as anyone can have their own signaling server but not good enough.

I want to implement the protocol I'm talking about in Cryptic

[sharp osprey]

https://en.wikipedia.org/wiki/Content-addressable_network check this out

sounds close enough to what ur saying

[prisma flax]

Ill check this out too then

Also I think I found a possible reason why it's better than a DHT

DHTs are brittle to frequent joins/leaves unless carefully managed, while this system can self-heal through local reconnection (no global rebalancing).

DHT is more efficient but this is more resilient and better structured

Also there's no assumption of content addressing. The mesh exists purely as a communication graph. You can later layer storage, messaging, or computation on top.

[prisma flax]

@sharp osprey I drafted a RFC, it's still lacking some things but I'm too tired to finish it right now.
It explains the basic concept pretty well though.

[errant bay]

Please react with ✅ to upload your file(s) to our paste bin, which is more accessible for some users.

[prisma flax]

Its in a .odt file but this server wont let me upload it

[exotic yew]

A bit of an unfortunate abbreviation

[prisma flax]

Yeah Ill probably give it another name im not really fond of three letter names

[exotic yew]

It's also the name of a concept that's very hype these days, so people will have trouble finding your thing.

[prisma flax]

What is it?

[exotic yew]

Model Context Protocol for LLMs

[prisma flax]

Oh yeah I actually did look up MCP and saw that but I thought it was just a niche thing I could ignore

But I never wanted to name it MCP

[exotic yew]

It's not niche at all

[prisma flax]

It's just temporary

[wild estuary]

also solved by kademlia

peer lists are frequently refreshed and number of "neighbours" can be adjusted based on churn rate

[prisma flax]

Kademlia isn't zero-trust

It assumes cooperation

[vagrant cove]

does anyone know how to download the NSL-KDD dataset?

-or where

[tulip gull]

does anyone know how to use speech_recognition library

[ruby jolt]

im having a hard time tryin to understand what a multi-cast is at ipv6 precisely how the Neighbor Solicitation from the DAD works... i guess precisely whats tripping me how multi-cast happend without going through the router.. how dont is not the router involved and how the message reaches its local destination

[latent onyx]

Use kaggle
Link : https://www.kaggle.com/datasets/hassan06/nslkdd

[ember ledge]

https://jrocnet.com

[carmine prism]

today's rabbit hole was optimizing websocket masking in pure Python, i had an idea to use memoryview striding to mask 4 bytes at a time, but when i looked at the websockets library implementation i saw that you can do it without any iteration at all:

def apply_mask(data: BytesLike, mask: bytes | bytearray) -> bytes:
    """
    Apply masking to the data of a WebSocket message.

    Args:
        data: Data to mask.
        mask: 4-bytes mask.

    """
    if len(mask) != 4:
        raise ValueError("mask must contain 4 bytes")

    data_int = int.from_bytes(data, sys.byteorder)
    mask_repeated = mask * (len(data) // 4) + mask[: len(data) % 4]
    mask_int = int.from_bytes(mask_repeated, sys.byteorder)
    return (data_int ^ mask_int).to_bytes(len(data), sys.byteorder)

this is about 10x faster than my solution, which is 3x faster than the naive byte-by-byte XOR

[carmine prism]

the moral of the story is Python doesn't mind XORing integers of unlimited size

[somber quiver]

Working on a project, and I'm writing up the spec of the API for it. Is there a better way anyone would recommend formatting the packet in the text (not how I have the packet setup, but my diagram)?

[somber quiver]

I decided to use the format of: MsgType: uint32, etc. instead of how I had it there as it seems a lot more concise.

[lyric oak]

The IETF RFC documents use ASN.1 for that, but you can choose whatever you feel convenient for you and your audience.

[somber quiver]

Okay thank you

[tidal juniper]

https://github.com/lwthiker/curl-impersonate

for this library does anyone know the windows support for it

also

when bypassing this
https://www.scrapingcourse.com/cloudflare-challenge

does everyon use

or do you guys use this

this is all from header

[wild estuary]

... why exactly do you need this

[tidal juniper]

to get around this

https://www.scrapingcourse.com/cloudflare-challenge

curl doesn't work with -h attached cf_clearance

so trying other stuff

headless=False browser w selenium works
but can't get headless browser to work :/

[cloud spruce]

!rule 5

[errant bay]

Rules

5. Do not provide or request help on projects that may violate terms of service, or that may be deemed inappropriate, malicious, or illegal.

[cloud spruce]

captchas and such challenges are there for a reason, we will not help you circumvent them

[tidal juniper]

Protected by the 1st amendment btw

Also I’m literally asking if a library exists not for help

But sure whatever

[grand kayak]

lol I checked your link earlier and it was offline...
then I knew about cloudflare's outage XD

makes sense

[cedar forum]

it's not something we allow here & the first amendment doesn't apply to discord servers, even discussing libraries is against our rules here unfortunately. almost all cases where people want to do this are nefarious.

[tidal juniper]

i mean fair just annoying i cant get help w this project XD

the 1st amendment comment was just a joke

and wdym u guys dont discuss libraries

[lyric oak]

You rarely get help on kind of abandoned projects like that one here or at another place, and the maintainers are not on this channel. But it has been forked by someone with a business behind, so you might want to check out if that fork (and its new maintainers) is worth your time.

[cloud spruce]

we do discuss libraries, as long as they doesn't break the #rules or #code-of-conduct of this server
but any project, library, code or even techniques that aims to circumvent bot detection or human verification measures like captchas or similar are clearly against rule 5 and thus nothing we will help with here in any capacity what so ever

[undone quartz]

does anyone know of an easy way of transferring tcp over udp? ive made a udp holepuncher, but it doesnt support tcp so i need a translation layer, and it would be great if a good one already exists

[grand kayak]

hey!
have you tried openvpn? it uses udp by default to create a tunnel

[undone quartz]

can it be pointed to a udp socket?

my holepunching program connects 2 unportforwarded computers, and makes a socket on both, where you can send stuff to it from localhost and it bridges it to the other side

[grand kayak]

hmm good point, let me see if I got it right.
Is your program listening in any localhost ports?
and just spitting traffic out in the other end, also in a UDP port?

[undone quartz]

it creates a bidirectional socket at a port you choose. if it gets data from localhost, it gets forwarded to the other computer.

however that socket also sends additional packets between the two computers to keep the connection alive

when a non-local packet is received, it gets forwarded from that socket to the last connection that sent something to the socket

its all UDP ports, i dont think it can even be done with TCP

ive made it to play games peer to peer with my friend, since my internet is really bad, but since it only supports UDP a lot of games dont work with it.

my hope is to use some program, that has a TCP socket, connect that program to the UDP forwarded socket, and in this way get TCP games working as well. i tried to just send the packet body over the forwarded socket, but quickly realised that im stupid and packet loss exists, so that didnt work

ive seen a protocol called UDT, which is basically reliable UDP (what i want), but the library implementing it is confusing to install and im not sure if its cross platform

another option i have is to somehow tell the os to give me the raw packets, which i then wrap in UDP, but i havent found a way to do so

[grand kayak]

I would 100% try out openvpn at first.
It supports connecting via UDP port, you can specify the port number on both ends i think

once connected to the VPN you just use TCP normally inside that network

[undone quartz]

would openvpn seamlessly connect to another instance of itself? and what address would i need to go to, to access, for example, the game on the other pc?

[grand kayak]

if the additional overhead makes the game too laggy, then you can look into other miscellaneous solutions

[undone quartz]

i dont really want to bother with making an openvpn server

[grand kayak]

openvpn is a client-server protocol.
in one end, you boot up a openvpn server, in the other end you connect to that server

[undone quartz]

aw

[cloud spruce]

you might want to turn you program into a kind of VPN solution that wraps ip packets of any kind in your udp packets (that should not be lossless, that is the job of the tcp stack that you are transporting packets for, and you don't want retransmissions in two layers as that is a recipe for really bad behavior)
your program would create a virtual network interface that would route traffic to the other peers virtual interface creating a general use tunnel between the two of you

[undone quartz]

would that work cross platformed?

[cloud spruce]

it would be more or less whar GRE does but with hole punching ability built in

[cloud spruce]

sure, but you probably need specific code for each platform you want to support to create that virtual interface locally, unless you can find a ready made library that takes care of that and abstracts that away for you

[undone quartz]

yeah i was just hoping to find something to slot in without much work...

[undone quartz]

whats GRE?

[cloud spruce]

don't think that code would be python native anyways, ot would be more like a library written in some other language and then with python bindings if you can find such a library

[cloud spruce]

https://www.cloudflare.com/learning/network-layer/what-is-gre-tunneling/

it's a protocol that let's you encapsulate ip packets within other ip packets with a very thin layer in-between

[undone quartz]

cant that be used then? it seems like the perfect thing

[cloud spruce]

not directly, it's a protocol directory on-top of the ip protocol, not on-top of udp

also, GRE is quite ill-suited as there is no security built into the protocol, so if you want to have any security the application traffic with the tunnel would need to be secure or you need to wrap it in something like ipsec

i only compared it to something quite close to GRE, not really saying you should be using that specifically

but what i think is key here is to make a virtual network interface that passes the traffic through your process that would wrap ip packets for that traffic in those udp packets
that would make it general use and you would be able to transport whatever ip based traffic you wish over that tunnel

[cloud spruce]

@undone quartz on linux and bsd this would be a "TUN" interface and on macOS you got "utun"
but i don't know if there is something native like that that comes standard with windows, so you may need to require a prerequisite of installing the openvpn tun driver, or the wintun dll (from https://www.wintun.net/) originally from the wireguard project

[lilac sail]

use camoufox

[restive grotto]

GRE is used from router to router. but it also extends all broadcast and multicast traffic across the link. so whats often done is to put GRE inside IPSEC, as ipsec cant do those things

ipsec adds the encrpytion and gre adds the lan extention and other tools

yes its techainly vpn in vpn with the overhead

thats added

[magic rock]

glad thats not your ip

[strong pond]

!code

[errant bay]

Formatting code on Discord

Here's how to format Python code on Discord:

```py
print('Hello world!')
```

These are backticks, not quotes. Check this out if you can't find the backtick key.

For long code samples, you can use our pastebin.

[strong pond]

Hello i'm writing a python client that connect minecraft server lobbies and make it possible to remotely change the amount of ram it should use etc. In the code there is a function called "new" that breaks the number 1. rule in programming:
Don't repeat. And I want to simplify it from like 20 lines to 15

def new():
    new = tk.Toplevel()
    new.title("dashboard")
    label2 = tk.Label(new, text="dashboard lobbies 1-x", font="Arial 16")
    label2.grid()
        
    #lobby1
    ram1 = publick.get("lobby1", {}).get("ram", "") 
    max_ram1 = publick.get("lobby1", {}).get("max_ram", "") 
    lobby1_ram_entry = tk.Entry(new); lobby1_ram_entry.insert(0, f"{ram1}"); lobby1_ram_entry.grid(row=1, column=1)
    lobby1_max_ram_entry = tk.Entry(new); lobby1_max_ram_entry.insert(0, f"{max_ram1}"); lobby1_max_ram_entry.grid(row=1, column=2)
    start_button1 = tk.Button(new, text="start"); start_button1.config(command=lambda: threading.Thread(target=run, args=(1,)).start()); start_button1.grid(row=1, column=3) 
    
    #lobby2
    ram2 = publick.get("lobby2", {}).get("ram", "")
    max_ram2 = publick.get("lobby2", {}).get("max_ram", "") 
    lobby2_ram_entry = tk.Entry(new); lobby2_ram_entry.insert(0, f"{ram2}"); lobby2_ram_entry.grid(row=2, column=1)
    lobby2_max_ram_entry = tk.Entry(new); lobby2_max_ram_entry.insert(0, f"{max_ram2}"); lobby2_max_ram_entry.grid(row=2, column=2)
    start_button2 = tk.Button(new, text="start", relief="flat", width="200px"); start_button2.config(command=lambda: threading.Thread(target=run, args=(2,)).start()); start_button2.grid(row=2, column=3)

[steady horizon]

To keep this sane

def new():
    new = tk.Toplevel()
    new.title("dashboard")
    label2 = tk.Label(new, text="dashboard lobbies 1-x", font="Arial 16")
    label2.grid()
        
    #lobby1
    ram1 = publick.get("lobby1", {}).get("ram", "") 
    max_ram1 = publick.get("lobby1", {}).get("max_ram", "") 
    lobby1_ram_entry = tk.Entry(new); lobby1_ram_entry.insert(0, f"{ram1}"); lobby1_ram_entry.grid(row=1, column=1)
    lobby1_max_ram_entry = tk.Entry(new); lobby1_max_ram_entry.insert(0, f"{max_ram1}"); lobby1_max_ram_entry.grid(row=1, column=2)
    start_button1 = tk.Button(new, text="start"); start_button1.config(command=lambda: threading.Thread(target=run, args=(1,)).start()); start_button1.grid(row=1, column=3) 
    
    #lobby2
    ram2 = publick.get("lobby2", {}).get("ram", "")
    max_ram2 = publick.get("lobby2", {}).get("max_ram", "") 
    lobby2_ram_entry = tk.Entry(new); lobby2_ram_entry.insert(0, f"{ram2}"); lobby2_ram_entry.grid(row=2, column=1)
    lobby2_max_ram_entry = tk.Entry(new); lobby2_max_ram_entry.insert(0, f"{max_ram2}"); lobby2_max_ram_entry.grid(row=2, column=2)
    start_button2 = tk.Button(new, text="start", relief="flat", width="200px"); start_button2.config(command=lambda: threading.Thread(target=run, args=(2,)).start()); start_button2.grid(row=2, column=3)

This shouldn't be a new function. This should really be two classes. One for each lobby thing which is repeating throughout this function and one for the top-level widget encompassing this whole thing

As this problem has nothing to do with networks, you should probably continue this in #user-interfaces

[strong pond]

thank you

[steady horizon]

You're welcome

[round skiff]

Hey guys! I have started mastering aiohttp library by making a robust asynchronous scraper to understand deeply how network fundamentals work. Before diving into the library and making a code. What sources of information would you recommend to learn as a theoretical part like: HTTP (overall) , TCP/IP and etc. Btw , what certain networking concepts should I know to cope with my project? Thanks in advance!

[grand kayak]

hey there, nice to meet you

I recommend starting with networking fundamentals.
my favorite resource is the book Computer Networking: A Top-Down Approach

this book is single-handedly the best resource for beginner to advanced.
it has the best explanations about every important concept, and the content is ordered in the easiest way to follow.

a mini roadmap would be

intro (packet switching vs circuit switching) -> application layer (HTTP, DNS, SSH etc) -> transport layer (TCP and UDP) -> network layer (IPv4, CIDR, NAT, subnets and supernets) -> link layer (mostly about Ethernet but also STP and VLANs) and physical layer (coaxial, UTP, fiber, collision)

book by Kurose and Ross btw

[round skiff]

Sounds too good for me , thanks, appreciate it!
Have you heard of book called "
TCP/IP Illustrated: The Protocols, Volume 1 (Addison-Wesley Professional Computing Series)TCP/IP Illustrated: The Protocols, Volume 1 (Addison-Wesley Professional Computing Series)" I suppose it is more for advanced? I firstly heard about it from Coding Jesus channel (he has too much great videos btw).
And one guy recommended me to read "Beej's guide to network programming" but it touches C examples , which I am not familiar with, I only know python for now , however, I am still interested to learn low-level fundamentals and get deeper knowledge about computers overall.
So what book to choose? Probably, I will consider it in an online format

[grand kayak]

trust me on this one, get the Kurose book.

you will not regret it.

but of course you can still read TCP/IP Illustrated as a complimentary resource

if you don't understand something well enough with a book, you can always read the same topic in the other one 😅

but the best read is Kurose. by far

[round skiff]

yeah of course , thx a lot! have a good day!

[grand kayak]

np anytime! study well

[round skiff]

Vielen Dank! 😁

[strong pond]

Anyone know a library or anything to make a basic FTP server?

[grand kayak]

do yuou have to make an ftp server? like is it an assignment? or do you want to learn and dive in on your own?

[strong pond]

I just want to learn it.

[grand kayak]

I recommend coding challenges
https://codingchallenges.fyi/challenges/intro

he doesnt have an FTP challenge yet, but there's building HTTP server for example
and the build your own dropbox as well

if you are sticking to FTP though, it's all good

I think you can use python's socket lib https://docs.python.org/3/library/socket.html

ftp uses both TCP and UDP. it has to run in multiple threads or something like it

[cloud spruce]

i would really recommend that you skip FTP as a protocol to learn about implementing network protocols, it's really old and arcane, it has really bad design decisions that aren't very relevant in today's world but might have been relevant back when it was designed some 40 years ago (and maybe in part relevant up until about 15-20 years ago)

[cloud spruce]

do you have a deep understanding about bits, bytes, bit and byte order (little-endian, big-endian, network byte order, least or most significant bit first), bit operators, hexadecimal, octal number systems and such more general computer knowledge?

if not, start there first, it will be very important if you are going to learn low level networking as the protocols are specified down to single bits

you can learn the protocol stack top-down or bottom-up, depending on what you want to prioritize first (from the application protocols done to more detailed low level details or understanding the low level details first and work your way up to higher levels to add more practical functionality)

either way you will want to learn about important auxiliary protocols such as DNS, ICMP, ARP and DHCP and concepts such as NAT (Source NAT and Destination NAT), subnetting and routing and switching in addition to the more immediate protocols such as HTTP, TCP, UDP and IP (start with IPv4, you can dig into IPv6 much later as that will affect quite a bit of stuff) and eventually TLS, QUIC and DTLS

@grand kayak darn, sorry, i meant the first of those two last messages to be directed at @strong pond, but responded to your message (which was also a good answer to his message) by mistake

[cloud spruce]

@strong pond see this message 👆 (that i'm responding to with this message) that should have been directed towards you

[grand kayak]

it's all good =)

do you know of any good networking roadmaps? I have searched in roadmap.sh but there seems to be none

[cloud spruce]

no, not a roadmap as such
but what are you wondering about more specifically?

it sounds like you know some, at the very least, if not quite a bit, of it already
in general i don't personally think roadmap.sh is all that accurate much of the time (especially the ordering and grouping of the different subjects)

[grand kayak]

I'm thinking of creating a roadmap like that to send to people here in the channel

A beginner friendly order of stuff to learn

You can create your own roadmap inside roadmap.sh, I do it all the time

And yeah I have heavy work experience with networks and programming

[cloud spruce]

i can really relate to that last part 😄

[round skiff]

Thanks!

[strong pond]

Thanks

[boreal gull]

https://discord.com/channels/267624335836053506/1447013470485741699

[tidal juniper]

Connection error: It seems you are trying to reach a Socket.IO server in v2.x with a v3.x client, but they are not compatible (more information here: https:/
/socket.io/docs/v3/migrating-from-2-x-to-3-0/)

does anyone know if I uninstall socket.io-client
and install the v2.4.0 of socket.io will it fix this error?

seems to have worked

nvm

[real cape]

how do you test your network apps?

like how do you run in an enviroment that isn't localhost?

[grand kayak]

if your only constraint is not using localhost, you can spin up a VM (using VirtualBox for example) or a container network (with docker)

are you familiar with any of these?

or do you mean hosting it on the internet?

[real cape]

no no, just testing client-server/p2p apps

VM's might be heavy for my pc (docker might've too)

i have 8gb's of ram

i found "mininet" online, does it sound familiar?

i'm looking into it rn

[grand kayak]

oh yeah, actually my research project uses mininet heavily XD

but I don't recommend it to you, it's specifically for studying SDN

you can spin up a ubuntu server VM for example with 1gb ram it should work OK

and won't slow down your PC

even with 8gb ram

or you can use docker

mininet is too niche

[real cape]

what do you use for vm's?

[grand kayak]

virtualbox, you can get a ubuntu server image from the internet and install it via the ISO image inside virtualbox

have you ever used VB?

[real cape]

no, but i use vmware

[grand kayak]

if you already got vmware configured in your pc, even better

spin up 1gb ram VM and install the ubuntu server image

that of course only applies considering your app can run on 1gb ram

[royal gust]

Is it better to make your flask local or by a app

[woven pike]

Can anyone help me

network project

[grand kayak]

What do you need help with exactly?
What is the project scope? And where did you get stuck?

[prisma cobalt]

What do you mean by this when you say by a app?

[vast talon]

hello

im having an issue on a game related to routing (i think)

if someone who knows how internet works could help me it'd be very awesome

[steady horizon]

What are you struggling with

[unreal jay]

how do i download pip ?

[lyric oak]

As long as you downloaded Python, it is already there as python -m pip.

[unreal jay]

ok thanks

[grand kayak]

try typing in import pip

[unreal jay]

it didn't show anything

[grand kayak]

yes -- this means pip is in fact installed -- because you didn't get an import error

[unreal jay]

oh thanks

[grand kayak]

what you're seeing right now is the python IDLE (which is a python specific CLI)

to actually use pip you must open a windows CLI

which is the windows CMD or you can use windows terminal (recommended)

then you can run pip commands

[normal oyster]

in some versions of python there is get-pip.py file somewhere which installs pip for you if you don't have it installed

running get-pip.py should get you pip, however it is discontinued in newer versions of python, but it should still work

[cloud spruce]

you have probably already solved your problem with the help of @grand kayak at this point, as the problem you had was that you where using the python REPL (within IDLE) instead of CMD or PowerShell or some other shell (such as bash, zsh or similar)

but for future reference:
one can often use ensurepip from the python standard library, it's an optional module, most python distributions will have it, but it might not be available in all python distributions

python -m ensurepip

get-pip is a stand-alone project (note to @normal oyster ,it's not discontinued) made by PyPA (the official Python Packaging Authority) and is available as a zipapp https://github.com/pypa/get-pip

if one is using python installed by a package manager pip can quite often be found as an additional package that might not have been installed yet and needs to be installed with the appropriate command for that package manager, some of which are listed here: https://packaging.python.org/en/latest/guides/installing-using-linux-tools/#installing-pip-setuptools-wheel-with-linux-package-managers

more information can be found in the official guide: https://pip.pypa.io/en/stable/installation/

[errant bay]

Read-Eval-Print Loop (REPL)

A REPL is an interactive shell where you can execute individual lines of code one at a time, like so:

>>> x = 5
>>> x + 2
7
>>> for i in range(3):
...     print(i)
...
0
1
2
>>>

To enter the REPL, run python (py on Windows) in the command line without any arguments. The >>> or ... at the start of some lines are prompts to enter code, and indicate that you are in the Python REPL. Any other lines show the output of the code.

Trying to execute commands for the command-line (such as pip install xyz) in the REPL will throw an error. To run these commands, exit the REPL first by running exit() and then run the original command.

[normal oyster]

my bad, I was sure I've read that get-pip.py is deprecated in newer versions of python than 3.10.
I think I saw that somewhere in https://github.com/pypa/get-pip/

[gloomy summit]

✩hi,i try to write a not prefect how to use the httpx markdown nya~~ ✩

✩who is want to see it nya~~?✩

✩if someone want to read it,i will send nya~~✩

✩it idea is from htppx quick start part nua ~~✩

✩i know how to update post data to the https server ,and received the different https sever status code nya~~ ✩

[gloomy summit]

https://github.com/aliceSmartCastle/httpx-learned

[neat fog]

Hey how to design apis and connect it to web?

I know how to connect to the front end

But I get confused while creating apis

It seems like I am not clear that what apis will I require

So how to plan that?

[gloomy summit]

✩maybe ,you read httpx offical document ,you wiil know how to do it~~✩

✩explame,test the connecting to the wed status code ,or headers information nya~~✩

[cloud spruce]

it's different for each project and what your intentions with the api is, like who is going to consume it (is it for a specific site or an open api) and what is it for?

[quaint girder]

if there is anyone who would be able to teach me a few things about connects a ssh to a vps and running your network traffic through the ssh tunnel please dm me!!

[plucky nova]

pain

[cloud spruce]

and this has to do with python in what way?

[fringe lantern]

hi guys i made this two blade of that facedev who made rugpool clone in python anyone wanna check it out
:https://github.com/skandapypatriot/ycapemailprotocol

[wild estuary]

ai slop 😔

[meager flower]

so corny

[neat fog]

its mixed like some apis users can directly hit to download data and some apis will be used when user do specific analysis from the web app

[oblique imp]

So sad that now it’s almost everywhere in coding industry and not everyone is willing to put in the work to learn

[grand kayak]

this was true even before AI, i.e. script kiddies

but yeah now it's even easier to not study programming and get away with it

[meager flower]

Agreed. And it’s only going to get worse

[urban spoke]

Would this be the appropriate channel to talk about ansible and terraform?

I’m a network engineer trying to learn about network automation

[cloud spruce]

if it's ansible or terraform in general i would think #tools-and-devops might be better suited, but if it's for network automation i guess here might work too

try, and ask your more specific question

[urban spoke]

I want to get more familiar with network automation I believe that the modern day network engineer needs to have proficiency with python

I have lost too many job offers because I didn’t have the necessary python knowledge

So I guess a question would be:

what are some examples of network automation with Juniper, Cisco or Fortigate equipment (router or switch)

[errant temple]

Did they provide any more information as to why python was needed? I'm not a network engineer myself but I would imagine configuration management tools/configuration management as a concept would be more important like just being able to understand and use ansible and automating everything, not clicking through interfaces or doing unnecessary manual work.

[urban spoke]

Software Defined Network

Utilize tools like Terraform and Ansible to build, maintain, and integrate automation scripts that streamline operational tasks and support project delivery.

Automation: Working knowledge on network automation, specifically using Terraform and Ansible Automation Platform.

Automation First: Proven experience in scripting and automation tools (like Redhat Ansible, Terraform etc.) to significantly reduce operational effort and improve network reliability.

Strategic & Agentic AI: A foundational understanding of how agentic AI and Generative AI tools can be leveraged to further streamline workflows, move beyond simple task automation, and ultimately free up our cycles for high-value strategic thinking and innovation

[cloud spruce]

knowing programming in general (and often python more specifically) at least helps in many aspects of automation, but it's not really necessary for [network] automation if you are using ansible and/or terraform and all the functionality you require is already available through existing modules/providers
on the other hand, if you need to do more custom things you might need to integrate it yourself with a programming or scripting language

another tool that i think you will be working with extensively is revision/version control systems and source code management (for example git)

if it's about building tooling for others to use you will definitely need to be able to write code in a programming or scripting language of either you or [sometimes rather] your employers choosing

[urban spoke]

Nah it’s more working with ansible and terraform

Where could I find some examples of these two software‘s working with switches and routers?

[cloud spruce]

for cisco, as an example, you can start your research at https://developer.cisco.com/iac/ and sometimes you can find examples in reputable github repositories

[urban spoke]

@cloud spruce if you could send some of those repositories on GitHub, that would be great

[summer dragon]

hey yeah

[karmic jay]

hi

[somber quiver]

Working on a voice chat server for a project, obviously has live streaming of audio. Any recommendations for how to encode the audio packets?

[oblique loom]

why da faq is my vm showing netowrk speed 4.2mb?
it literally shows 500mbps on my original OS
VM has python too

[prisma cobalt]

webrtc is a great starting point and well supported, even in the browser.
as for the actual, specific encoding, have a look at the Opus codec for audio

[pure cloak]

Anyone ever came across any resources to learn how vpn is created.
Like a simple hop through an ec2 instance is this it?
Are there more intricacies?

[signal carbon]

Am curious to learn python wifi hacking...

[errant temple]

I don't think anyone here can answer or help with this on account of rule 5.

!rule 5

[errant bay]

Rules

5. Do not provide or request help on projects that may violate terms of service, or that may be deemed inappropriate, malicious, or illegal.

[signal carbon]

Yeah

[rustic pollen]

hey guys im getting this error

how do i resolve this

running this on docker container within a vm using container labs for network simulation

andx86_64 architecutre

[pallid sierra]

The issue occurs because the script needs deeper access to the network, but the current system setup doesn’t support that level of access i think that the issue

[cloud spruce]

!rule 6

[errant bay]

Rules

6. Do not post unapproved advertising.

[cloud spruce]

both your messages are more advertisement more than anything else, which is against our #rules , please read up on then again

!rule 10

[errant bay]

Rules

10. Do not copy and paste answers from ChatGPT or similar AI tools.

[grizzled juniper]

guys how is workng as data scientist be like? you're always training models?

[grand kayak]

hey this channel is about computer networks, not data science

ask in #career-advice or #data-science-and-ml

[grizzled juniper]

oh sorry i thought this was about neural networks

my fault

[cloud spruce]

what does this has to do with python or computer networking (which this channel is about, not "social networking" )?
if you want to just talk about whatever (as long as you still follow the #rules and #code-of-conduct) you can do that in one of the three off-topic channels, for example #ot0-psvm’s-eternal-disapproval
also, this reads a bit like some kind of advertising for "making money fast", which feels boarder line like a "hidden" scam attempt (trying to skirt the rules)

[ocean kiln]

!clban 1448078344930463794 dropshipping scam

[errant bay]

:incoming_envelope: :ok_hand: applied ban to @mint dagger permanently.

[dawn saffron]

i

@lunar furnace

[spice wind]

Assuming I had 10 packets that could all be combined into 1 packet, would it be more performant over the network to send the large packet or the 10 smaller packets individually?

[boreal scroll]

Sorry if this isn’t the right place, but I’m working on a proxy for Minecraft Bedrock Edition that forwards clients to a target server and can modify packets. I’m currently using PrismarineJS in Node.js, but I’m not very comfortable with JavaScript. Is there a Python library that supports the latest Bedrock protocol?

[cloud spruce]

yes, it would be more performant from a bandwidth perspective as each packet has overhead in the form of quite a few headers in the different layers of protocols that is used to transport the data

but it might be better to use small packages if it is in a latency critical application where you can use or present the data independently of the othere pieces (for example a voice or video call using codecs that encode a fully independent part in each of the 10 pieces, or say when you type characters in a ssh session) as otherwise you would have to wait for all the data to be collected/generated and accumulated before it can fill a full package (for example a MTU of 1500 bytes for IPv4 on the internet) and be sent (if you wanted to maximize bandwidth utilization/efficiency)

[spice wind]

Hmmm, fair enough! Thank you for the response!!!

[lunar scroll]

hello guy's!

[torn sable]

It would be so nice if socket.create_connection implemented RFC 6555
I see that there's already an issue for this but don't see any movement on it :(
https://github.com/python/cpython/issues/88810

[unkempt marlin]

Hello everyone I am new to Networks and I have a question

How can I differentiate between twisted pair cables if they have the same outer jacket?

Because RJ45 seems hard to recognize

[cloud spruce]

the RJ45 connector is really easy to recognize, and all normal eithernet copper cables with RJ45 connectors that i can think of are "twisted pair"
but are you getting at which are straight-through and which are crossover cables maybe?

[unkempt marlin]

I mean the RJ45 can be recognized with colors or what?

In cat5-5e-6 etc

cat5 and 5e looks very similar to each other

[cloud spruce]

they look the same, yes, but the category should be printed on the cable

[unkempt marlin]

Yeah I see that

[sleek breach]

the colors depend on the standard the company uses, theres a standard t568a and a standard t568b

the color of the cable itself doesnt mean anything, you will always rely on the cables labeling in the feild

[nimble galleon]

Hello everyone I’m new to networks n coding’s what things i should learn first

[prisma cobalt]

I learnt python sockets first and then moved onto internet infrastructure and web technologies

[nimble galleon]

Can you tag me in that chat please

Also looking for like ppl to help me if anyone will be ok wit helping me learning stuff …

[steady horizon]

If you want a deep understanding you should read the Wikipedia or the RFC for protocols like IP, UDP, TCP or even ones like Ethernet and ARP, or any protocol that you would like to learn about. Use a packet sniffing tool to monitor and analyze network traffic for live experimentation.

[wispy panther]

Using the socket api in C will also really help

[rocky grotto]

Can you. Tag me in the Wikipedia

That’s my other Acc

[cloud spruce]

what do you even mean by this?
you can just look up the protocols that you are interested in on wikipedia or either rfc-editor.org or datatracker.ietf.org if you want the real source of the full specification

[polar silo]

does anyone know any good proxies for web scraping? and unauthenticated ones preferrably.
using nodriver library importing the cdp functions.

just want to know how to use authenticated proxies with cdp, as so far ive only been able to use unauthenticated ones. thank you!

[cloud spruce]

and what's wrong with using your own source ip address?

[polar silo]

i just plan to scale thats all

but i feel that using proxies will be necessary

as some parts I try to web scrape for, dont work due to region lock

[cloud spruce]

so, to get around rate-limiting restrictions?

[cloud spruce]

and getting around geoblocking...

[polar silo]

yes that

[cloud spruce]

!rule 5

[errant bay]

Rules

5. Do not provide or request help on projects that may violate terms of service, or that may be deemed inappropriate, malicious, or illegal.

[polar silo]

is it malicious if its web scraping though?

im not asking to stress a server

lol

[cloud spruce]

that is not something we will be helping out with as that almost always are against their ToS, the restrictions are there for a reason and you are trying to circumvent them

[spring violet]

hey

[wooden rock]

!rule 5

[errant bay]

Rules

5. Do not provide or request help on projects that may violate terms of service, or that may be deemed inappropriate, malicious, or illegal.

[cloud spruce]

@sly nebula when we talk about ports on a router we are most often talking about L3 ports, so they won't be able to do switching, only routing, so stations on the two switches connected to two different ports in a router won't be able talk between the two switches without routing the traffic through the router that must be correctly configured for this
furthermore, protocols that require communication below L3 protocols won't work, such as broadcast data and the spanning tree protocol for example

[muted lark]

Hey I have a question, what is the best way to add Linux on windows, is that virtual machine?

[inland rampart]

the easiest way to get a linux environment on windows without reinstalling the system would be to install WSL2. also, not really a #networks question.

[muted lark]

Actually I think networking is important to know Linux

[steady horizon]

How is it any less important on other platforms? How is this more relevant here than in #unix ?

[cloud spruce]

not really, and it's still not a networking specific issue/question

[muted lark]

Wow I think you guys are talented In your career but not good with begginers , the admin pays you guys to answer networking question? Because you are just focused on being professional but guess what politely is more important than the subject , when the question is not in networking just talking , just be polite

[cloud spruce]

you have already gotten good advice about using WSL2 to run a linux environment on windows
and you have also been directed towards #unix for your linux related questions, but i don't see you writing anything there ( we unfortunately don't have a channel geared towards windows)

[uncut storm]

hey,can any one explain what is meant by port in networking?

[steady horizon]

It's an identifier. You usually pick one and communicate using only that one.
This means that sending packets and listening for packets on a specific port number is the same as sending and receiving/monitoring messages over a protocol.
Standard protocols use conventional port numbers. HTTP uses port 80 over TCP for example. So web servers will listen for packets on port 80, and web browsers would send requests to web servers to port 80 so the web servers will notice and process the web browsers' requests then send responses, etc.

[gloomy hemlock]

Anyone encountered an issue with boto3 where SSL negotiation takes an insane amount of time?

[somber hound]

What can i do to serve live video stream from an onsite machine behind symmetric NAT without egress cost hitting through the roof

[cloud spruce]

you would need a live stream distribution service as a cloud service that you connect to and broadcast through (only one video stream goes out through your internet connection that you initiate from you side, out towards the internet), then all clients can just connect to the cloud service and stream it from there, the cloud service will be replicating the stream to all clients that connects to it to be able to watch/listen to the stream/content

i think youtube live, twitch, facebook live, instagram live, vimeo, streamyard and mobizen studio offer such services for free (you can probably find others as well if you google it or ask a ai chat bot), some might require payment from you to unlock certain premium features if you require such extras
some of the services also offer the functionality of recording the stream and making it available on-demand later on if you like to offer that to your audience as well

which service to use depends on what kind of audience your content is geared towards and what platform they are on, as well as what kind of devices you are targeting (i think instagram live for example is mobile app only, and facebook live requires the viewer to have a facebook accountand be loggedin, while youtube is for both mobile app and computer and doesn't necessarily require any kind of account for the audience to watch the stream)

some of the above can be used directly from the respective mobile app or web page to broadcast from, others will require you to have a some third-party mobile app (i belive Prism Live Studio is free) or software on your computer (such as OBS Studio which is free and even open source and available for most computers) to be able to broadcast through their service

[stiff zinc]

anyone her can help where I start networking in python

[cloud spruce]

what do you want to know more exactly and what are you aiming to use it for?
the subject is just to wide without knowing what your current level of knowledge about computer networks is

[stiff zinc]

I can dm you ?

[cloud spruce]

sorry, i stopped using DMs here on discord entirely a few years ago
any particular reason for not wanting to chat in public?

[modest kiln]

hi im trying to implement raw socket for a port scanning script, it is a basic script inspired from nmap stealth scan (-sS). and im facing some problem, can someone help me ?

[steady horizon]

Probably

[onyx osprey]

WSL

[civic nacelle]

hi community

novice but have gotten my hands dirty over the last 6 mo. built & deployed a simple chess web app w django / postgres.

lately have been learning on a more fundamental level, re: sockets.

wondering if anyone can advise on clever tricks, tools, etc. on sockets that you've come across. my current favorite resources are the python sockets module, and telnet.

thx

[toxic mural]

not sure what you're asking for tbh. maybe look into the socketserver module

[languid isle]

scapy is huge if you wanna do more low level packet stuff

[civic nacelle]

thx @languid isle

[steady horizon]

You can learn the select and selectors modules.
If you know asynchronous programming with asyncio, you can dabble in its networking APIs too

[civic nacelle]

thx @steady horizon

[verbal echo]

hey guys how are you doing?

[cloud merlin]

Hi, mates, I'm a full stack developer.
I've hands-on experience in the development of web applications (dashboards/admin panels, e-commerce platforms, and social networks), API integrations (authentication systems and rest APIs), data pipelines (scraping, cleaning and storing data) .
My stack includes the react/next.js (sometimes wordpress) for frontend, node.js(express.js)/Django for backend and python frameworks and libraries (TensorFlow, Pytorch, OpenCV, NumPy etc) for AI/ML integrations. I'm good at AWS/Docker for infra and building scalable apps.
My focus is just building a real world products, but not demos. I'm passionate with my work and want to collaborate with more friends and great projects. I believe that effective communication and collaboration with experts are key to successful product development.
Thanks for reading carefully.

[wild estuary]

uh, this channel is for networking as in the way computers communicate, not networking as in the way humans get jobs /hj

[vale raptor]

<@&831776746206265384> malware script

[ocean harbor]

!ban 1472171737654235238 posting ddos scripts

[errant bay]

:incoming_envelope: :ok_hand: applied ban to @pearl osprey permanently.

[lyric oak]

Books like https://python-automation-book.readthedocs.io/en/stable/ can give you some basic ideas.

[modest kiln]

this is my project still not completed: https://github.com/NakshatraSirohi/TCPrax/tree/develop
also im new to git and github, so if there is any mistake or bad practice then pls lemme know it will help me improve and same goes for python

[clever bay]

google wsl2 or windows subsystem for linux for beginners . but if you're in windows 11 , just type WSL in the windows search. you can DM if you need assistance but you should be able to figure it out. but the process it pretty simple these days., enable hyper V in optionalfeatures, you can install and setup wsl in windows GUI or CLI. But in powershell wsl --help will show you more but you can download the distro directly to your local device from the internet with --web-download

[mystic arrow]

it's not for beginners

[steady horizon]

For beginners until otherwise proven not for beginners by you

[analog finch]

Does anyone here have network setup with mikrotik/ juniper router or switch and is interested in testing and working on a python project? The project is ready for testing but i have problem with mikrotik chr vm, the gns project will be good too if anyone has that kind of setup. https://github.com/DevNetSolutions/Networking/releases/tag/v1.0.0-alpha

[clever bay]

bro, you can begin anywhere. what are you talking about?

https://medium.com/softaai-blogs/what-is-wsl2-a-beginners-guide-to-running-linux-on-windows-5a7b2190f333

Exactly what I said to google, a beginners guide. Nobody ever said this is where beginners start. This isnt about you, its about their question.

[cloud spruce]

just booting a live linux distro from a USB stick can be a good start if you want to try out linux and see how it works on a specific hardware

[lyric oak]

Producing such a USB stick isn’t trivial for beginners either. WSL2 is just a few clicks away.

[prisma cobalt]

I'd say that creating a live USB is much less complex than configuring WSL

[cloud spruce]

i kind of agree with you that sinply running wsl --install is the easiest option
as long as you meet the prerequisites, which can be trickier for a beginner if they aren't already, as that means they must first:

• not be running windows in "S Mode" (probably not that common, and if they are you need to get out of tthat)
• enable hardware virtualization extensions such as Intel VT-x or AMD-V or SVM Mode in their BIOS/UEFI (and often this defaults to being disabled "for security reasons")

but i was more suggest a live linux distro on a USB stick over having to download and install virtualization software for windows (such as for example VirtualBox, or even VMware Worksatoon Pro which is now free since about one and a half year by the way as long as you don'trequireany official support) and get a VM up and running as was suggested by @ember ledge

there are lots of tools nowadays to create a bootable linux USB stick, for example UNetbootin, Rufus, UUI (Universal USB Installer by Pendrivelinux), Fedora Media Writer, balenaEtcher, Raspberry PI Imager or even an alternative like Ventoy, some of which will even download the ISO for you and put it on the USB stick in one go if you only have a USB flash drive to sacrifice and completely wipe

(and now i think this should really have been posted in #unix , but I'll keep it here to maintain context of the conversation)

[cloud spruce]

the above is probably best directed towards you, as you were the one asking originally more than two weeks ago

[jagged pagoda]

probaly (in my opinion) the easiest way to get familiar with linux for beginners, wsl

[versed falcon]

anyone who knows numpy

[next jasper]

You're in #networks, try #data-science-and-ml

[cloud merlin]

Hi, mates, I hope you are doing well.
I've experience in the development of web applications (dashboards/admin panels, e-commerce platforms, and social networks), API integrations (authentication systems and rest APIs), data pipelines (scraping, cleaning and storing data) and DevOps projects. These days, I've developed several AI products including Voice AI, video/image generation tool, LLMs and trading bot. I'm passionate in my work and constantly learning new technologies.
My focus is just building a real world products, but not demos. I'm just in the search of startups or developer teams to work with. Feel free to reach out to me if you are building/planning to build something great.

[steady horizon]

Reread this server's rules, notably rule 6. Read this channel's topic. Social/job networking isn't this channel's purpose

[mortal anchor]

@cloud merlin hey I am really intrested as well,but I am an absolute rookie in programming

[lyric oak]

This is not a support channel for Reddit. You can read the channel description and see what it is for.

[magic jetty]

Ok thanks

[frail cobalt]

Anybody suggest best channel for learning networking

[lyric oak]

“Networking” is a word with many meanings. No one can guess the context unless you reveal more.

[storm vessel]

Yeah

[civic forge]

e

[chrome token]

Yesterday was pi day

[ashen lintel]

Quick one, learned gents, before I waste time coding something. I'm a little disappointed that my new fibre line drops several times a day. It's quick, but shouldn't happen, so I want to monitor and log downtime. Is there something I can use for this?

[cloud spruce]

sorry for answering late, maybe you have already resolved this and found something on your own or with help elsewhere, is this still unresolved?

there are a lot of things that could be used to monitor your connectivity, how you do it depends on your circumstances and knowledge and how much/what you want to know about the problem exactly

what OS are you running and do you have something like a raspberry or something else that runs linux or other unix-like OS?

what kind of devices is the fiber connectd to and do you have administrative access to it? (this is probably the best source of information if you do)

[ashen lintel]

Thanks. I'm running Windows 11, with WSL, I totally own the fiber from the ONT thru the wifi router, and while I have a Pi that I thought would be great to use for this, I don't have a display. Thing is, all this really needs is polling with a ping command, and it doesn't seem like Win 11 has a tool that can do that, so it's probably a lot simpler, and some fun, for me to code something in Python.

[cloud spruce]

why i ask about the device that the fiber is connected to is to see if you have a log from that device about the interface going down and up again or other error messages (if the problem is on the line connecting to you or sometimes even if the problem is between you and the ISP device providing you with an IP address with DHCP)

and even headless a raspberry pi is great to ssh into and run your script on to monitor your internet connectivity with and log the results to file on, then shutting down or rebooting your windows computer will not affect your monitoring

and honestly, the easiest thing is probably to just use the shell to run a normal linux command for the monitoring

[ashen lintel]

Great, thanks. I haven't used my Pi for so long, I didn't think of headless, but I think that will be the way to go, and it will be fun to get it running again. I see my router has logging as well, but docs on that seem sparse.

[cloud spruce]

the logs on the router can be helpful to understand what the problem is, if it's close to you or if it's further in the ISP network, and if it is directly adjacent to you you might get exact timestamps for when it goes down and come up again
some routers offer the option to sent the logs to an external/remote syslog server/host, that can be really useful

[ashen lintel]

Yes, it's right next to me, the middle of 3 nodes here, me the router and the ONT, and I see it has a remote logging feature, so if I can get that sending to a web app running elsewhere the whole OS dependence is moot.

[cloud spruce]

i was think that if it's specifically sending syslog (which it usually is, rather than a web app based logging thing) you could setup rsyslog on the raspberry to receive the syslog and write it to file, rsyslog is very efficient and can handle fairly high volumes of syslog messages without requiring very much resources, so it should run well on modest hardware such as even an older raspberry pi
looking through the logs might help you find out the cause of the dropping line and you might want to reach out to your ISPs support team for help in that case
either way, when reaching out to them it can be good to have data about how often you are impacted and for how log, and even better if you can even provide them with timestamps for such events

[cedar forum]

i had a similar issue with isp drops and found https://oss.oetiker.ch/smokeping/index.en.html pretty good at generating some longer-term graphs whilst also being pretty lightweight (entire thing is just a perl app)

https://github.com/oetiker/SmokePing

you can also set up some basic alerting functionality (which is what I did) to log a timestamp to a file anytime something failed

https://oss.oetiker.ch/smokeping/doc/smokeping_config.en.html#I____Alerts____

[cloud spruce]

yeah, smokeping is nice

[cloud spruce]

@ashen lintel see above suggestion
also, the linux ping command has some really useful flags that the windows ping implementation lacks
to monitor for when you don't get a response from something you could run something like

ping -qnDOi 0.5 -W 0.2 IP_ADDRESS_OR_HOSTNAME_TO_REFERENCE_TARGET_HERE | perl -MPOSIX -wlpe 's/\[\s*\K(\d+)(\.\d+|)(?=\s*\])/strftime("%F %T$2 %Z",localtime$1)/eo'
```the perl stuff is just to convert the unix epoch time (seconds since 1970-01-01T00:00:00 UTC) to something more human readable
and if you want to monitor ping response time when things aren't down you'll just remove the `q` from the options, you can have an interval in seconds (the `0.5` after the `i` option) as low as `0.2` without being root (and i see no reason to run this with root privileges, 5 times a second should be more than most people need in most situations and almost border towards abuse, at least if pinging even more often then that)

when the internet connectivity is unavailable you might want to try doing a `traceroute -n 8.8.8.8` (on linux) or something similar to see how many hops along the path you can manage to get before you can't get any further

[ashen lintel]

Thank you, this is way more than enough for what I need, as my connection hasn't even gone down long enough for me to notive with my wifi on auto-connect, it's just someting I'm curious about because fiber shouldn't drop regularly at all, but setting up this monitoring is going to be most educational.

[fleet haven]

I had a problem like that. It was the converter box that converts the fiber optic signal to my regular network at home. Not that it has to be the same problem for you. Tested different cables and routers and talked with my ISP to remove other posible problems.

[cloud spruce]

i agree with this, @ashen lintel change the cables that you can change, connect the cable to another device than the router you're currently using
when you can't change anything more on your own, contact your ISP for help

[ashen lintel]

It's not any cables I can change, I can tell than much, because I can sometimes see by the lights on the ONT that it's reconnecting. I would just like to, say, show my provider a table or graph of lost connections. I'd rather they didn't just come round, interrupt my connection and then see no problems in the time they're here and leave.

[cloud spruce]

have you gotten any data from the ping command above yet?

[ashen lintel]

I just tried it now in wsl, on 8.8.8.8 and no reply, but a plain ping to that address works.

[cloud spruce]

do you mean "no output" rather then "no reply"?

because with the q option the command will only give output when you are not recceing a reply, so that you can get a log with timestamps for when the connection is not working properly, to try it out you can run it against 4.4.4.4 instead, it shouldn't give you any reply and thus give you output

[ashen lintel]

My bad. I only had 8.8.8, only 3 8s, the first time. It's running on q now with no output yet

[cloud spruce]

you shouldn't see any output until the target that you are pinging isn't reachable anymore
when that happens you will have line of output every 0.5 seconds
if your terminal history scrollback buffer isn't long enough and the outage is either too long or too frequent you will loss the output data that can't fit in the buffer

if you want to make sure it ends up in a file so that you don't loss any data, you can run it like so

ping -qnDOi 0.5 -W 0.2 IP_ADDRESS_OR_HOSTNAME_TO_REFERENCE_TARGET_HERE | perl -MPOSIX -wlpe 's/\[\s*\K(\d+)(\.\d+|)(?=\s*\])/strftime("%F %T$2 %Z",localtime$1)/eo' | tee -a unreachable.log
```or similar, then it will log the output to the file with the above name as well as the terminal

[rugged atlas]

guys i am building email verification/validation tool. i heard microsoft and gmail lie during smtp handshake. every smtp handshake returns 250 even if email address is dead. is it true?

[cedar forum]

yep

[rugged atlas]

are there other services that you know also lie during handshake?

[cedar forum]

their smtp servers are built to explicitly prevent anyone being able to tell if an email address is valid or not (unless you send an email)

most commercial email servers do

this has long been something that you can't rely on in SMTP

as an example, see how many servers support VRFY

[rugged atlas]

does this mean every VRFY server lie during handshake?? i was thinking of running browsers to check valid/invalid email from google/microsoft sign in page. but if there are more servers who do this than its useless to build this product. it wont be accurately tell which email is valid or not

[cedar forum]

there's so many reasons not to do this

[rugged atlas]

i know the main one is TOS

[cedar forum]

so, TOS is the one that means we can't help with this here

!tos

[errant bay]

Did you mean...

» tools
» microsoft-build-tools
» off-topic-names

[cedar forum]

Urgh

!rule tos

[errant bay]

Rules

5. Do not provide or request help on projects that may violate terms of service, or that may be deemed inappropriate, malicious, or illegal.

[rugged atlas]

i also thought to send real emails through amazon SES and see which one bounce. but they have limit of 5% bounce rate if it exceed than they will terminate account so that idea is also not good. tools like neverbounce have probably database of millions email so they just check from previous data to mark valid / invalid email.

[cedar forum]

but on top of that, you (or your users) are going to get detected by Google or Microsoft and your IP reputation will get absolutely trashed, this is what causes you to have to solve a captcha anytime you do so much as search something

[cedar forum]

for the same reason, yeah

[cloud spruce]

please don't be a part of the problem with spam, we have too much of that already

[steady horizon]

Reread this server's rules, notably rule 6. Read this channel's topic. Social/job networking isn't this channel's purpose

[silver dock]

okay

[cloud spruce]

remove this post at once, as this post breaks the server rules

[cloud spruce]

thank you

[cunning goblet]

Hi How do i get activated in the channel for voice verification?

[cloud spruce]

this channel is about computer networking and protocols and such in a python context, not "social networking"
but the information you are looking for is in #voice-verification

[tawdry matrix]

Hey, does anyone know how to test a demo like, demo verification code sent to the client email other than directly via domains with vps?

[errant temple]

Use your app locally and use a fake email server to test it manually. That's your simplest bet, you can start a fake smtp server in docker that will record all emails sent through it and show it to you in a web interface. Then, once you get that working, you can look to write automated tests that run locally to see if emails will be sent as expected if you feel up to it.

[tawdry matrix]

Do I have to use Docker? Is there no other way? Because im still a novice

I mean, the easy one for a novicr like me

[cunning garden]

it's worth learning as it will simplify a lot of things for you

[cunning garden]

You have a customer, I would not call you a novice anymore

[tawdry matrix]

Alr thx sir

[tawdry matrix]

Actually i js want to learn hehe, thanks

[errant temple]

This could be a good opportunity to learn. All you have to do is have docker started and use it to start the service and expose a port.

It's a simple as choosing what software you want to start and how you want to expose it so you and your software can access it.They can run outside of docker I'm sure but it's a lot easier to use stuff that is dockerized, and a lot easier to help with too.

https://github.com/maildev/maildev

docker run -p 1080:1080 -p 1025:1025 maildev/maildev

[tawdry matrix]

Ahh thx sir, is this free? What if i suddenly have to pay for the license?

Owh nothing thx sir, you helped me

[errant temple]

Then you choose a different option for testing. This isn't the only one. This one is free and I don't foresee it becoming paid.

[prisma flax]

<@&831776746206265384>

[round glen]

!cleanban 1005020360187576352 Only here to advertise

[errant bay]

:incoming_envelope: :ok_hand: applied ban to @maiden hearth permanently.

[balmy garnet]

i have a container running a wg client

i have a wg server with the following rules
rule family="ipv4" forward-port port="25565" protocol="udp" to-port="25565" to-addr="10.0.0.2"
rule family="ipv4" forward-port port="25565" protocol="tcp" to-port="25565" to-addr="10.0.0.2"

i am running a ncat server in a other container with --net=container:wg
-p 25565:25565/tcp
-p 25565:25565/udp
my netcat server can receive tcp packets from the public ip of the wg server but,
udp connections only work from local host

how could i get upd connections to work from the public ip of the wg server?
i need this to run an mc server

i also tried sending udp packets to 10.0.0.2 and got that one

when trying to run
nc -vul -s <ip of vpn> -p 25565
i get nc: bind: Address not available

idk what i changed but listening on 0.0.0.0 recieves udp packets from the vpns ip

if i start the container and start the nc -vul -s <vpn ip> -p 25565 server first it doesnt work but if i test 1 packet with the localhost and 10.0.0.2 versions than it works
but even if the one with the vpns ip works with a client and a server in the container it doesnt work if i move the client to an other pc
or if i run it on the host without being connected to the vpn nor does connecting to the vpn on the host and than trying to access the server work

running the client in the module it shares network with also doesnt work for any of the ips i tried

the being on the same vpn part was prob affected by trying to connect with the same auth twice to the vpn but i closed the hosts connection and reconnected with the container to wg after that test

any1 any idea how to get udp working?

[elfin bramble]

ngl i dont

even know wut tf u talking about

[elder cloud]

so then dont reply

[elfin bramble]

y not?

the person could explain if he knows i dont understand

[elfin bramble]

u had no point in replying aswell so tf u on about?

[elder cloud]

no need to get defensive

[sleek breach]

troll

[round skiff]

Hey! How as a beginner in networking can I understand the term “switches” , “switching”? Talking about a modern router, it’s the part of it where bits are going in/out to the access network? Or if we use Ethernet cable the bits sent through it also go through switching mechanism? Correct me please if I’m wrong , I am bit of confused with the terms

[cloud spruce]

switching is usually (there are some corner cases) what happens in a switch for layer 2 traffic, traffic that should just be directed to the right interface to reach its next hop within the same network segment as the packet is currently in on its path towards the destination
while routing is usually what happens (again, there are other cases as well depending on how the network is built) when you traverse the boundary between to different network segments, this typically happens on layer 3)
nowadays many switches are layer 3 switches that can both do switching and routing for you in the same device at the same time

[round skiff]

sorry, when counting layers we are counting top-down? like application layer is the 5th layer?
and as i understood routing covers Network layer while switching not always? switching is more about link layer?

[round skiff]

i understand more the concept of routing. it's happening when a packet is already inside the router, during the processing delay, the router looks at the packet's destination in a payload and choosese the most sustainable link to route the packet to (may be based on the length of the route or if the link is busy)
however the whole actual process in deep is more complex in action

[cloud spruce]

yeah, layer 1 is the physical layer (light, electricity, radio waves and so on), layer 2 is typically ethernet, layer 3 is typically IPv4 or IPv6, layer 4 can be for example TCP and UDP and so on

[steady horizon]

An Ethernet switch is a network device with ports connecting to multiple other network devices which acts as a crossroad between them. It forwards frames from a sender device to a target device by looking at the MAC addresses in the Ethernet frame.

Routers in a LAN typically aren't connected to their subnet entities with physical Ethernet ports. Their part in a network involves forwarding IP packets and making educated decisions, figuring out which way IP packets should take, hence routing them.

[plucky lintel]

Heellllo

[limpid dagger]

this chat seems hella inactive but im desperate 🙏 (chatgpt gave up being helpful) (but im not a vibecoder im just tired) (7 hours on this project today)

P2P messaging program between Alice and Bob, which uses a node to establish the connection (by sharing IP + port)

after that, Alice and Bob do some fun UDP hole punching

but the data never gets recieved on either ends; it sends fine but doesnt recieve

send loop:

while True:
    time.sleep(0.005)
    UDP_socket.sendto("HI BOB!!!!".encode(), alice_peer_int)```

listen thread:

```py
def listen_UDP(UDP_socket):
    UDP_socket.bind(my_addr)
    print("Started listening on UDP:", my_addr)
    while True:
        try:
            data, addr = UDP_socket.recvfrom(1024)
            print(data, addr)
            if data != None:
                UDP_socket.sendto("Pong!".encode(), alice_peer_int)
        except Exception as e:
            print(e)```

i AM expecting/hoping that the port which connects the public ip to the node can be reused with UDP by the other peer

A -> N -> B
to
A -> B (N)

but yeah its specific and so painful (i hate how IPV6 isnt common enough to solve this) so im just asking here incase anyone happens to be decent with P2P / hole punching

this is what the shell looks like: (on Bob)

>>> Pinging: ('XX.XX.XX.XX', 8080)
Alive nodes are: [('XX.XX.XX.XX', 8080)]
Listening...<socket.socket fd=660, family=2, type=2, proto=0>

Starting UDP threadStarted listening on UDP:
 ('192.168.0.30', 7001)
Connected to node
Can I connect to $XX.XX.XX.XX$7000$alice_id$? My ID is $bob_id
My client is: <socket.socket fd=572, family=2, type=1, proto=0, laddr=('192.168.0.30', 5718), raddr=('XX.XX.XX.XX', 8080)>
Alice, here is Bob's details: $XX.XX.XX.XX$5778$alice_id
('XX.XX.XX.XX', '5778')
Sending message to: ('XX.XX.XX.XX', 5778)
Spamming punches...
LOCAL SOCKET: ('192.168.0.30', 7001)
Alice addr: ('XX.XX.XX.XX', '5778')
[WinError 10054] An existing connection was forcibly closed by the remote host
[WinError 10054] An existing connection was forcibly closed by the remote host
[WinError 10054] An existing connection was forcibly closed by the remote host
[WinError 10054] An existing connection was forcibly closed by the remote host
[WinError 10054] An existing connection was forcibly closed by the remote host
[WinError 10054] An existing connection was forcibly closed by the remote host```

in this example, bobs ip got created as 5718 (longest line laddr, and the node shares this with alice, and vice versa

alice's was 5778

the node can chose to talk to either to alice or bob with my public ip (theyre all on same public ip) and one of those ports

[limpid dagger]

i think i found a solution - brute forcing ports (guessing ports)

i beleive im on a symetric NAT, therefore the port changes by a few (+)

[limpid dagger]

it doesnt work :(

[cloud spruce]

sorry for the late reply

i think that you're having problems because your firewall/gateway that does the SNAT will not allow u-turn/hairpin NAT and block the traffic

if the firewall/gatewat does in fact allows this traffic they can sometimes skip NAT:ing the traffic when it never leaves the internet facing interface that has the public ip address attached to it and route the traffic as it was (probably not what you are seeing in your case as you say the receiving side never sees the traffic), in those cases you would receive the UDP traffic if you allow traffic from any source ip address, but you would see the private ip as the source instead of the public ip

to test this properly i would recommend to use two different computers that are situated on two different LANs going out of different firewalls/gateways using different public ip address

you might be able to use your mobile internet connection through your phone and use the internet sharing feature on the phone to make it into a mobile wifi hotspot that you can connect a laptop or whatever to over wifi, just make sure you have disconnected the phone from your home wifi before that, or else the phone will typically use the wifi connection instead of its cellular network data connection for the traffic that it routes

and of course you would need another computer (laptop, raspberry pi or something) connected to you home network

you should probably implement self public ip discovery and if you are trying to connect to the same destination ip you should probably see if you can discover each others private ip addresses and connect directly to that in those cases or use a relay/proxy/ip-forwarding service to be able to connect to the other party if everything else fails

i'm guessing your using the ICE protocol right now or maybe the STUN protocol directly, you could look into being able to fall back to using the TURN protocol to do the traffic relaying that is just described above

i would also recommend running wireshark on both computers at the same time to help troubleshoot the issue, regardless of you are running your current setup of doing the above

[limpid dagger]

my knowledge is being messed up right now. testing with my gfs laptop (so different LAN) and whilst looking at debug, i see laddr=('192.168.0.30', 10433). 10433 is a local port, but thats what her laptop has been trying to holepunch at. but the 10433 is shown on my laptop node as a raddr?

you are giving me way to much credit for using protocaols like ICE (idek what that is) or STUN.

this is the relevant code:

def listen_UDP(UDP_socket):
    UDP_socket.bind(my_addr)
    print("Started listening on UDP:", my_addr)
    while True:
        try:
            data, addr = UDP_socket.recvfrom(1024)
            print(data, addr)
        except Exception as e:
            print(e)
            print("Error in listen_UDP thread")

UDP_socket = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
print(UDP_socket)
listener_thread_UDP = threading.Thread(target=listen_UDP, args=(UDP_socket,))
listener_thread_UDP.start()
print("Starting UDP thread")

node_client = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
node_client.connect(alive_nodes[0])
print("Connected to node")

peer = alive_peers[0]
peer_ip = peer[0][0]
peer_port = peer[0][1]
peer_id = peer[1]
peer_data = f"${peer_ip}${peer_port}${peer_id}$"
print(f"Can I connect to {peer_data}? My ID is ${my_id}")
print("My client is:", node_client)
node_client.send(f"Hello, node$Can I connect to {peer_data}? My ID is ${my_id}".encode())
data = node_client.recv(1024).decode()
print(data)
data_split = data.split("$")
data_trimmed = data_split[-3:]
alice_ip = data_trimmed[0]
alice_port = data_trimmed[1]
alice_id = data_trimmed[2]
alice_peer = (alice_ip, alice_port)
alice_peer_int = (alice_ip, int(alice_port))
print(alice_peer)
print("Sending message to:", alice_peer_int)
print("Spamming punches...")
print("LOCAL SOCKET:", UDP_socket)
print("Bob addr:", (alice_ip, alice_port))
for i in range(5): # loop back from start
    time.sleep(1)
    alice_peer_int = (alice_ip, int(alice_port) - 100)
    for i in range(1000):
        if alice_peer_int[1] % 100 == 0:
            print("Trying", alice_peer_int)
            time.sleep(0.1)
        for i in range(5):
            UDP_socket.sendto("HI BOB!!!!".encode(), alice_peer_int)
        alice_peer_int = (alice_peer_int[0], alice_peer_int[1] + 1) # check the next port

only protocols i am deliberately using is TCP and UDP (and IP blah blah whatever)

Anyways, results of testing across 2 LANS: Same as on 1 LAN. UDP packets never recieved

Interestingly tho, gfs laptop had a thing from Windows defender about "do u wanna allow this connection"

I don't believe there is a logic error in the immediate code (there could be tho), and I don't believe it's a u turn around something NAT issue

Why do I torture myself like this with gcses literally this week 🙏

[jaunty thistle]

i have a potential position that'll need knowledge in network/serial communication on windows (and using sdks) for data acquisition hardware, any pointers for where to start? my network experience is more or less 0, and not sure if i'll need high or low level knowledge on the subject (probably high?)

[distant cobalt]

Start with TCP/IP basics and sockets first, then serial comms (UART/COM ports). You prob don’t need super low level stuff at first unless ur writing drivers/firmware. Wireshark and a small TCP client/server project would help a lot

[tall dome]

I'm writing a server application which accepts TCP socket connections from many clients simultaneously using the socketserver library. socketserver has both ForkingTCPServer and ThreadingTCPServer and I'm not sure of the pros and cons to threading vs forking in this instance. Can anyone offer advice?

[ember ledge]

guys who use scapy?

[mystic haven]

This is not a question that helps you, assume there are people here who know how to do networks in python

[cloud spruce]

ForkingTCPSwever (is multiprocessing) is good if you need to do CPU intensive work but i don't think it's compatible with windows without wsl, cygwin, msys2 or similar that gives you a unix/posix environment, otherwise threading is preferred as there is less overhead (both in terms of memory and cpu) and is great for when you need network i/o concurrency

[cloud spruce]

i do at times, do you have a real question?

[lone scarab]

Hi everyone
I begin with socket and I try to just send a text msg from a computer to my raspberry pi 3 (server), and my computer(client) always get an connection error 111, if check my firewall on my pc, I try changing port, checking if they were open, and nothing, I think I am lost...
Anyone can help get pass this error pls 🙂

[prisma cobalt]

are you sure your devices are on the same network

make sure raspberry pi doesn't have a firewall configured blocking the connection (check ufw or iptables etc)

[short sequoia]

is a metered network basically one that has limited data?

[cloud spruce]

can you show the code you are running on the server and the client respectively for this?

!paste

[errant bay]

Pasting large amounts of code

So that everyone can easily read your code, you can paste it in this website:
https://paste.pythondiscord.com/

After pasting your code, save it by clicking the Paste! button in the bottom left, or by pressing CTRL + S. After doing that, you will be navigated to the new paste's page. Copy the URL and post it here so others can see it.

[prisma cobalt]

not necessarily. metered means the amount of data sent/received is measured and usually implies it's either limited or you pay for the amount of usage or both

[short sequoia]

yeah which means that it's not unlimited and can get cut off at any time from exhausting the plan that the network is on

[prisma cobalt]

as I said, a metered network doesn't always mean it's limited, only that it implies it

[lone scarab]

Yes they are bc I ssh they 😁

[lone scarab]

Thanks but after watching tutorials I discover I new way to proceed that I need to check 😁
I will came back if it doesn't work either....lol

[cloud spruce]

just remember that there are quite a few things to be careful with when it comes to network programming for it to work reliably

[misty orbit]

Hey guys, I'm setting up a physical server for my friends. Is this a good place to ask for security measures? Or not quite...

[prisma cobalt]

yes, if it's related to networking security

[brazen ferry]

Might interest people on this channel https://pypi.org/project/whereismyip/

[cloud spruce]

you'll have a lot of work in front of you if you want to security properly, network segmentation would be a good idea if you are able to

[misty orbit]

Well so basically I got a PC that will work as a server for the business of a friend. Himself and his coworkers will access this PC via Windows RDP. So yeah, I can imagine a few different methods of securing the connection and everything, but I'm 100% sure there is more to do, and I'm certainly not a professional on this. So I'd like to know where can I read on proper security for these types of things.

I really want to do a great job.

And by the way, they have their own domain and want to have [e-mail], I know you can pay for example to protonmail or google to have this mail service with your domain, but if I'm not wrong, you can also "host" this yourself and avoid paying monthly to a mail provider. What's your opinion on this? Worth it? Or no.

And thanks 🙏

[cloud spruce]

you will have to make sure that your ISP that you have the server connected through doesn't block inbound and/or outbound TCP port 25, check both to make sure it really work, a lot of ISPs block it to make sure their customers doesn't send out spam

[cloud spruce]

i would for sure run linux on a server, but each to their own i guess

[misty orbit]

Understood.

[misty orbit]

They are used to working like that. To be frank, they just save important data in the server... do some excel and they also want the business email in their windows user (user in the server).

But yeah I want all of this to be 100% secure, I clearly don't want a random being able to connect to the server for example.

[cloud spruce]

you have a well formed plan for backups? (off-line backups as well)

[misty orbit]

Yeah I want to setup serious security measures, I just don't know what.

[misty orbit]

Air Gapped

RAID1 also

[cloud spruce]

raid is not even close to backups, it only protects against hardware failure of disks, nothing more, don't count it as a part of your backups strategy

[misty orbit]

So then air gapped backup

[cloud spruce]

check out the 3-2-1 backup strategy if you are serious about protecting you/their data

and keep several revisions so that you can go back in time in your backups

[misty orbit]

They don't want to use the cloud for this

I even told them to get a VPS but they dont want their info out there lmao

idk

thats why I thought of air gapped backup

[cloud spruce]

that's fine, make sure there is at least one backup copy that is both off-line and off-site at another location (protects against both malicious erasure as well as fires)

[misty orbit]

So regarding network security?

what should I do when I've the PC in my hands? How should I set it up?

[cloud spruce]

when it comes to network security i would build that in the network primarily, not on the server other than make sure as few server processes that listen on the network is running and having the host firewall configured to block as much as possible

for access to the server i would have a network firewall (separate device) and run a VPN to not only rely on the security of the access control of RDP or what ever you are running there

if it's a really sensitive environment i would also block most outgoing network traffic with a separate firewall appliance or router with such capabilities and have a proxy inspect and filter outgoing traffic

[misty orbit]

Hi, a bit tired to process this, will come back to this tomorrow 😎

Thanks!!!

[neon palm]

is python just really slow at recieving tcp/udp packets? im sending image data from a esp32 over a socket with my python code recieving these packets, but its struggling to recieve packets even when i make my esp32 code send packets ever 200ms

is this a python issue or a skill issue

[wintry depot]

the latter

[lyric oak]

Did you build simple test apps in different languages to compare? For example, some simple test with Python/Go can reveal enough. Python is often slower but won’t be too bad either.

[clever pewter]

Networking

Networking is a very important concept in Computer Science, and Python is an excellent language to write networking scripts with.

There are numerous high level APIs for networking in Python such as urllib for HTTP, ftplib for FTP and smtplib which provide high level access to networking APIs.

Additionally, Python provides access to the lower-level areas of networking through modules such as socket which allows you to get into the nitty gritty of network programming, modulating network packets directly.

There are also great third party modules for networking in Python, for example, scapy is a library allowing you to craft packets using a high level Pythonic API.

This channel is intended for discussion on networking, not strictly Python, though when talking about networking from a programmatic view Python is preferred.

Resources on networking with Python:
• TutorialsPoint
• RealPython
• FutureLearn Course

[ember ledge]

very cool

ok joe

[twin edge]

Would requests count for networking? Just curious

[cedar forum]

Ideally we'd be looking for lower level than requests

[modest pebble]

Well, urlib is listed

Requests is just a better interface to urlib

[cedar forum]

@modest pebble yeah, I said ideally.

We'd rather have this as a more discussion based channel and would rather not have questions on how to make requests to APIs and such in this channel, more about the specifics of networking in Python

[modest pebble]

True

And about networking in general, like how http requests work on under-the-hood, would that be accepted?

[gentle jungle]

that seems to be right, yes

[topaz root]

hey new channel

okay

so

[cedar forum]

yeah, that's the kind of stuff we are going for Akarys

[topaz root]

this channel is related to the network (internet) in general?

[cedar forum]

networking, see the pinned message

interaction with packets so on so forth

[topaz root]

cool

[solemn sinew]

is this new

[shy pebble]

yes

[solemn sinew]

i needed this channel a long time ago.

[jade aspen]

nice

[rugged ridge]

What can Python do with network?

[shy pebble]

everything any other language can

[quick hatch]

Ohh nice

Transfering data safely is interesting uwu

[native escarp]

Nice being a network engineer this is awesome

Learning alot on apis

[ember ledge]

Yay

[strong thorn]

@floral thistle Please don't repeatedly spam messages that don't add anything to relevant constructive discussion in this channel.

[spark geode]

hey guys is there a lib where i can manage a network?

[floral thistle]

;-;

[exotic bramble]

What do you mean by "manage a network"?

Cisco has Python-based automation scripting ability now I believe

[lavish rock]

I'd heard rumours of that, too, back when I did networking stuff. Never actually saw it 🙂

[exotic bramble]

It's on the certification exams now

https://developer.cisco.com/codeexchange/github/repo/CiscoDevNet/python_code_samples_network/

[exotic bramble]

https://github.com/CiscoDevNet/python_code_samples_network

[elder cargo]

Mm. One of my favorite topics in all of computers.

[wide sail]

Networking is the coolest 😄

especially with all the new fancy iot shit coming out 😄

[kind osprey]

https://python-socketio.readthedocs.io/en/latest/client.html
Maybe we can build bots for games using this

[ember ledge]

in most games, botting is against the ToS @kind osprey

[wide sail]

not to mention botters are the lowest of lowest of scum

[chilly hemlock]

Not really

[modest pebble]

Bottling is a pain in the arse to forbid too

[ember ledge]

Ye

Also does anyone know any good library that would be great for TCP connections? Something like the Sockets library in the standard library.

[umbral yew]

Hey guys I'm just starting my journey on networking as I want to create a multiplayer fighting game for me and my friends to play. Currently I'm playing around with socket library and I think I'm going to use pygame for the game representation. I was just wondering if sockets is actually still a good library to use with threads or asyncio, or use something like curio by David Beazley. Any networking suggestions at all I would be appreciative of.

[cedar forum]

@ember ledge isn't that just socket?

[ember ledge]

Ye but is there any other modules?

Or is there just a socket module...

[cedar forum]

socket is the one most people know of, what are you trying to achieve that socket does not provide?

@umbral yew for asyncio, asyncio implements a lot of the networking stuff socket does in an async friendly way

[ember ledge]

IDK, I can't find a way of how to secure a connection with Sockets.

[cedar forum]

socket will not be async friendly in a lot of situations

for example, if a socket blocks and doesn't respond in an asyncio task the entire application is blocked up

see this for asyncio networking https://docs.python.org/3/library/asyncio-stream.html

@ember ledge so you can use the ssl module for TLS w/ low level sockets, check out https://docs.python.org/3/library/ssl.html

import socket
import ssl

hostname = 'www.python.org'
context = ssl.create_default_context()

with socket.create_connection((hostname, 443)) as sock:
    with context.wrap_socket(sock, server_hostname=hostname) as ssock:
        print(ssock.version())

that creates an SSL context and allows you to use socket as usual

[ember ledge]

Oh okay thank you!

[cedar forum]

there are examples of that for doing the inverse as well and hosting a server with certificates

but if you use ssl and wrap a socket you basically use your socket just as usual and ssl abstracts away all the TLS stuff

so it's basically a drop in replacement

[ember ledge]

I want to create a local LAN chat system with ranks which will involve UI but I already have the UI.

[cedar forum]

ah gotcha, if you are going for LAN then you might want to look into local network broadcast to send to all listening clients on a network

[ember ledge]

What do you mean

[cedar forum]

well, actually I guess it depends on the setup

[ember ledge]

Can't I just grab a message from a client that sent a message and for each client thread send the other clients the message?

[cedar forum]

yeah, I was thinking if you didn't want to have a server at all

but actually a server may be nice here

but yeah, what you suggested will work fine

[ember ledge]

Yes a server because I wanna control user ranks and stuff.

[cedar forum]

the broadcast address of a network is an address where if you send a message to it the router fans it out to all connected clients on the network https://en.wikipedia.org/wiki/Broadcast_address

so in some LAN situations it's quite useful

[ember ledge]

Wait so my socket can even communicate with the router in a context where my socket doesn't want to travel outside the local network but like talking with the router, getting info from it and stuff?

Also ye that's what I'm looking for.

[cedar forum]

broadcast is useful but not for where you have ranks and stuff

so you'll still want a server

[ember ledge]

Ye a server running on a raspberry PI.

Also regarding Sockets, am I able to send files? Like not in bytes object, just files?

So I don't have to create a new file on the receiver users machine manually.

[cedar forum]

you are not able to send files no

[ember ledge]

Aw

[cedar forum]

TCP packets are content blind, they have no idea what is in then, just arbitrary bytes

it's up to you to tell the receiving socket what to do with those bytes since it has no idea if it's an image, text file, executable or whatever

[ember ledge]

That does not bring joy.

[cedar forum]

that's why HTTP exists, because we can attach content type headers and so on

[ember ledge]

Yeah okay, I'll just send a message to the receiver as to what kind of file it is and it's name so we can send the bytes after that message and compile those two messages into a file.

Sounds great.

[cedar forum]

yep

[umbral yew]

Thanks for that joe

[cedar forum]

no problem!

@umbral yew for what it's worth, if you do hit concurrency issues with networking and don't get anything here, you can ask in #async-and-concurrency as well, that is for all the async implementations in Python (asyncio, trio, twisted, etc.)

[forest frigate]

@Null#6465 might wanna use something higher level than just bare TCP, like some RPC library

unless you wanna learn yourself or they dont meet requirements

[exotic bramble]

If you want to make standard packets use socket, that's the universal standard and there's usually no reason not to

The primary alternative is scapy, which allows you to construct your own packets

However,it requires knowledge of how said packets are constructed

It's generally used by network professionals for troubleshooting and testing or security folks for fuzzing, scanning, custom protocols, etc.

[ember ledge]

anyone here taken a cisco class

[ember ledge]

@ember ledge aye

have CCNA cert

what u need my dude

[ember ledge]

Just wanted to know if there were other cisco related people in here for future questions, just finished cisco 1 @ember ledge

👍 very nice

[blissful juniper]

Netmiko is a good place to start for managing devices via python

probably the simplest and best out there

[placid sparrow]

Hey does anyone understand this formula, I can't wrap my head around it
P = ( v1, v2, ..., vn ) ∈ V x V x ... x V

[jade blaze]

Hi guys good day to you all

Is there anyone around here, who knows how to open VNC viewer in python?

[cerulean tusk]

does someone have a quick and dirty script to stream data over sockets?

[lofty vine]

Hello. I'm quite new to networking and stuff and I was wondering, do you know any good tutorials about this? Not only python tho (I mean, especial for python but also all those names, protocols and stuff)

[atomic warren]

there are some good resources in the pins

[lofty vine]

Thanks

[distant pilot]

anyone knows where can i learn sockets completly

[obtuse wharf]

Hey does anyone understand this formula, I can't wrap my head around it
P = ( v1, v2, ..., vn ) ∈ V x V x ... x V
@placid sparrow P is an n-tuple belonging to set (V)^n for simplification consider an example of a coordinate on graph (2D) which is written as (x1, x2) we say that this point belongs to set R^2 or simply R x R

[placid sparrow]

Oh so could you simplify it as P = (vn) ∈ (V)^n

[restive blaze]

off topic

[cold charm]

guys can client socket talk to server socket on different devices but on the same [local] network?

I'm trying to send json object from my mobile (client) to server(pc) but it doesn't seem to work. It dose work when client is created on pc though. Hence a little confused

[ember ledge]

yes, that should all work fine

what interface/ip are you binding to?

[cold charm]

AF_INET

[ember ledge]

you shouldn't bind to 127.0.0.1 or loopback

that's not the interface

can you send us your code?

!paste

[errant bay]

Pasting large amounts of code

If your code is too long to fit in a codeblock in discord, you can paste your code here:
https://paste.pydis.com/

After pasting your code, save it by clicking the floppy disk icon in the top right, or by typing ctrl + S. After doing that, the URL should change. Copy the URL and post it here so others can see it.

[cold charm]

oh I'm binding to my PC 's IP

on port 5050

[ember ledge]

PCs IP on your local network or what

try binding to all interfaces (using ip 0.0.0.0) and see if that changes anything

if not, it might be a firewall issue?

you using windows?

[cold charm]

yes

[ember ledge]

it might be windows firewall being fucky

[cold charm]

let me try the solutions real quick

Connection refused

@ember ledge is it firewall then?

[ember ledge]

i think it might be

[cold charm]

what's weird is client won't even work on the same machine as the server on 0.0.0.0

it used to work on machine when I was on local IP

Tried turning off firewall and running on 0.0.0.0 Still doesn't work

[ember ledge]

can you post your code?

[cold charm]

sure

https://hastebin.com/cimufeqehi.py

server.py ^

https://hastebin.com/sibamejesi.java

client.dart ^

@ember ledge

[ember ledge]

in your client file, you still need to put the IP of the actual server

not 0.0.0.0

that's only for the server, to listen on all interfaces

outgoing (client) connections need proper IPs

[cold charm]

oh gotcha

lemme fix and try it

still doesn't work when the client is from different device on Local network

[ember ledge]

let me try.

give me a min

[cold charm]

oki

[ember ledge]

the python side of it looks ok to me

i'm not proficient enough in dart/java to comment on the client code

but if it connects on localhost, i don't really see why it wouldn't over local network

dunno. tried other devices?

[cold charm]

when I try it on android emu on my machine where server is, It works but not when I try to do it from my mobi

lemme try on some more devices

[ember ledge]

your phone doesn't happen to be connected to a vpn or something, right?

[cold charm]

nope. It's just connected to wifi

nope. Doesn't work form other devices as well

[restive blaze]

Why socket?

It's possible Windows Firewall is blocking Python

otherwise, is connection even being made or connection made but not accepted?

[cold charm]

firewall is disabled @restive blaze

connection is not made either

I just tried to ping my mobi from my pc and that seems to work

[restive blaze]

ping means network is operational

BTW, it's possible that mobile device may not allow socket connections

[cold charm]

then why did android emu allowed it ?

[restive blaze]

not sure

it may or may not be the problem

[cold charm]

I've seen some threads related to this but non matched my issue

it's almost right but still wrong somehow

it won't throw any errors either

[restive blaze]

I'm just wondering if Android has sockets disabled or it's an option for phone makers to do so

almost no android app would require sockets

games would probably be it and VOIP apps would be it

[cold charm]

I've played games like air console on same devices which rely on sockets for communication if I'm not mistaking

that seemed to work just fine

[restive blaze]

Yep

If you are not connecting, either A) Server is blocking or B) Client isn't connecting to right address

or C) Client is blocking

[cold charm]

can u suggest something to debug this?

[restive blaze]

for A) get another client and try

for B) Double check the code

C) Look for documentation on said client or see if you can debug on mobile directly (I don't do mobile development so I'm clueless here)

[cold charm]

okay. I'll try these

[ember ledge]

dumb q, but you've got <uses-permission android:name="android.permission.INTERNET" /> in your AndroidManifest.xml, right?

@cold charm

[cold charm]

yup

[cold charm]

Here's the code again, just in case I'm still making some dumb mistake

https://hastebin.com/vaxoyazepi.java

client ^

https://hastebin.com/dewovejana.py

server ^

[restive blaze]

your PC has IP of 56.1?

[cold charm]

I'm using socket.gethostbyname(socket.gethostname()) to print the IP everytime the server runs to ensure I'm connecting to right IP so I'm sure has 56.1

@restive blaze

Something fishy with that?

can this be an issue?

[restive blaze]

Possibly

[slim aurora]

How to get around Network address translation?

[hexed epoch]

why do you want to do that?

if you control the nat gateway (or whatever is doing the translation) you can just disable it

[slim aurora]

Let's say I want to create decentralized messaging app, when clients are behind a router. Simply: Send data, receive data

[ember ledge]

@cold charm yes, the first option turns on a strict firewall

you should leave it as the second option

actually was debugging this the other day in a lab environment, couldn't connect to RDP on some machines and had no idea why - turns out, some machines had that option enabled while others didn't

[exotic solar]

I'm doing a project that I know will need networking, and, I was wondering (basic question), what are the different types? I know that sockets are one (but I don't actually know what that is), what are some others? and, what are the differences?

[hexed epoch]

what are the different types of networks?

[exotic solar]

@hexed epoch yep 👍

[hexed epoch]

do you know what layer you're talking about?

(referring to the OSI model https://en.wikipedia.org/wiki/OSI_model )

when computers talk to each other, they always use sockets

[exotic bramble]

sockets are simply a means for your program to communicate on a network

[exotic solar]

oh, I was unaware that there were layers

[hexed epoch]

take a read of that wiki page, it's pretty good

[exotic bramble]

you create a socket specialized to the type of protocols you want to use

[hexed epoch]

and based on what you said and what most people talk about, I'd imagine you might be referring to the transport layer https://en.wikipedia.org/wiki/Transport_layer

[exotic bramble]

usually you use a combination of protocols when communicating with another device

hence "layers"

[hexed epoch]

or the application layer https://en.wikipedia.org/wiki/Application_layer

[exotic solar]

Ah, thank you! I will take a look at those :D

[restive blaze]

SuperMazingCoder, Sockets is basically building your own Layer 7 protocol. Besides that, there is HTTP/WebSocket (subset of HTTP) which are obviously most popular, AMQP, SMTP, SSH to name a few

[cold charm]

@restive blaze @ember ledge finally it's working. The main reason was socket.gethostbyname(socket.gethostname()) for some reason was printing IP of my VM instead of my actual machine. I realised it after I checked my IP on ipconfig.

[ember ledge]

ahhh

I had no idea you were doing it in a VM

[cold charm]

I was not doing it in a VM, I just had it installed for running Linux. Although I don't understand how it messed up the IP when it wasn't even running

[slim aurora]

I want to create decentralized messaging app. How to receive data when clients are behind a router?

[forest osprey]

does anyone has any explanation? the date is almost true, It is 5/6 13:45 but the year is definitely not 2090...

[ember ledge]

can you paste your code?

[forest osprey]

def get_time():
     ntpip = 'time.google.com'
     packet = IP(dst=ntpip)/UDP(dport=123,sport=50000)/NTP()
     respond = sr1(packet, verbose=0)
     unix_timestamp = respond[NTP].recv
     return time.strftime(r"%Y-%m-%d %H:%M:%S", time.localtime(int(unix_timestamp)))

>>> get_time()
'2090-06-05 13:48:52'
>>>

as you see I work from the console

[ember ledge]

what's the actual timestamp you're getting?

[forest osprey]

got 3800343422.180988
for the date '2090-06-05 13:57:02'

the current timestamp is actually ~1591354653

so why am I getting this time? and why is everything is right except the year?1

[ember ledge]

unix epoch != ntp base time

unix epoch is 1st january 1970, ntp base time is 1st january 1900

[slim aurora]

Why no one can help me?

[ember ledge]

@slim aurora it's only been a few hours since you posted your question

chances are, not many have even seen it yet

have patience...

[forest osprey]

so how didn't they run out of numbers?

doesn't it work with 32 bit int + 32 bits floating point?

[ember ledge]

it does

it hasn't rolled over yet

it will happen in iirc 2036

[forest osprey]

but unix is going to run out of numbers in 2038

and it started 70 years later

[ember ledge]

unix timestamp is stored as a signed int

ntp is not

[forest osprey]

ohh

right

thanks

[exotic solar]

Has anyone watched either of these series:

1. https://www.youtube.com/watch?v=cNwEVYkx2Kk&list=PLDQaRcbiSnqF5U8ffMgZzS7fq1rHUI3Q8
2. https://www.youtube.com/watch?v=rIZ61PyDkH8&list=PLR0bgGon_WTKY2irHaG_lNRZTrA7gAaCj&index=1

if anyone has, which one would they recommend?

[exotic solar]

I ended up watching the second series. I just have one question so far; it seems like your computer has its own ip address? and your isp gives a different one?

[ember ledge]

each machine on a local network has an IP. For example, the gateway is 192.168.1.1, your phone is 192.168.1.101, PC is 192.168.1.102 etc

because there aren't that many IPv4 addresses (we ran out last year), your ISP can't afford to have your outward-facing IP to be different for each machine - so your router does something which is called masquerading and all connections that go through your router has your router's external ip (in the most common scenario - there may also be scenarios where multiple households share IPs)

you should read up on NAT

[blissful juniper]

While you are on NAT
also read out
public and private ip address range to make this more clear

[ember ledge]

hello

anyone ever heard of EconetAPS?