networks

[ebon silo]

okay, it seems like it was a mac address conflict (what's even the chance of that?)

653eeec58a96  ~  ip neigh show     
10.80.16.3 dev eth0 FAILED 
10.80.16.4 dev eth0 lladdr 02:42:0a:50:10:02 STALE 
10.80.16.1 dev eth0 lladdr 02:42:37:01:f2:a0 STALE 
10.80.16.2 dev eth0 lladdr 02:42:0a:50:10:02 STALE

[tribal dagger]

so i have this issue that before when i used to use wifi adapter for internet i was able to connect python flask application from other pc on the same network using my ipv4 address but now after moving to ethernet i cant connect to it from other pc

[next jasper]

Windows? If yes, it sees wifi vs Ethernet as different "networks" and can have different config (e.g. have different type of connection set - private vs public network, and public may be set to block all incoming connections).

Of course also make sure you're using correct ip - e.g. if you set it as static before, wifi network card and Ethernet card have different MAC addresses so router will see them as different devices and won't give you that same ip.

• If the other computer is still wifi, make sure you don't have some kind of network separation between wired and wifi ("guest WiFi" configs usually have separation from main network).

[royal cradle]

Sup, recently worked on a Shift manager app but not in Python.
I wanted to do something more complex that could be useful for offices. like AnyDesk, Vision Client (For Schools)

However, I have bare understanding of how to write proper applications especially when it comes to the networking part.
Bandwidth is ~100 Mbps so I can monitor and interact with multiple clients at the same time. This is where I'm stuck at.
How does one create a proper Network Manager for the Server-side (Administrator) so it can accept such heavy tasks simultaneously, like screen-sharing from 2 clients at the same time.
but also background tasks like info update going on in the background.

I have never worked with anything network related that is this complex before

PS. I wanted to use QTcpSocket for this.

[boreal oasis]

Question for AI, Cybersecurity, and Remote Sensing Experts:
Is it possible to detect cybercrime-related buildings or infrastructure (like botnet hubs, phishing farms, illegal data centers, or crypto-mining stations) using satellite imagery or geospatial analysis?

Are there any known datasets, real-world use cases, or projects where AI or satellite imagery has been used to identify such physical infrastructure linked to cybercrime?

[lyric oak]

That belongs to #cybersecurity

[prisma cobalt]

It would be very difficult, any computer could be used for cybercrime. The physical building or infrastructure is usually irrelevant and it's not something distinguishable by satellite imagery. I doubt a dataset exists for this but Google is your friend

[obsidian topaz]

doubtful however that large scam farm that got busted in Dubai might point to some clues but not by sat imagery. More like resource usage

sat imagery can be used as a frontend sort of deal for humans to look at, but not for AI to analyze

a building is going to look like any other building to it

[cedar forum]

yea i think that there are probably some heuristic methods you could use to determine this based off some building factors but not from imagery/public datasets

you really need to know things like power consumption and traffic to start making determinations like that

and identifying infrastructure != identifying malicious infrastructure

like you could easily go "okay so i can see a 11kV transformer here, a cooling unit here, etc." but that gives no determination over the legitimacy of a space vs illegitimacy

depending how "rogue" or undercover type operations you are looking for i guess thermal would probably be the way to go

like if you see a heat signature in a residential area chances are you're looking at something untowards, but still no guarantees it's AI (chances are it'd probably be a weed farm 🤣)

[silk cosmos]

anyone know any good tools that can help me make an API with type hints? im working on a client server architecture where clients perform read/writes to tables in a database through an API on the server and having no intellisense in vscode is a pain

[steady horizon]

Sounds like a #editors-ides / #type-hinting question

[upper iron]

what are csrf tokens?

[cloud spruce]

fastapi with pydandic or msgspec perhaps

[jovial sail]

Hello there people. Anyone here with Django+React experience?

[cedar forum]

what's up?

[silk cosmos]

realising a better description of what i want is an RPC library that still lets me have intellisense on function arguments and return type

does anyone have any recommendations or am i better off creating my own implementation?

i have a module that ive written that contains classes representing different tables in the database with a bunch of utility functions, very nice to use. i want clients to be able to use this module since its very useful, but they cant normally as they do not have authentication details to the database, and i do not want to nearly double my workload by creating different REST endpoints for clients to invoke

i have an idea of how i could implement RPC on the module, but i dont want to do it if theres already a library for this exact scenario

[real cape]

not related to python but is there any guide to networking?

ive taken computer networks class and liked it

i heard the TCP-IP Illustrated book is good?

[cedar forum]

anything that will do codegen will do this for you

e.g. protobuf

https://protobuf.dev/reference/python/python-generated/

expanding that into RPC you're probably wanting gRPC https://grpc.io/docs/languages/python/quickstart/

which again, will generate stub files (though they're not really stub files, they are the actual implementations) that are typed and intellisense etc should pick up on them

[jovial sail]

Everything cool, I am just looking for some experience about those tools, because I am using them, I am trying to become a fullstack developer, mostly oriented to BackEnd

It is being a little hard to get my first job on this industry😅

[cedar forum]

both are nice tools -- i recommend you look more in #web-development for questions on those topics, but if you want react and django to work together look at Django Rest Framework which allows you to make nice REST APIs that React can work nicely with from Django

[cloud spruce]

i think they are generally good even though they can feel a little bit antiquated by now in some aspects
but the lower level protocols hasn't changed that much, even if things have been added to them since then
due to changes like this, IETF has in 2022 released an updated RFC 9293 that obsoletes the trusty old RFC 793 from 1981 (and a few other related RFCs) that was the official RFC standard specification for TCP up until then
the IPv4, UDP, ARP and Ethernet RFCs from the same era hasn't seen the same refreshes to my knowledge but has related RFCs that update or amend them in different ways

[hallow tree]

3

[copper dagger]

I'm taking a networks course which is more high level. Is there a good resource that isnt too complex which will help me better understand all the protocols and networks in general

[narrow ridge]

https://hpbn.co/
If u want to understand TCP/UDP

[grand kayak]

I highly recommend Jim Kurose's book Computer Networking: A Top-Down Approach

[mellow flare]

Is it that good and u can read it with 0 knowledge

[grand kayak]

Yes. It starts from the very basics and fundamentals

And in a way that's not overly complicated

[mellow flare]

Oh thank you teach those noobs here smt good

[prisma robin]

nvm - it seemed to resolve itself

[weak swallow]

he has a good c course on networking

[bleak stratus]

Is there any way to achieve true real-time email updates in Python?
I’ve implemented solutions using the IMAPClient library and tested various approaches (including polling intervals, IDLE mode, and different IMAP servers). However, I consistently experience a minimum delay of 15–20 seconds before new emails are detected regardless of configuration.

[cloud spruce]

have you verified with an email client like Thunderbird or similar that the server even supports something close to real-time delivery or notifications for new emails?
IMAP isn't really made for this and even if both sides has support for the IDLE extension there is no promises made from the server-side on how timely or fast it vill notify the client
check if the server has support for
any api or other form of streaming or notifications over some othere protocol

[gaunt hemlock]

Hello guys i have written a python chat. Intern the program works but when i connect with client(windows)----->server(Linux[extern]) it doesn't work. Client is modified on IP of external Server and Server is listening on 'localhost'. There is no error so Server is working on my debian12 Linux OS. When i try to connect from Windows with Chat Client connection brakes down. I even shutdown the UFW What do i wrong ?

[lyric oak]

If your server is listening on localhost and not interested in any external traffic from other hosts, it is meant to see the client on Windows failed to connect. https://en.wikipedia.org/wiki/Localhost You might want to study some TCP/IP basics and see what are the right options to listen on.

[cloud spruce]

if you want any other computers then the one hosting the server to be able to connect to the server you need to let your server either listen to the ip address of the servers network interface card (NIC) or just use "0.0.0.0" as the address to liten to all interfaces of the server

[gaunt hemlock]

ty bro it works

[swift canyon]

nice

[winged topaz]

https://github.com/orgs/community/discussions/166965

Someone help me...

[cedar forum]

you've got replies on the post already by the looks of it, with good advice

[boreal dirge]

from dataclasses import dataclass

from fastsocket import BaseModel, FastTCP


@dataclass
class Request(BaseModel):
    message: str

    @classmethod
    def from_bytes(cls, data: bytes) -> "Request":
        return cls(message=data.decode())


@dataclass
class Response(BaseModel):
    message: str

    def to_bytes(self) -> bytes:
        return f"{self.message}\n".encode()


app = FastTCP()


@app.handler(request_delimiter=b"\n")
async def handler(request: Request) -> Response:
    return Response(request.message)


app.run()

I'm making a library like fastapi but for TCP and UDP servers

This is the TCP echo example, any suggestions?

Start a tcp asyncio server
accept a connection

parse request, using either size or delimiter, and from_bytes
response = handler(request)
send response, encode using to_bytes```

[untold scroll]

💡 In a use case like log tailing, it must be possible to exit the current command.

async def handle_command(
    message: str, reader, writer, manager: WorkflowManager, client_id: str
) -> Optional[str]:

.....
    match parts[0].lower():

        case "tail":
            if len(parts) < 2:
                return "❌ Usage: tail <workflow_id> [<step>]\n"
            workflow_id = parts[1]
            step = parts[2] if len(parts) > 2 else None
            wf = manager.active.get(workflow_id)
            if not wf:
                return "❌ Only active workflows support tailing\n"

            writer.write("📡 Tailing logs... type 'exit' to stop\n".encode())
            await writer.drain()
            last = 0
            try:
                while True:
                    try:
                        line = await asyncio.wait_for(reader.readline(), timeout=0.1)
                        if line.strip().lower() == b"exit":
                            writer.write("👋 Tail stopped\n".encode())
                            await writer.drain()
                            break
                    except asyncio.TimeoutError:
                        pass

                    logs = wf.logs.get(step, []) if step else sum(wf.logs.values(), [])
                    new = logs[last:]
                    if new:
                        writer.write(("\n".join(new) + "\n").encode())
                        await writer.drain()
                        last += len(new)

                    await asyncio.sleep(0.1)
            except Exception:
                return "❌ Error during tailing\n"
            return None
....

[hot vessel]

hi

[boreal dirge]

you mean log tailing over the wire?

I had a project idea for this usecase,
pipe over network

host

tail -f log.txt | pipex

client

pipex unique-id-24

[untold scroll]

I'm referring to the case where I stream information over a socket (like a tail of logs), and I need to be able to exit the command cleanly.

This is the part that needs to be exposed:

try:
    line = await asyncio.wait_for(reader.readline(), timeout=0.1)
    if line.strip().lower() == b"exit":
        writer.write("👋 Tail stopped\n".encode())
        await writer.drain()
        break
except asyncio.TimeoutError:
    pass

In my use case:

 nc localhost 9000
tail 24df785d-5786-4754-a012-a47f54b58ffd
📡 Tailing logs... type 'exit' to stop
[2025-07-27T12:19:34.448479+00:00] Workflow 24df785d-5786-4754-a012-a47f54b58ffd started
[2025-07-27T12:19:34.451540+00:00] Step 'step1' started
[2025-07-27T12:19:34.701729+00:00] hello
[2025-07-27T12:19:34.701930+00:00] [wait] Sleeping 4.0s
[2025-07-27T12:19:38.703553+00:00] hello
[2025-07-27T12:19:38.705686+00:00] [wait] Sleeping 4.0s
[2025-07-27T12:19:42.709195+00:00] hello
[2025-07-27T12:19:42.711410+00:00] [wait] Sleeping 4.0s
[2025-07-27T12:19:46.714392+00:00] hello
[2025-07-27T12:19:46.714714+00:00] [wait] Sleeping 4.0s
[2025-07-27T12:19:50.716746+00:00] hello
[2025-07-27T12:19:50.717044+00:00] [wait] Sleeping 4.0s
[2025-07-27T12:19:54.718694+00:00] hello
[2025-07-27T12:19:54.724266+00:00] [wait] Sleeping 4.0s
[2025-07-27T12:19:58.727619+00:00] hello
[2025-07-27T12:19:58.727977+00:00] [wait] Sleeping 4.0s
exit
👋 Tail stopped

[median fractal]

Hi

[bitter rose]

Hii

[fierce plume]

hi everyone

[untold scroll]

Would it be relevant to override StreamWriter to add a json(dict) method that performs validation of the model passed via annotation?
What do you think?

[winged topaz]

Thanks

[coarse summit]

which library is that?

[sleek rune]

I am French and on Visual Code Studio I cannot install Selenium in my terminal. Could someone translate everything into French in a private message please?

[wispy jackal]

Hey all,
I just released new versions of CIDR-Man (a faster, more convenient, and somehow more pythonic alternative to ipaddress) and CIDR-Bottle (a Patricia Trie implementation for processing routing tables, ROAs, etc).
CIDR-Man: https://pypi.org/project/CIDR-Man/
CIDR-Bottle: https://pypi.org/project/CIDR-Bottle/
Would really appreciate your feedback!

[cloud spruce]

only looking at that image i don't think networks.subnets makes any sense since it depends on the prefix length of the desired subnet
a subnet of one network isn't strictly just one bit smaller

[wispy jackal]

network.subnets is a handy shorthand for getting 1-bit more specific, you can use network.contains(subnet) if you want to check if something is a subnet of a network.
But thanks for the suggestion, I'll add something like network.get_subnets(bitsize) to the next release

The reason I went with 1-bit down was to make it conveniet for building a Patricia-Trie (for implementing fast Longest Prefix Matching algorithms)

To that end we also have network.left and network.right

[untold scroll]

I started building a TCP server-oriented framework, inspired by FastAPI.

[wispy jackal]

That sounds pretty cool, pure python? or something faster like a PyO3 lib?
If you're doing pure python, it's not really going to be that useful for the kinds of things we would normally use a raw TCP service for, beyond it's use as a learning experience for you. At a stretch it might get used for testing some protocols.
Compared to FastAPI, which is honestly a relatively thin routing layer on-top of Starlette (which is doing the majority of the heavy lifting), you're going to have your work cut out for you.
Looking forward to seeing your progress!

[untold scroll]

Yes, as you mentioned, Starlette is the actual entry point for FastAPI, and it's Starlette that implements the ASGI spec. I haven’t rewritten the server yet, but I’m considering forking Uvicorn and adding a custom protocol, this should be simpler in the short term.

The current version of the framework doesn’t rely on the fork yet; it’s still built on top of a pure Python TCP server. I’ve deliberately scoped it to full JSON communication for now (but the protocol will be extensible to support things like MessagePack or even Protobuf).

I’m currently refactoring the code for clarity, and I hope to have something clean to push to GitHub by the end of the month.

[golden coral]

i am aking a tool can some1 help me

making*

[wispy jackal]

What are you making?

[cloud spruce]

you need to give more information than that for us to be able to help you with anything

[barren palm]

How many software engineers are still in this server? I'm about to start college, and I was wondering if someone could give me some tips. I'm a beginner/intermediate in Python, and I'm currently using freeCodeCamp.

[cloud spruce]

this is probably the wrong channel for this
it's not a channel for networking with people but rather doing things with python in the realm of computer communications over a computer network

[glacial blade]

I find collaborator to expand my business.

[cloud spruce]

this is not the place for that

[clear yacht]

does QUIC support P2P communication with hole-punching or is it only for server-client? it's using UDP but I'm not sure

[cloud spruce]

QUIC can be used for P2P communication (there are libraries for this) even if the protocol was first developed for use in client-server applications such as http/3
i think ICE (Interactive Connectivity Establishment) can be used for hole-punching for QUIC but i'm not sure how well or easy that is to achieve in practice

[clear yacht]

so the idea is that you first use ICE to establish P2P and then use QUIC afterwards with the same connection?
also could you share mentioned libraries please?

[cloud spruce]

sort of, even if UDP doesn't really use connections as such, but high-level abstractions or programming APIs can present it as such
aioice looks interesting (and dare i say promising?🤞) even if i have never used it myself: https://pypi.org/project/aioice/
unfortunately i don't see any mention of DTLS support for the STUN communication over UDP (that would be support for RFC7350, which might have been desirable/optimal) for that library
you will have to either use a public or commercial STUN and/or TURN server (at least one of them, but a server that handles both is the optimal situation) or operate one yourself that is accessible to both clients for hole-punching with ICE to work

i have not really looked into QUIC for P2P connections with python specifically
libp2p comes to mind for Go, Rust and to some extent JavaScript, but their Python support/implementation isn't there yet, a work in progress and not recommended for use yet
but i'm sure there are more options than that
for python, i don't know if the QUIC stack/API of aioquic might be used for P2P, but maybe, as it's pretty standalone (it's just the protocol layers, byte streams, no real I/O, that part has to come from somewhere else)

just remember that with P2P connections anonymity of users are out the window (or at least the IP addresses of clients are known to clients that are connected to each other)

[clear yacht]

thank you for the detailed answers, I have all the information I was looking for 🙂
I'll try setting up a STUN server and toy around with it

[jovial ocean]

We’re looking for high school students to kick off projects on Research Bee. There’s no application or long process — just go to the site, make a project, and we’ll help match you with a team working on something similar.

The goal? Get you working on real research, with a shot at publishing by the end of the year. We’re also exploring future funding opportunities for selected projects.

If you’ve ever wanted to get into research but didn’t have the resources or people to do it with, this is your chance.

Start your project now: researchbee.me

[lyric oak]

#rules Rule 6

[median fractal]

Helo

Can someone help. Is the following workflow correct?

I got my pc connected to router which forms a LAN.
The router is connected to the internet
Now I make a request to google.com So the flow be like

Application program send a message DNS is checked and ip is resolved for Google and the message is sent to transport layer

Transport layer makes sure of ip and port and tries to establish a connection with the web server process running on the google pc. 
it sends the segment to network layer

The network layer takes this segment from transport layer and gets the destination ip it looks up in the routing table of my pc to check whether googles pc is in my LAN. if not, it sends it to the gateway private ip of my router in my LAN
The MAC address of my router is returned by ARP
It sends the packet to the data link layer

The data link layer takes the packet and uses the destination mac address to send the frame from my pc to the router

The bits are physically sent to the router as signals

Home Router
its NAT-Router it replaces the private source ip address with its public ip address
Does all of the below

Router receives a frame
It looks at the destination MAC in the Layer 2 header.
If it matches the router’s interface MAC (or broadcast), the router accepts it.

Strip the outer MAC header
Router removes the entire Layer 2 header and trailer (Ethernet header + CRC).
Now it’s left with the IP packet (Layer 3) inside.

Decide where to forward
Router checks the IP destination address.
Finds the next hop in its routing table.

Add a new MAC header
Router finds the MAC address of the next hop (via ARP or other link-layer protocol).

Builds a fresh Ethernet frame:
Source MAC = router’s own interface MAC for that outgoing link.
Destination MAC = next hop’s MAC.

Send the new frame

[fair kindle]

I think that in hostname resolution checking the host file is the very first thing that is done. Then it checks the DNS cache, and only then does it reach out over the Internet to the DNS server you have set up

[median fractal]

Yes thanks, understood.

[lyric oak]

you should be banned due to Rule 6 #rules

[glacial blade]

sorry my mistake.

[ocean harbor]

!ban 1400230359643197531 advertising

[errant bay]

:incoming_envelope: :ok_hand: applied ban to @glacial blade permanently.

[real bone]

poor dude and he apologized too

[lyric oak]

This "networks" channel is for computer network related discussions, not "social networking". You might want to visit another channel instead.

[dense field]

Hi im trying to get a Degree Apprenticeship and want to choose between Cyber Security and Network which is better in terms of career progression and exit opportunities?

[lament atlas]

I think both look pretty good! I went to school for programming and networking and had a decent time finding opportunities post college. I will not say which is affirmatively better as I think that comes down to the person, job market, and your overall goals. Network is a ton of fun from my experience, It is interesting to learn how our world communicates at different levels. That being said, security is also very cool, safeguarding all of the important things which we rely upon. My advice is to do some research and find which niche suits you best! I wish you luck.

[hearty wagon]

@vale wolf

[alpine sparrow]

Network security is better i use it evrey time and it works perfectly

[pure fable]

Dns runs the internet

[copper dagger]

a A corporate network includes an IDS that inspects and potentially blocks all the
TCP traffic towards hosts not associated to domains on a trusted whitelist.
Moreover, each client on the network is configured to trust a Certificate
Authority belonging to the corporation.
i) How can the security team leverage this setup to inspect also TLS traffic,
minimizing the performance impact?
ii) Despite the security measures considered above, a malicious insider could
manage to exfiltrate sensitive data towards a non-whitelisted host under
their control. Describe how that can be achieved, and propose another
security measure to mitigate or prevent such exfiltration.

for part 1 I mentioned using a proxy to decrypt payloads

for part 2 I mentioned that the insider could use dns tunnelling as it isn't checked by ids. is this the correct answer?

[wild estuary]

for 1 isn't it just mitming the server certificate with the CA's?

[cloud spruce]

!rule 8

[errant bay]

Rules

8. Do not help with ongoing exams. When helping with homework, help people learn how to do the assignment without doing it for them.

[cloud spruce]

the line of questioning makes it sound like the above is applicable

and of course it's an AI network, because all companies obviously need to jump on the bandwagon of the latest buzzwords...

[copper dagger]

yes, having the proxy as the mitm. I'm not sure of my answer to part 2 though

[cloud spruce]

is the above part of some test?
it sure sounds like it

[copper dagger]

yeah it's from a past paper

[cloud spruce]

for no 2:
DNS can use both UDP as well as TCP, so DNS doesn't automatically mean that it would evade TCP inspection due to this
sure, you probably mean to use DNS over UDP specifically and exclusively, just know that it's not automatically implied
but do note that the scenario says that TCP traffic is the one that is being inspected
on a normal network you will have both UDP and ICMP traffic at a very minimum, but probably a handfull of other protocols as well

[smoky socket]

https://blog.cloudflare.com/cloudflare-incident-on-august-21-2025/

4 hour response time is pretty quick

[cloud spruce]

you know this is the channel for "networking" as in computer communications, protocols and such as they pertain to python and not"social networking with people", right?

[wild estuary]

many such cases

[ember ledge]

Is there any places where I can learn the basic concepts of networking, trying to understand what subnet masks do and why we need them

[brazen badge]

I need some help, i don't know if this is the right place to talk about this but there are some networking related stuff. I'm running a Python socket server on my computer. The computer connects to the internet using a phone hotspot.
Because of NAT (Network Address Translation) and hotspot restrictions, clients outside my LAN can’t connect directly to my server using my public IP + port. Even if I bind my server to 0.0.0.0 and share my public IP, the connection fails since the hotspot doesn’t support port forwarding. I'm looking for a way for clients to reach my socket server over the internet, without paying for a VPS.

I considered DDNS and XAMPP self-hosting, but the real issue isn’t changing IP — it’s NAT traversal / hotspot blocking.

But i considered alternatives like Cloudflare Tunnel, ngrok, or a VPS to bypass NAT and make my server reachable.

Is there any other ways i should consider, can i really trust cloudflare as my alternative in terms of privacy and security issues?

It's my personal project, and the module I'm designing deals with connecting clients to the server and I'm using the socket library

[cloud spruce]

yes, you probably need to establish an outbound connection for most stuff, for example if you want to expose a local web server to the internet behind a NAT where you can't do port forwarding to it
for P2P protocols you got other options for NAT traversal (with ICE together with STUN and/or TURN), but for other type of connections that usually isn't an option
Cloudflare and other similar services that you list are all valid options, but usually they terminate the TLS connection and then re-encrypts everything again and send it back through your outbound connection that you have established to them for the tunnel service
but that means they can see all the request and reply traffic unencrypted, it's up to you if you think that is fine or if you got privacy concerns about that
additionally they often only support http/https (or something that can be transported by those protocols such as websockets) over the tunnel, there might be some services like ngrok that support any TCP sockets (if they still do that)

[cloud spruce]

if it's your software being used on the client-side as well, P2P connection and NAT traversal through hole-punching using ICE together with STUN and/or TURN might be an option in this case

[brazen badge]

👏👏 perfect, that's exactly what i was looking for. Let me research on that and see if it's a perfect fit for my problem. A huge thanks for your help, thank you

[cloud spruce]

there are quite large lists of free public STUN servers that you can use together with ICE, as STUN just provides the exchange of information necessary to connect the two ends with each other
however, if both of the peers of a connection are behind quite restrictive NAT you will most probably not be able to establish a direct TCP connection between the peers using STUN (know as TCP hole punching, in that case you will either have to use UDP instead (which is often much more successful but not guarantied to work either depending on the NAT implementation and configuration) or using TURN instead
there are also a few free public TURN servers, but they are not as many as they need to transport all the traffic like a proxy between the two peers in the conversation, so it wouldn't be true direct P2P connectivity but usually can overcome most NAT implementations and configurations unless the gateway at one of the peers is heavily firewalled, thus the TURN server would be able to see all the traffic between the two peers using that server (so encryption becomes even more important, even if it probably is a good idea even without TURN), also their might be quite a bit of congestion and hence latency through the free public TURN servers

[brazen badge]

You are right, I've done some research on my own and it looks like STUN would be the best option but I'd have to do some TCP hole punching because of the restrictive NAT. And with TURN there'd be some latency and the demand for extra bandwidth, i think the best option and easiest, cost effective plus scalable would be cloudflare tunneling. Thanks for the help once again ❤️

[cloud spruce]

as i remember it cloudflare require the traffic to be valid http traffic (after decrypting the TLS layer of https
hope that isn't an issue for you if you are doing sockets or if they changed it now to be more permissive in regards to protocols

[brazen badge]

I don't know, I'll try and see. If that doesn't work then i might have to go with Ngrok

[craggy ridge]

check msg reuquest rnd

check ur msg reuquest cosmic

[cloud spruce]

i generally don't do DMs
can't you say it here in public?

[craggy ridge]

word limit

but sure

anywyas

i need some help

with a project

could oyu maybe help me

?

@cloud spruce

[cloud spruce]

is it network related?
otherwise we shouldn't be i. a topic channel
and it depends on what it is and if i know the answer and have the time for it
the best place is probably to make your own help thread #❓｜how-to-get-help

[craggy ridge]

no

its on thread

rn

you cna check

first one on top

[rich turret]

@young pine

[prisma flax]

I don’t really know what channel to ask this in but I am trying to create a chat platform, P2P, no central server. Its main goal is to be a secured chat that passes through no third party servers or anything, just one side hosting and the other connecting as a client. I’ve tried implementing this but its my first time working with this kind of stuff and its probably all wrong.

I don’t really know if network is right but id also like to implement chat rooms if possible

[cloud spruce]

a decentralized chat app/room without no central server is quite hard to implement, you will have to find a way for two or more clients to find each other
and for NAT traversal using ICE (in the case that both parties are behind NAT) will require some kind of third party even if it is only to initially exchange information to setup a direct connection between the two parties using something like STUN if possible, or if that doesn't work let all traffic go through a third party in the case of using TURN

this is a quite advanced project if you can't find a few different libraries that will handle much of the heavy lifting for you

you can find out more about this subject if you read the conversation i had above just a few days ago with someone else about something very similar

[prisma flax]

As I said I’m using a DDNS way of finding each other, basically right now you can save people by whatever name and their DDNS, which in theory works. I can even link you the repo, it’s just one file for now and it’s very simple to understand

So to be able to first connect the two people would have to tell each other the DDNS

Chat Rooms are a definitely secondary implementation and they won’t necessarily require the same security as direct chats (although they could have it if they worked with 1 person hosting the rooms and the other people connecting)

https://github.com/StraReal/Cryptic

This is the repo if you wanna see the code directly

[cloud spruce]

okay, but the initial communication to exchange the DNS entry that each party is using must be done in some other way beforehand and then they must see to it that it's up-to-date so that it actually points to the correct and current IP address

[prisma flax]

Right now I’m using DuckDNS to get DDNSs but I get that that’s probably not very good

[cloud spruce]

yes, if one member is acting as kind of a dynamic central server for that room that would work as long as everyone that wants to connect there has the DNS entry for that person

[prisma flax]

It wouldn’t be very different to how phone numbers work in the sense that you need to know that to get “access” to a person

Also it’s not supposed to be something commercial but rather used in a workplace

Or a group

[cloud spruce]

sure, DuckDNS can be a bit unreliable/unstable but should work for now i guess, but any DDNS provider should work

[prisma flax]

Still, when I try hosting it just doesn’t work

I assume it’s an issue with port forwarding

I checked with canyouseeme and it says that the port I try opening don’t actually open

[cloud spruce]

you probably don't want to rely on people having to setup port forwarding on their gateways, in many places they won't even be able to

[prisma flax]

That’s why I came here

To seek a solution that doesn’t require manual setup by the user

I can ask people to get a DDNS but not to setup port forwarding in their routers

[cloud spruce]

exactly and it might not even be their routers if they are somewhere else then home or behind a CGNAT

to solve that read the conversation that starts here #networks message and my responses to them and the conversation that follows

you have more or less the same problem as them and the solution is the same

[prisma flax]

Oh alright I’ll read it rn

[cloud spruce]

ICE as in the Interactive Connectivity Establishment protocol

[prisma flax]

Alright n. 1 woah he has like the exact same problem as me even the wording looks like mine and n. 2 I don’t know what STUN or TURN or nat punching (I don’t know most of the terminology this is my first time doing this) but by how he answered it seems like it might be the right stuff for my problem so I’ll definitely look into it

Is there a simple way to explain them?

[cloud spruce]

do you know more in detail how NAT works and what CGNAT is?

and why they are used

[prisma flax]

No not really

I started doing this today basically

Ive looked online a lot (even committed the sin of asking ais for help) and this is where I got

[cloud spruce]

having some understand of network concepts like routing, ip addressing, firewalling and NAT (Network Address Translation), TCP, UDP and even ICMP would be useful when tackling a project like this

[cloud spruce]

i need to go afk like 30 minutes

[prisma flax]

You’re right its just that when I started I figured it couldn’t be that hard so I didn’t like inform on anything much

Just looked directly for how to do it

[prisma flax]

Oh yeah no problem

Alright, I looked up these things and what I got for now is that STUN is asking a public server to help you and another user communicate ips. This method only needs to communicate ips, not messages, but I’m not really sold on it.

I didn’t really understand what TURN is, I get its a fallback but not what it does

What I’m looking at is also talking about DHT which looks interesting

It seems to solve the issue of changing IPs while also permitting fully uncentralized chatting

Everyone has their own “contact book”

It sounds like the perfect thing

[prisma flax]

I just don’t know how I’d implement that

[cloud spruce]

it's not just IP addresses that are communicated, but rather port numbers for the connections as well
this is needed due to how NAT works and you will have to try to do hole punching:
https://en.wikipedia.org/wiki/Hole_punching_(networking)
https://en.wikipedia.org/wiki/TCP_hole_punching
https://en.wikipedia.org/wiki/UDP_hole_punching
to do the above you start with: https://en.wikipedia.org/wiki/Interactive_Connectivity_Establishment using: https://en.wikipedia.org/wiki/STUN, you can first try with TCP if that is your preferred protocol and then fallback to UDP or the other way around if UDP is your preferred protocol
if that doesn't work you do try to do the same thing using TURN instead: https://en.wikipedia.org/wiki/Traversal_Using_Relays_around_NAT start with your preferred protocol and then try to fallback to the other
or if you only want to support TCP or UDP you only try that one and then go on to the next method, but you will have less chance to successfully establish e connect to the other party, but most of the time TURN works even if it's not idle as you are connecting through a third party instead of doing true peer-to-peer connections

[cloud spruce]

the third party server in TURN connects the two connections and forward packets between them back and forth, all the packets, so not idle as i said above

[prisma flax]

What about using DHT? Doesn’t that solve all of my problems?

Although I didn’t fully understand how it works

Like you can get the ip of anyone knowing their id but there’s not central database with ids and IPs linked

[cloud spruce]

i don't think that DHT will solve your problem as you still would need to form an overly network of nodes in participating in the DHT network and be able to connect to those nodes

[prisma flax]

Oh alright

[cloud spruce]

the "database" would be distributed between the nodes of the DHT network and you still need to bootstrap the connection to that network somehow and then be able to connect to at least one of those nodes to get to the rest of the information in the network

[prisma flax]

So you were talking about STUN and TURN but from what I understood they also require third-party package passing

[prisma flax]

Do the nodes need to be always active or can they be the users?

[cloud spruce]

yes, there more or less isn't a way around that that works reliably over the internet if both parties that want to talk to each other are behind different NATs

[prisma flax]

And there isn’t a way to have direct P2P communication without port forwarding?

Im looking into hole punching

And from what I understand it uses a public server but only to open a connection, not to transfer data, which means that data doesn’t pass through any centralized server

Which is my main goal

[cloud spruce]

port forwarding shouldn't be needed, connectivity can often be establish (depending on how the NAT that each party is behind) through hole puching
but they still need some initial communication through a third party (STUN is just a standardised and well proven way of doing this) but after just a few messages through the that server the two parties should by able to establish a new direct communication channel between then despite of NAT, unless both NATs are really strict

[prisma flax]

So what you’re saying is that once I open a connection using hole punching and STUN the connection can stay open and be reopened without using them again?

[cloud spruce]

exactly, but you need the "server" or third party to exchange a little bit of information to then be able to established the direct connection

[prisma flax]

Could it be made automatic, as in, when I want to establish a connection I can automatically send a bunch of useless messages to do it without giving away any message?

Any real message

Also using this method what would two users need to do to connect? What do they need to know?

Their IPs?

(Which could be swapped for DDNS)

[cloud spruce]

it can stay open as long as you keep it alive with traffic between you or the NAT decides that the connection isn't allowed to stay open any longer and shuts it down (this last one is quite rare though) and you would need to establish a new connection
but each new connection would require you to go through the whole initial connection phase again with the STUN server

[prisma flax]

Then I could also send “useless” packets to keep the connection open, right?

[cloud spruce]

you don't send any "real" traffic through the STUN server, it is only used to exchange information to establish the direct connection, then you would exchange the real data through that direct connection

[prisma flax]

I think I need to go look at what these things are better because right now I just can’t follow you

[cloud spruce]

if your application is built for this, the application code takes care of all this in the background, it's not anything that the user will notice other than that it takes a little bit longer for the first message to reach the destination when there isn't already an established direct connection between the two parties

[prisma flax]

Alright that’s good

And using it are chat rooms possible?

[cloud spruce]

yes, exactly that, it's called "keep-alive" packets/traffic

[prisma flax]

I guess they could probably have an host and then clients connected to it, like I proposed before

And for two users to connect what do they need? Their ips?

[prisma flax]

I'm trying to use UDP Hole Punching. I just need a clarification: is a signaling server something that i have to build myself or are there pubicly availiable ones i can use?

[prisma flax]

What i need is probably webrtc

But I cant host a server

[lapis quail]

Someone help me pls

[prisma flax]

Above you can see when I had just started the project

And that guy told me for the first time to look into UDP Hole Punching

[solid bluff]

Responding here because I'm half busy with stuff and pygen moves fast.

But setting up your stuff locally with the following setup:

• Server on my desktop, using ngrok to get a url aiohttp won't hate and to provide an endpoint
• Both clients on my laptop on a hotspot (so different network entirely)

First client instance creates the room. 2nd client instance joins it. Both are connected just fine.

When I go to send a message though I get an error due to your usage of global variables and where you define your variables. Lesson there: use globals less or at least consistently. If you have state, just use the proper tools to manage state.

So what exactly is your setup? Where is your server setup? How is the server being exposed? Where are your clients running from?

[prisma flax]

So you can connect at least?

Alright so until now I have only been able to test my code with the server hosted by myself, I run a client and I got my friend to run a client on his pc too

And we tried connecting

But we both just didn’t get the others packets

My server uses a cloudflare tunnel which links a domain to my pc

(Specifically localhost:5000)

[solid bluff]

But yeah, it works for me with a super basic ngrok setup

[prisma flax]

Oh no yeah this works for me too

The problem is with the clients

After it says connecting…

Does it confirm the connection?

[solid bluff]

It does. It crashes when I try to send a message because of global var usage.

[prisma flax]

Oh alright maybe the problem then is my friends pc or mine

Can you show the error message for the crash?

Maybe I can fix it right now?

Btw is it actually dangerous to leak my ip?

[solid bluff]

So I would reduce what you're doing to barebones and reduce the variables with your network setup.

Can you reproduce what I'm doing with your friend? Like, your friend runs 2 instances and see if he can connect via your server. Because if you can't do that something else might be happening with your docker+cf+domain thing

[solid bluff]

I wouldn't share it in a discord server this big. There are lurking bad actors that might genuinely try to (D)DOS.

[prisma flax]

But wait you’re connecting with localhost?

So from your pc to your pc?

[prisma flax]

You should delete the image now that I’ve seen because the ip is legible

I didn’t read it but someone else could

Also I wouldn’t know bad stuff to do with someone’s ip and I’m not interested in doing any

[solid bluff]

oh lol, I just mean from people scraping channels for valid IPs. If someone wants to stare at a screenshot and try to figure out what it is and try to DDOS it by the time I disconnect my hotspot..... they need to get a better hobby

[prisma flax]

Lol true

[solid bluff]

The 2 instances I'm connecting to each other are localhost to each other. Same IP, different ports. Server is on a different network though in terms of exchanging info.

[prisma flax]

Yeah but the problem is when the two clients are not on the same ip

It works for me too with localhost

[solid bluff]

Did you try sending a message on localhost? You should also get the client crash I get

[prisma flax]

No actually I could send the messages

It worked like it should

I could even send small files

(Through the /sendfile command)

[solid bluff]

And you're running the current code on your github?

[prisma flax]

Yes

Look if that’s fine with you maybe we could try connecting? It would be on your terms, I’d join your server so you’d be the one seeing my ip

Idk if that’s still dangerous for you if it is then it’s not a problem if u don’t want to

Just trying to connect maybe would help

But it’s weird that you can’t send messages

Can you send the error?

[solid bluff]

I would double check that. Because I'm getting a NameError because of how you're working with globals

[prisma flax]

Alright I’m pushing right now although I’m pretty sure I didn’t do anything that changed it working

[solid bluff]

Also I am now connecting from different networks/computers and it's working

[prisma flax]

Can you send the console output of the clients?

Also using different machines from the same network definitely wont work

Just to lyk

[solid bluff]

They are not on the same network

[prisma flax]

Yeah ik it was just to tell you

Are you able to send messages between localhost now?

Wait I didnt push

Uhhh it doesnt let me push wait im trying to fix this

Oh yeah its not letting me because its all already identical

So yeah the code in the repo is the one im running

Here you can see my console output

Local is Term2, Local (3) is Term1 and Local (2) is the server

[solid bluff]

Question, looking at your code to narrow down the issue you might be seeing. Why do you start the threads in daemon mode?

[prisma flax]

Oh wait youre right

So I probably copied the code because i didnt know how to open a thread

Like copied the first somewhere

And then copied the first line everywhere else

[solid bluff]

If you start a thread in daemon mode, it'll suppress errors from it

[prisma flax]

yeah i just looked up what it means

Alright i pushed

I mean you dont need to go get the code from the repo its just two lines that need to be changed

[solid bluff]

So if you/your friend run the code now and try to connect, you should see error messages now to help you narrow down the issue

[prisma flax]

Well I can't test with my friend right now

If I use my phone's hotspot that IP is useless right?

Like I can share it?

The fact that there are differences from me running the code to you running the code is pretty concerning

Like how will I know that I havent fixed the problem on my machine?

[lyric tulip]

Hey i am learning networking from Networking Fundamentals - Practical Networking (it is a yt series) Is it a viable source of knowleadge?

[lyric oak]

If you just want to learn for fun or basic usage, any reasonable materials are OK. But if your goals are higher, you can never skip classics like TCP/IP Illustrated.

[lyric tulip]

Interesting,could you explain further?

[cloud spruce]

!rule 9 6

[errant bay]

Rules

6. Do not post unapproved advertising.

9. Do not offer or ask for paid work of any kind.

[cloud spruce]

this channel is for computer networking such as TCP/IP and other protocols and technologies as they relate to Python, not social/people networking!

please read the server #rules and #code-of-conduct as well as the channel topic/description before posting

there simply isn't any channel on this server where things like this are allowed

[ember ledge]

Is TCP/ip and tcp different?

[cloud spruce]

IP encapsulates TCP, they are two different protocols, TCP/IP is just a way to mention both at the same time, like saying "TCP over IP"

[lyric oak]

TCP is more about the single protocol itself. TCP/IP is more about the entire network stack (and all protocols involved).

[cedar forum]

not really

TCP/IP is about the TCP & IP part of the network stack

the nature of having layered communication means that when you talk about TCP/IP you don't need to worry about the protocols that come under that in the network stack, you can run TCP over IP, you can run UDP over IP, you can then run IP over Ethernet, 802 wireless networks, PPP etc.

[lyric oak]

Only if you count IPX/SPX (or similar) which is not widely used today.

[cedar forum]

ICMP?

[lyric oak]

https://en.wikipedia.org/wiki/Internet_protocol_suite

[cedar forum]

yeah I know it's commonly referred to as that, but I think you need to be careful when explaining that it's because of the layers

[lyric oak]

ICMP is optional, and many servers have that disabled.

[cedar forum]

ICMP is not optional, responding to ICMP echoes is optional, there are many other features inside ICMP that you use every day

ICMP is the protocol used to notify you of unreachable destinations, time exceeded, in IPv6 it's used for NDP and loads of multicast things

it is still very much the default side-channel for IP and is used considerably more frequently than you think

ICMP is much much more than just ping

[cloud spruce]

no, that's very much not the case

ICMP is essential because of path MTU discovery and signalling other problems as well

[ember ledge]

I’m just going to pretend I understand all of that. Otherwise I’m going to have a heart attack for my upcoming assignment

[cloud spruce]

are you familiar with either the "OSI model" or the "TCP/IP model"?
if you are going to dive in to this sort of topic it can be helpful to at least be vaguely familiar with them and understod the basics of the thinking behind them

[ember ledge]

I was taught it in class but I just struggle to understand it. I might just look over the PowerPoints and. YouTube videos

[gusty lintel]

Its basically like those Russian dolls. The higher levels encapsulate the information of the lower ones.

Which means If you speak TCP, you also speak IP and whatever is used for Layer 2 and 1

[cedar forum]

echoing what Steffen said, it's useful to just look at it graphically. instead of just hearing the layers and whatnot

you have some data called XY with value Hello world! needs to go across the internet. encoding that and sending that down a wire is obviously not going to get you anywhere, so you need to gradually prepare that data by adding all the information that will allow it to be routed through all the various places it needs to go whilst travelling across the internet

the first problem you're going to hit is that you oftentimes can't just put all your data in one packet, say you were downloading a game and you had 4GB of data, a 4GB network packet would cause you all sorts of problems

there are the obvious ones like if a byte is missed, misread or otherwise is not processed for whatever reason you need to restart the transmission of the whole packet

there is the less obvious ones (without understanding routing) about how it'd mean you need to have a fairly constantly open data stream for a longer period of time which not only wastes your resources but the resources of networks helping get you along the way

so TCP is the first step to preparing that data to be sent across the internet

for the purposes of the example, let's say that we decide we can only fit 6 bytes into a single packet at a time, we're now going to have two packets containing the data "Hello " and "world!"

TCP being the reliable and ordered protocol will attach some metadata to each packet, importantly the sequence number which is quite literally just the number packet it is in that sequence of data transmission

so we'd assign the first bit 1 and second bit 2 for this example, that way even if packet 2 is routed a different way, arrives faster, or whatever, when the message gets reconstructed we have our original Hello world! message

TCP also has things like checksumming to make sure that once the data is received by the destination device it is intact, so it's just a simple mathematical check to say "If you calculate a checksum of the data and get (for example) 36 then your message was intact"

If the checksum is wrong, a sequence number is missed or the recipient otherwise things there is a problem with that packet it can request a retransmission which is all handled seamlessly by the TCP implementation your device is using

it'll rebroadcast that segment

that's a very high level overview of TCP, the segment structure is fairly simple, basically just "this is a message from port XYZ to port PQR", the sequence stuff, the acknowledgement stuff for the return packets, some control bits and other segment metadata, a checksum, and then your data

of course you can see there that it doesn't contain the actual destination, there is no way to know where a TCP packet is destined (apart from the destination port, but that's obviously useless without the IP address)

so, that's where we get TCP/IP, we create our TCP segments, get them all ready for transmission so they're serialised to a bunch of bytes, and then we have to put that TCP segment inside an IP packet

that IP header looks like this

you can see in that one the source address and destination address are finally included! we now know where that packet is going

(this is ipv4 but v6 is fundamentally similar)

so you prepare all the data to fill the IP packet headers, where is your packet going, coming from, etc., how long is the data, a checksum of that header data to make sure you're not sending packets to the wrong address if the header data is corrupt

then inside the data of that IP packet you put your prepared TCP segment from earlier

you basically then follow that process, serialise the IP packet with the TCP packet inside it to bytes again and put that in the lower layers like the link/network layer

and then obviously on the other side you just peel the layers off again

(some of my wording is a bit off for bits of that explanation, layer 4 messages (like TCP) are generally called segments (apart from UDP which is datagrams), layer 3 (like IP) are generally packets and layer 2 (like ethernet/wifi) are called frames, but I forgot that when I started writing 😅)

[proven otter]

from a specific book?

[cedar forum]

Unsure, just took the first one that fit appropriately

[ember ledge]

I love networking

[marsh geyser]

Wow

[hexed kestrel]

wow

[lofty bough]

@gusty lintel That's not appropriate for this server. Please keep it PG-13

[cedar forum]

the PG version which is appropriate for here:

People
Don't
Need
Those
Stupid
Packets
Anyway

[lofty bough]

Is it a good theoretical model anyway, if you need a mnemonic to remember what's in it

[cedar forum]

yeah that's why I prefer th DoD model

[lofty bough]

That's a phin model

[next jasper]

When you understand even slightly what each happens in each osi layer, the names are easy to just get from that. But if I need to say a number of the layer, I need to write all down, lol

[gusty lintel]

Aww

[ember ledge]

OSI Model all day every day

[rugged atlas]

I have created local proxy server. I want to redirect url or show custom html page when specific domains are accessed. It is working but partially. It works on http request i think not https. It works on firefox. Some domains are redirecting to http version and displays custom page. but not on chrome i believe. Any solution? Can we redirect domains to other domains/URLs?

[cedar forum]

you'll need to build a man-in-the-middle proxy if you want to reliably redirect HTTPS requests elsewhere

you can just about build a filtering proxy without needing to be in the HTTPS request by using the SNI field of the TLS handshake (for now) but anything more advanced where you actually want to return something you'll need to build your proxy to run over HTTPS

[rugged atlas]

i am trying it with node package "http-mitm-proxy" but no progress yet. Ai tools are just putting me in a circle of errors. the main problem i see is, installation of certificate might drive users away from app?? can we bundle certificate installation along with software installation? to make it look trusted and easy to install

from where i can learn more about MITM proxy? any book or docs?

[cedar forum]

you can't really do that, no

man-in-the-middle proxies are pretty hostile and nasty pieces of software

on android for example the behaviour you're describing will never be possible because Android doesn't allow trusted certificates to be added unless you root your phone

[rugged atlas]

i only want windows and macos proxy

[cedar forum]

what are you actually trying to build here?

[rugged atlas]

websites blocker app

like freedom.to

[cedar forum]

right, then yeah you need to install a cert and have the user open their privacy settings and accept your trusted cert I'd guess

though some browsers have their own certificate store so you'll need to provide the user with the raw certificate and guide them on how to install it into the browser

[rugged atlas]

i am scratching my head. idk how they made a proxy because i cant see it in windows proxy settings (it is toggle off). it is also not a DNS because i have seen DNS setting too. if i force close freedom proxy it turns itself on automatically. they dont ask for cert installation

[rugged atlas]

is it possible to generate cert automatically and ask the user to install it within one click? because if installation process takes some time its better i quit this approach and go browser extensions way

[cedar forum]

does freedom have a browser extension?

https://support.freedom.to/en/articles/1347519-freedom-browser-extension

However, certain types of URLs and certain browsers do not reliably display this green screen. To fix this issue, we've created the main Freedom browser extension. This extension is available for Chrome, Firefox, Edge and Opera!

i would guess this is HTTPS sites

[rugged atlas]

no

oh

i was impressed by their HTTPS redirect toward block.freedom.com

[cedar forum]

lol half the things in this FAQ read exactly like how malware works

preventing itself being uninstalled lol

[rugged atlas]

that is their selling point. it is made for master procrastinators

[cedar forum]

yeah i get it, just amusing how much overlap there is here between malware

[rugged atlas]

they have negative reviews on playstore because their approach is VPN on android. users just turn off VPN and it becomes useless application

[rugged atlas]

actually this is very similar to my current software lol. my current software also display block page (HTTP version) on some websites but it fails on some websites especially on chromium. so i can also try hybrid approach like their. i used to think their approach has no flaws but i was wrong lol

[cedar forum]

anything like this is going to have flaws because every operating system is battling to keep software from this being able to work

[lyric oak]

There are MITM proxies open sourced, but they might not advertise themselves very well. You can find posts like https://www.reddit.com/r/opensource/comments/14l945q/opensource_alternative_to_charles_proxy_telerik/ and see into their designs. That's how people study such topics when no clear docs or good books are available. If they were not written in Python, you can replicate the ideas in Python code usually as the basic building blocks are available.

[cloud spruce]

either way you would need to get a private CA certificate (or trust anchor if we should be fully accurate) on to their device, and sometimes even into the application
i would consider that highly malicious

[lyric oak]

Enteprises deploy such to desktop/mobile via policies and that's a common practice. I am not sure where your "highly malicious" comment came from.

[cloud spruce]

this is because it's exactly what TLS and related tech is built to protect against, together with HSTS (especially preloaded), encrypted SNI and even DNSSEC it will make life difficult for you unless you have a foothold on the end device

[cloud spruce]

i know, often through MDM platforms to get their CA certs in

[cloud spruce]

i mean if you don't own the device (at the very least one an administrator level but preferably legaly as well, as in that it is the companys property) you are doing this against, like if you are trying to do this with other software that you install, like it sounded like @rugged atlas wanted to do

that's what that comment came from

if it was a service i certainly wouldn't trust them with all my data that suppose to be encrypted by TLS, and would never knowingly install such software on my privately owned devices to get some filtering done by a third-party by letting them break all my TLS sessions

[rugged atlas]

How many steps are required to install certs? Does it require to install on every browser? I think it will be annoying for users.

[cloud spruce]

but on company owned devices that's another story as long as it's initiated/controlled (on some level) by that same company

[cedar forum]

It is made purposefully difficult and requires user consent (often in multiple places) because of the extreme power that it grants

[cloud spruce]

MDM software is allowed to install trusted certificats on all major platforms an enterprise would typically hand out to their end users, even android

[cedar forum]

yeah -- should've clarified I mean specifically for user installed certificates on non-MDM devices

[cloud spruce]

without rooting the device

[cedar forum]

but yeah assuming no MDM there is no way to put those in the system trust store now last time I checked

[cloud spruce]

i think you can still install a CA certs on android from the local file system on the device
but last i checked it's really cumbersome (if it is even still possible) on iOS/iPadOS without using MDM, requiring hooking up the phone to a mac with special software to install a packaged management profile on it almost like setting up your own MDM and enrolling the device with it

[rugged atlas]

i guess i have to decide whether to keep blocking websites with proxy server (tunnel failure error page) but no custom page or choose extensions method. certs are only required for https redirect or display custom page. proxy is already blocking websites.

[cloud spruce]

yeah, proxy, DNS or firewall (for example through a VPN tunnel) will all be able to block access to TLS sites, but without any fancy pages to be displayed, insted you will either get a message that the site cand be found or just a connection error as if the site was down

[gusty crane]

hi will learning bash scripting helps

[cloud spruce]

help what?
and why are you talking about that in #networks when it sounds more like #unix ? (where i think you should probably move this discussion to)

[glossy remnant]

sorry, I'd like somebody help me with programation in the python

I'm a beginner

[sharp vortex]

How much of a beginner? I'm also a beginner

[rich egret]

hello

completely lost

erh

i bought a dedicated machine

from worldstream right

i have a webserver on 0.0.0.0:5173 perfectly fine and accessible

created windows firewall rule

also fine

but i cant seem to like access it from public ip:5713

nothng in the worldstream dashboard abt creaitng a firewall rule to allow

port 5713 external traffic

so

i contact support and theyre like reverse dns! or get another expert to help u!

[ember ledge]

Curious, when someone talks about Ethernet address, they mean MAC address right?

[vague cedar]

Yes and no. Yes, MAC addresses are a Layer 2 addressing scheme, and Ethernet is a layer 2 protocol. But Ethernet isn’t just MAC addresses, it’s how all data is framed up on such networks, whether it is TCP/IP or something cooler or weirder.

[ember ledge]

Reason why i ask this, is because I got a question on a assignment asking how many bits does a Ethernet address have, I just wrote 42 bits because I just thought it meant MAC address

Now the question has me confuse haha

[vague cedar]

Eww I think your answer is pretty good, I dunno what an “Ethernet address” is.

I think it would be trivial to run a different addressing scheme on top of it that wasn’t MAC but maybe I’m missing something.

[ember ledge]

“How many bits are in an Ethernet address” I couldn’t find a straight answer for it in the learning material, I’m just leaving it as 42 for now. Couldn’t find much about it on google either

Now I’m trying to figure out what is the frequency range of IEE802.11a thats going to be fun

Which is 5ghz let’s goooo

[river sphinx]

Ethernet is a protocol. Layer 2

So yes, Mac addresses

[ember ledge]

Learning is so much fun and stressful at the same time

[modern hemlock]

I have a community project called "Portfolio do <dev>". Our main goal is to bring together people who want to participate in complex projects that are difficult to complete alone and start developing projects as a team. This is not a paid work, it's a voluntary work, in the end of each project you will be able to show off yours skills and put that project on your portfolio. We're strongly focused on networking and team work, interacting and having fun while developing. Feel free to chat me on private

[lyric oak]

off topic on this channel, as it is dedicated to computer networks, not social networks.

[round glade]

helow

[bronze gust]

Cloudflars free WARP version safe? Does it steal data?

[cedar forum]

it doesn't, but you need to understand how it works

remember that sites you visit can still see your original IP if they use Cloudflare

it is solely a tool to try and optimise for speed and avoid censorship from your ISP if applicable, it is not a privacy/anonymity tool to sites that you visit

[bronze gust]

Yeah. I did a bit research on it. It suppose to faster my wifi speed so i was interested

[bronze gust]

Yep

[ember ledge]

What does the quic protocol do

[cedar forum]

it's an alternative transport layer protocol, comparable to TCP

as a very basic summary it's a combination of UDP + TLS

[ember ledge]

Yea, thats just something I been trying to understand. Spent like some time googling it and watching YouTube videos on it. I know it’s meant to provide faster loading times, and be secured. But thats just my limited knowledge on it

[cedar forum]

that's basically it

previously HTTP/1 and HTTP/2 have been on top of TCP

so there were a lot of guarantees made by TCP that were good for HTTP, things like ensuring packets are received and organised in the order they were sent, etc.

but there were also a lot of things that were prohibitive in there because TCP is meant to have a lot more guarantees about reliability

QUIC kind of sits as a bit of a middleground between UDP (throw stuff and see what reaches the destination) and TCP (sent packets will reach the destination) and has a lot of optimisations based on things we've learnt since TCP and UDP were first deployed

TCP and UDP are very very generic, optimised to carry basically any and all data, QUIC focuses more specifically on building a protocol that is designed for HTTP & browser based traffic

[ember ledge]

So in short, instead of having layers, just all built into quic

[cedar forum]

sort of, there are still layers

it's more just the layers are optimised for browser and HTTP traffic

[ember ledge]

Guess, I’ll do some more research for it. I had an assessment where I had to explain the quic protocol and why it’s better over tcp. And I’m quite confident I didn’t explain it right. So yeah, just waiting on the feed back for it

[vague cedar]

also check out how "Time-Sensitive Networking" (TSN) works.

For a very different approach to solving some problems.

[ember ledge]

What you mean? Is this like some YouTube channels or like is it another protocol

[vague cedar]

https://en.wikipedia.org/wiki/Time-Sensitive_Networking

Some applications need better latency guarantees than you can get even with QUIC

[ember ledge]

Ohhh I see

At the moment, I’m just focusing on quic for my school work. But I’ll take look at that as well

[vague cedar]

Yeah, don't need to study it that much, just be aware that this problem/solution exists

Because it helps you understand what QUIC "can't" do

And that's guarantee that the whole process happens in less than x milliseconds or whatever

[wild estuary]

ah, so DTLS /j

[cedar forum]

🤣

[wild estuary]

also, whats the difference between a datagram and a packet?

[cedar forum]

the definition i was taught is that datagrams are any pieces of data delivered via a protocol that does not make delivery guarantees

[wild estuary]

right

[cedar forum]

though i'm not sure if that's a definition derived from the fact we call UDP packets datagrams or if UDP packets were called datagrams because of that

packets are level 3, frames are level 2

and level 4 is either segments/streams for TCP/QUIC or datagrams for UDP

[wild estuary]

huh

[cedar forum]

https://en.wikipedia.org/wiki/Datagram

The delivery, arrival time, and order of arrival of datagrams need not be guaranteed by the network.

[ember ledge]

Might as well start using UDP for everything

Say goodbye to tcp

[hybrid coyote]

I have a microsoft azure cloud vm up and running. I want to use it as a rendezvous server for my p2p video game
My idea is that it will exchange the IP and ports to the connecting clients and each client will do some UDP hole punching to allow data transmission among each other

Is this even possible, I mean I see this guy do it but I think he has open ports on the two clients, even though he states that they have full firewalls blocking all incoming connections
https://youtu.be/TiMeoQt3K4g

i was trying for quite a while and didnt get it to work, im probably going to spend some time trying again

[hot cloak]

Guys i have a query

If osi model is outdated

And not used

Why do we still start networking with ti

[vocal kernel]

Who said is updated?

[vague cedar]

IMO even if it has turned out to make implementation sense for some of the layers to "merge" into the same hardware/software over time, the OSI model is still indicating correctly the different "steps" involved, and is worth learning for that reason. Like, PHY/MAC stuff is all one thing now, and has been for decades for electrical efficiency reasons, but the two parts are in fact doing very different things etc.

In the early days the MAC was a pluggable hardware module

https://www.sciencemuseum.org.uk/objects-and-stories/arpanet-internet

https://www.rfwireless-world.com/terminology/phy-vs-mac-in-iot-networking

[cedar forum]

for the most part we do still follow the OSI model as well, it's just that most traffic nowadays (HTTP, etc.) uses protocols that span multiple layers. the protocols are all achieving the goals of each layer of the model (so TCP handles both session and transport, really)

there are other protocols that are commonly used where something is used at every layer of the OSI model that is distinct

like going for a bit more of a complex example than HTTP, let's say you've got a phone system and we'll stray away from using more traditional protocol parts

you might have something like:

Layer 7 - Application: SIP (Session Initiation Protocol)
Layer 6 - Presentation: something like G.711
Layer 5 - Session: RTP (Real-Time Transport Protocol)
Layer 4 - Transport: UDP, fairly standard and what RTP runs over
Layer 3 - Network: MPLS (Multiprotocol Label Switching), just a way of routing packets around a wide area but using labels instead of IPs
Layer 2 - Data Link: PPP (Point-to-Point Protocol), an alternate layer 2 layer that's frequently used by ISPs and phone providers (specifically PPPoE)
Layer 1 - Physical: DSL (Digital Subscriber Line) - the technology used to physcially move data over copper telephone lines

that's a slightly contrived example but you can see how there are still a lot of application deployments that use all 7 layers and have distinct protocol suites running at each layer

even in setups where we've merged the layers a little bit and you have protocols that span multiple layers, each layer is still a requirement that the protocol must implement, so even when looking at HTTP you have like:

Layer 7: HTTP/1.1 or HTTP/2 or whatever
Layer 6: TLS (at this layer TLS handles the encryption, compression, formatting, etc., handles message authentication)
Layer 5: TLS again (but at this layer it's managing the session lifecycle at a higher level, this part of TLS is more around the handshake and exchange of keys, management of a "session")
Layer 4: TCP, ensure that all of our packets at higher layers are delivered in order and redelivered if they fail
Layer 3: IP, routing packets to machines over the internet based on IP addresses
Layer 2: Ethernet / Wi-FI, frames those higher layer IP packets and handles targeting local delivery using MAC addresses
Layer 1: Copper/Fiber/RF, the actual encoding of all the higher layer packets into a bitstream that can be sent over a physical medium

OSI model is still very useful and very correct at explaining what a set of protocols need to do for data and complicated parts of networking to be abstracted away from a user, DoD model is a little closer if you're looking specifically at how many protocols are used to achieve those tasks, but there's a reason they map mostly 1-to-1 (just with some layers merged)

[cedar forum]

hahaha we had one of these in the reception of the Computer Science building at my University, hooked up to show fun status lights and stuff

[hot cloak]

I am not too sure about the example you gave first but this one is a bit convoluted in the sense you might trying to fit in something that doesn't belong in this box http is more less atleast predominantly referred with TCP/IP stack as implementation of the stack is closer to reality than osi

[cedar forum]

you asked generally about the OSI model and how it's outdated, I gave a set of protocols and a hypothetical use case where all 7 layers are used

[hot cloak]

Hypothetical being the keyword here

[cedar forum]

TCP/IP stack is still explainable using the OSI model

[cedar forum]

it is this

you still have 7 things happening, just some of those things are handled internally within a protocol instead of being handled by an encapsulated protocol, hence the merging

if you actually look at how TLS works, you'll find it is actually much closer to a 2 layer protocol

[hot cloak]

Sure but osi model was not concocted for the sake of explanation it was just as real model as TCP/IP stack but no one ever actually implemented it

[cedar forum]

that's not what the OSI model is

the OSI model is a very general abstract view on how data can be routed across a network

also, for reference, the OSI model came after we started talking about TCP/IP and the Internet protocol suite

we designed OSI to describe those protocols

[hot cloak]

Nope.

[vague cedar]

OSI has been implemented

Just not recently

[cedar forum]

it... was

[vague cedar]

The layers collapsed for performance reasons.

[cedar forum]

https://en.wikipedia.org/wiki/Internet_protocol_suite

The Internet protocol suite predates the OSI model, a more comprehensive reference framework for general networking systems.

[vague cedar]

the full "physical incarnation" of the model is not energy-efficient

[hot cloak]

Osi and TCP/Ip were very much competitors

[vague cedar]

If you've used an older ATM (asynchronous transfer mode) network, you used a "real" OSI network.

[vague cedar]

I had a 45mbps version of that across-town at a job in the 90s to link our two datacenters etc.

"The map is not the territory" is correct, but the rest of that take seems unnecessarily combative to me.

[cedar forum]

the thing is OSI seeks to solve a lot more problems than TCP/IP applies to

[vague cedar]

There are a lot of useful maps that "aren't the territory".

[vague cedar]

100%, more useful when discussing Time-Sensitive Networking standards etc.

[hot cloak]

Osi may have been remodeled for academics or debugging but at no point of time it is a standard , its a mental model you can use for debugging but it's mostly subjective

[vague cedar]

Nobody's saying otherwise that I can tell

That it's an abstraction to differentiate the various possible layers you might choose.

It's not a "spec"

[cedar forum]

my initial answer came from how you started the conversation by saying OSI is "outdated" and "not used", which is false because obviously there has to be a model to fit all the other traffic on the internet

[vague cedar]

It's more like saying "Formula 1 cars are always gonna have four wheels"

(one had 6, was too good, they banned it)

[cedar forum]

TCP/IP fits HTTP, email, SSH, etc.

TCP/IP does not fit local stuff like ARP, ISDN or any other number of telecoms applications, DNS (sort of), so something else has to

[vague cedar]

and TCP/IP has no place at all in a real-time world.

[cedar forum]

yeah

now with protocol merging and different applications having different session, presentation, etc., needs, the OSI model doesn't fit all those uses either, there are times where you won't need higher layers, you'll merge layers, etc., but the goal of the model is to be as all-encompassing as possible and you can (somewhat) pick and choose which bits you want to implement

[vague cedar]

To me it's like saying that knowing about Abstract Syntax Trees is useless if you don't program in Lisp.

[hot cloak]

Can you elaborate on this

[cedar forum]

(this is basically the answer -- most real-time protocols do not map well to TCP/IP)

a lot run on UDP, a lot don't use any IP addressing at all (that's where things like MPLS come in)

some people try and argue that UDP and stuff come within TCP/IP (because TCP/IP is the "Internet protocol suite") but it's easier to explain and show how layers interact when using other models

[vague cedar]

Real-time means things happen before a certain cutoff, or the world catches fire.

and TCP will re-transmit forever to get there, violating such constraints

[cedar forum]

there are a hell of a lot of other non-telecom things which use protocols to move data but do not fit within TCP/IP, basically any industrial network for machine-to-machine communication does not fit

[vague cedar]

Suite of IEEE specs designed for things that need more: https://en.wikipedia.org/wiki/Time-Sensitive_Networking

[cedar forum]

modbus, etc.

[vague cedar]

Oh yeah that too, "systolic array" kinds of networking etc.

[cedar forum]

it's things like this which is why other alternate and broader models are useful and are relevant

[vague cedar]

That makes sense to me, use TCP/IP for command and control to talk to Linux servers easily, and do whatever deterministic thing you need for the actual industrial interface.

PROGRAMMING your assembly line isn't a real-time problem, but running it is, etc.

[ebon mist]

I am trying to get p2p comms working , where we have 1 ipv4 address behind carrier grade nat. can someone help me understand if there are any existing python libs that are already doing this job ?

[ember ledge]

Hey there. I would recommend you those libs here :

pip install p2pd
pip install pyp2p
pip install aiortc
pip install python-p2p-network``` 
These should do the job.

[slim latch]

hi im completely stuck with socket now. it is transmitting wrong data too often. its using simple sendall and recv with a buffer of 8192 bits. transmitted data has around 4096 bits. its about 50/50 if it transmits the correct integer or not

what should i do?

[vague cedar]

endian-ness problems?

[slim latch]

oh uhm first time hearing about this

how could it be related to this packet receiving problems?

should i just use x.to_bytes(1024*8, 'big')?

or should i also use recv with the actually expected datalength

[vague cedar]

What I mean is, make sure you are handling the bytes in "network order" at all times, many systems default to the opposite, so your receiver might be reading the bits "backwards" or your sender instead etc.

[slim latch]

i tried
s.sendall(big_int.to_bytes(1024*8, "big"))
and
big_int_recv = int.from_bytes(s.recv(1024*8), "big")
and it sadly still isnt working

[vague cedar]

Hmm. Out running an errand but I’ll try to ponder what else might be wrong. Try printing the data in binary string form and see if you see anything revealing

[slim latch]

ugh so whats interesting now is the byte data seems to be the same

but the int isnt

how tf

s.sendall(msg.to_bytes(1024*8, "big"))
and
rcv = s.recv(1024*8)
the bytes seem to be the same but int.from_bytes(rcv, "big") doesnt output the same number

im so confused

[vague cedar]

What library is that, providing from_bytes? I'll go look at the docs

just the built-in integer version?

The byteorder argument determines the byte order used to represent the integer, and defaults to "big". If byteorder is "big", the most significant byte is at the beginning of the byte array. If byteorder is "little", the most significant byte is at the end of the byte array. To request the native byte order of the host system, use sys.byteorder as the byte order value.

aha so maybe you want sys.byteorder on the from_bytes call?

[slim latch]

i used the byteorder "big" so i think that should do it

i really dont get why it gives an other int tho

[vague cedar]

What does it do if you just ask for it "Wrong", e.g. pass "little" on from_bytes?

[slim latch]

thanks a lot for helping with troubleshooting
the problem was the thing processing the numbers afterwards

my own function

[lyric oak]

You might define your own “protocol”, about what data are passed on the wire over a socket for this application and how each piece should be encoded/decoded. With that you can ensure the sender and receiver code is compatible in the long run and you won’t hit similar issues.

[vernal jay]

hello

[cloud spruce]

this is a very late response, but be aware of how socket.recv works, it's not at all socket.sendall() in that when receiving you only specify the number of bytes you want to receive as a maximum other then a possible exception you are only guaranteed to get at least one byte from the socket.recv() call unless the the peer has disconnected, in that case you'll get an empty byte object

in other words, you need to be doing socket.recv() in a loop until you made sure you got the expected number of bytes or a delimiter of some sort that you are looking for or get the empty byte object or an exception

[signal sonnet]

so theirs a bunch of old office compuiters sitting in my local ewaste collection site. and i was thinking i could buy one or 2 to run my own servers for random stuff honestly. since their like selling for dirt cheap. Any think i should know/ any reason why this would be a bad idea. I also want to do this to better learn networking since i am currently learning CS

[cloud spruce]

they might be bulky and use more power then just using one or a few raspberrys for the same purpose
depending on if you are okay with using ARM processors or if your application requires x86_64 or if you need extensibility that might be more then what you can easily get with a raspberry pi or if you need more performance than that

[signal sonnet]

well the computers are like half the price of a pi , and honestly i just really want to learn some networking host a minecraft server or 2, and maby host a my projects their insteaf of netlify or paying for a vm

[cloud spruce]

doing a home lab is no problem if you can spare the space and the money for the power draw as well as any noise such systems will make
the good with with a pi is that it generally runs on much less power (which otherwise cost more money in the long run) and can run without any active cooling which makes them silent and takes up very little space
but that is only things that you can decide upon
bigger systems has other advantages instead

but exposing even parts of the systems that you host yourself to the internet is quite dangerous unless you have very deep and broad skills already, probably not something you want to do while learning

either way, practical learning setting up a network at home for your home lab is a very good way to learn both networking and several aspects of cybersecurity as well as systems administration

[sudden arrow]

need help doing a test-run of my cryptocurrency project, specifically the networking code

[prisma flax]

Hey guys, I’m looking forward to create a protocol to create decentralized networks and I need to see if there’s any issue in the concept

The way it would work is simple: when you receive a request from a new client to connect you either add him to your connections if they’re not full or you “send” him to all your neighbors.

When he is placed somewhere a message is sent throughout the network so that all others drop it and the neighbor who connected him tells the other neighbors (now I’ll explain how he knows who they are) to connect to him too

With this protocol you can create networks of any dimensionality, possibly even 1d (although it would be very weak), to figure it imagine it as a 2d grid

Each node has a coordinate

When you wanna send someone a message you already know their coordinates (probably using a UUID) and send it to them through all the possible ways that have the same length (which even in 2d works pretty well).

If node (4,4) creates node (4,5) he knows his neighbors are (5,5),(4,6),(3,5) and so he tells each of them about it and to connect to him

Also you connect to everyone who’s far two (in a single coordinate), so if someone disconnects the network doesn’t get fragmented

So (4,4) would also connect to (2,4),(6,4),(4,2),(4,6)

Someone please help me find any flaws if there are

[keen knot]

Guys

I m gonna make a big prjct

[olive mural]

Is there a free way to send messages to Whatsapp and Textnow in python?

[wild estuary]

what if your neighbors become offline

what if their IPs change

how would you discover coordinates and link them to IPs

[prisma flax]

If a neighbor disconnects or their ip changes its the same thing: between each neighbor there is some kind of back and forth to check if they’re still online. If atleast two neighbors say he’s offline, he gets disconnected and they all have a free connection spot, so it should get filled again

Also you don’t need to discover someone’s ip, you just need their coordinates

Say I’m at (10,5) and you’re (4,6). To send you a message I’d send it to (9,5), then he’d send it to (8,5), then (7,5), then (6,5), and so on until (4,6)

I don’t connect to you, the message is relayed

And in this case the message is also sent in a lot of different paths at the same time

So it’s faster and no single person or even chunk of people can block everything

Also to enter a network anyone could establish an “entry point”, which is just a client which should always be up and of which the IP is known, so that you can access the network at any time

The entry point in no way controls the network and anyone can create their own

Also at (3,3) you normally have direct neighbor (2,3) (one hop) and also a 2-hop connection to (1,3). If (2,3) disappears, you then add a link to (0,3) (the node behind the 2-hop), so you keep two active connections looking left: (1,3) and (0,3). When (2,3) is filled again, you drop the extra (0,3) link and return to the normal neighborhood.

[trail bridge]

is the question you have possible to be asked again?

[prisma flax]

Simply, would this work? Are there flaws in the concept?

[trail bridge]

i have not scrolled up to read more. whats the TLDR?

[prisma flax]

Creating a fully decentralized network for chatting

[prisma flax]

This is how new users connect

[prisma flax]

This is important

[prisma flax]

And this

[prisma flax]

Here are the main things

[prisma flax]

And this is how you actually text someone

[trail bridge]

how would you transmit the text? tcp?

[prisma flax]

Between you and your connections you would have some sort of connection open, possibly even using aiortc but I’m not sure yet

(So udp)

How would it work with TCP?

[modest brook]

cool

back here

[prisma flax]

The network scaling up wouldnt really be an issue

Everyone will have at most 2x2xdimensionality connections

[modest brook]

wouldn't it be? you would have to figure out routes

ahh, so you're mapping each integral coordinate to a device

[prisma flax]

Say you're at (1026,987) and you wanna send something to (-10, 710)

You would send the message to the two people in that direction

[modest brook]

that would route you throught 1300 connections

[prisma flax]

And everyone who receives the message would do the same

[prisma flax]

Well it would take quite a while for the network to reach that huge a size

And remember this is in 2d

[modest brook]

indeed

why not 3d?

[prisma flax]

But it's probably gonna be more like 10d

[modest brook]

yeahhh

10d sounds more pleasing

[prisma flax]

2d is just for visualizing it better

[prisma flax]

In your mind

[modest brook]

10d means 2^10 connections

[modest brook]

yess, got that part

[prisma flax]

What?

[modest brook]

uh

[prisma flax]

In 10d youd have at most 40 connections

[modest brook]

maybe not

yea

i am not sure how the formula works out for it

each dimension adds 2 more connections

so 10d should have 20 max connections

[prisma flax]

And now Im not gonna do the calculations but I think that if there were 8 Billion people on the 10D network it still wouldnt take long for the furthest people to send a message

[prisma flax]

But you have 2 connections in each direction

Otherwise someone disconnecting would mean cutting off people from the network

[modest brook]

but that's assuming that the nodes in the network are honest and actually forwarding your message

[modest brook]

2 connections in each direction how?

the coordinates can't overlap for two machines

[prisma flax]

In 2d i said you'd send the message to the two people in the right direction

Like if youre at (4,4) and you wanna send to (6,7) you send the message to (4,5) and (5,4)

[modest brook]

ah, so in the direction of the diagonal

[prisma flax]

If youre at (10,10) and (9,10) disconnects youd be cut off from the network otherwise

Instead you also connect to (8,10) and if (9,10) disconnects you also connect to (7,10)

You always have to have 2 connections in each direction

[modest brook]

you could make it so that the network doesn't rely on the coordinate system and does something like graphs

use a* algorithm to find the shortest route to target

and then remember that path for subsequent packet transmission

like how google maps work

[prisma flax]

In 2d there are (x1-x2)+(y1-y2) possible paths

You send it to all of those paths

So no single user can block the message

In 10d its practically impossible for even a well organized group to do that

[modest brook]

hmm

true

but in that case

the network wouldn't handle large amounts of data

since you're essentially routing packets through entirety of the network

[prisma flax]

Imagine the network as a large matrix and two users who want to chat. They are the corners of a rectangle. Only the nodes in the rectangle have to relay that message

Idk if you understand what Im saying

[modest brook]

i got it

[prisma flax]

And in higher dimensionalities (sounds kinda crazy but its true) the fraction that the shape thats interested is of the total diminishes

[modest brook]

but that's still unreliable

[prisma flax]

Why?

[modest brook]

in a cube, you would have 3 possible routes to the end part

ie in n dimensions, you would have n paths to the ending

10 would result in 10

but how do you ensure that all 10 paths are continous

[prisma flax]

Now that I think about there is a possibility that with this simple mechanism the message just can't get to the user

Also just one of the paths needs to be continuous

[modest brook]

yea, true

[prisma flax]

But theres still a possibility that none of them is

[lapis osprey]

I don't understand the coordinate system you are using, does that reflect some kind of ID or does it mean literal location?

[prisma flax]

Imagine a shape like
XXXXXROOOOO
XXXXXOOOOOO
XXXXXXXXXXXS
XXXXXXXXXXXX
Where X is nodes, O is blank, S is sender and R is receiver

[modest brook]

its a hypothetical situation where each node is placed on an integral coordinate grid

[prisma flax]

The message wont get to R because there no path in the rectangle

[modest brook]

in that case

[prisma flax]

But wait

[modest brook]

you could make it so that it retransmits on blockage

[prisma flax]

The O space between R and X actually has a priority to be filled (I didnt say this but part of the concept)

Earlier I said that when you have a join request and many free slots you choose randomly which one to fill

[modest brook]

hmm

[prisma flax]

But I didnt mention that if you have a connection with someone thats not directly adjacent, as R and the X below it do, then X would have a priority to fill that one blank space

[modest brook]

you would have to communicate about that too somehow

[prisma flax]

Which I would have to do some fancy maths but I think makes it functionally impossible for this to happen

[prisma flax]

Not really

You know the coordinates of your connections

And if in a direction there is not one adjacent to you

You know that that's where you should place someone who's joining

[modest brook]

what if you made a tree like structure similar to a modern blockchain

[prisma flax]

But what if someone who has different "branches" coming out of him disconnects?

[modest brook]

indeed that's a problem

[lapis osprey]

the whole grid idea is odd to me

[prisma flax]

Wait you could just do the same thing I did and make everyone connect to two branches below, not just one

[prisma flax]

Why?

[modest brook]

yea but that's still not robust enough

[lapis osprey]

In networking, you often think of hops rather than some grid system

[modest brook]

you need some way to divide the network into more geographically close sections.

[lapis osprey]

You have peers who you are directly connected to, and peers that those peers are connected to (1 hop)

[modest brook]

like how ipv4 does it by dividing networks int A,B,C,D tier

[prisma flax]

Yes, thats what Im proposing, the grid idea is just easy to imagine and grasp (imo)

[modest brook]

mesh!

[lapis osprey]

It's just a simple network

[prisma flax]

But for maximum anonimity that's not very good is it?

[lapis osprey]

In this case, each node is it's own router essentially

[modest brook]

fun fact, your friend's friend's friend's friend's friend's is probably related to a very popular figure.

[prisma flax]

I guess you could have "recommended entry points" like one if you're in europe, in NA, etc, and the way it works people joining from those would just naturally be close and have less lag

[modest brook]

5 hops in the mesh of human relations can get you to pretty much every one on earth

[prisma flax]

Wasnt it 6?

Ive seen a vsauce video lol

[modest brook]

then how do you route on the network

[modest brook]

let the clusters and graphs form naturally

like a bunch of nodes select a leader router node

by vote or staking

[prisma flax]

Alright so that's the most difficult part. You could simply tell someone your coordinates and then they can just message you using those, but I dont love that

[prisma flax]

Centralizing much is it?

[prisma flax]

Yeah that would make them form naturally

[modest brook]

not sure i remember

[lapis osprey]

for something as dynamic as this electing a leader isn't a good idea

if you want to send a message you simply do it through the least amount of hops

[modest brook]

yea.

[lapis osprey]

in the set of paths with the least hops, choose randomly

[modest brook]

modern cryptocurrency blockchains function like that

for a truly decentralized form, you would have to blindly broadcast the packets to all your connections and hope that the mesh gets it to the destination

[prisma flax]

If europeans mostly join from the european recommended entry point then their connection would stay as close as possible to the entry point (simply from how the sharing of the join request is done) and being close means actually having less ping

[modest brook]

but how do you find least amount of hops

you could send out an initial full network wide signal to figure out path of lowest latency

in a recursive fashion

first transponder is the lowest latency

[prisma flax]

No need, if you send it in all directions then the one with the lowest latency will get there first

[modest brook]

entry points is more centralizing isn't it?

[lapis osprey]

in networking, routers have the RIP protocol (ancient) or the OSPF protocol (better), a system in which they will share their routing tables with peers and then run a shortest path algorithm based off the data they have

[prisma flax]

And you dont need to test anything, basically you can just blindly send it in the general direction and it will reach it

[modest brook]

but a lot of network's compute resources would also end up being wasted

[prisma flax]

They are to help people connect, you can just join a friend with their IP directly and be very very close to them

[modest brook]

so have a localised information database?

[lapis osprey]

yes, the routing table

https://bschr.dev/vpt can play with the ideas here

[modest brook]

the moment you use IP, you've defeated the purpose of a decentralized network

[modest brook]

i would love to, i never got to learning how router do their stuff

[prisma flax]

You have to know the IP of your connections.

You can just use a VPN anyway

[modest brook]

you would have to place trust in the VPN provider to not betray you

[lapis osprey]

thats a simulation I made a little while ago showing some neat algorithms and protocols like RSTP and RIP

[modest brook]

how do you form a trustless system

[modest brook]

yuppp, it'd be great

is it on web?

[prisma flax]

If someone really cares about privacy and their IP being secret they'll probably find a way

[modest brook]

cool, found the web versoin

[prisma flax]

You can't really have P2P connections without connecting to a peer can you?

[modest brook]

but that's the point, there is no way for you to broadcast a message without giving atleast some form of indicator of where you are geographically present or by putting trust in a third party to mask your location

[modest brook]

even p2p connections aren't p2p

they're routed through several machines/IPs

[prisma flax]

And what could this protocol possibly do to stop that?

[modest brook]

i am thinking, nothing

[prisma flax]

Also you can create a network by yourself

[lapis osprey]

I think it will be valuable to read up on some current examples:
https://www.sciencedirect.com/science/article/pii/S0160791X20303067

https://www.usenix.org/conference/13th-usenix-security-symposium/tor-second-generation-onion-router

[prisma flax]

Like there would probably be an open, main network but any group can just use their own

[modest brook]

cool

[prisma flax]

You can even create networks of different dimensionalities or anything

Now back on actually finding someone's coordinates

The ideal thing would be to let everyone have a temporary @username but that wouldnt be feasible, so atleast have the coordinates wrapped in something a little nicer than just numbers?

[modest brook]

hmm

naww

i am back

[modest brook]

was just fiddling with the simulator here

[prisma flax]

Oh sure

[modest brook]

hmm, how do we do that

we could use public key as the address of the recipient

and they can then use their private key to decode messages

[prisma flax]

Alright so one thing you could do is make it so that you can just save contacts locally, so if you wanna contact your friend they give you their coordinates and public key and save it with their name

So the first time you do have to save that stuff

But then it should be fine

Which might also mean you have to save eachothers to be able to chat?

Which doesnt sound bad

[modest brook]

i mean, how about let's think of a discovery protocol

[prisma flax]

The mesh we're using here is similar to gossip, and gossip is basically in itself a discovery protocol

[modest brook]

true

are we back on coordinates btw?

[prisma flax]

yes

[modest brook]

alr

[prisma flax]

You know how I talked about the rectangle affected by your messages?

[modest brook]

yess

[prisma flax]

Maybe everyone in the rectangle can discover you

[modest brook]

instead of rectangle, why not do path finding algorithms?

like a*

[prisma flax]

What do you mean by path finding?

The rectangle would just be abunch of lines

Which are all of the same length

And are all the shortest paths to the recipient

[modest brook]

path finding algorithms try to find the shortest path from point a to b with obstacles in between

[modest brook]

technically true but for obstacles around

[prisma flax]

With the obstacles being blank spaces, which makes sense, but as I said before the priority system and the higher dimensionality makes it impossible for there to be NO valid path

[modest brook]

i am starting to think of something

are you familiar with physics?

[prisma flax]

Also if a path finds an obstacle and is blocked it wouldnt be a problem

[modest brook]

makes it less likely yea

[prisma flax]

Like each time you send something you know the direction and if theres an obstacle the "blank space" would just be filled again

wait I think id need to make a short animation to make you understand what I mean (i wont do it)

[prisma flax]

Anyways yes id say

[modest brook]

honestly man

i am just sleepy so its just me being slow

[prisma flax]

Nah Im not really good at expressing some ideas I have so i dont even think you can understand what I mean

Like I just visualize it

[modest brook]

its a 50-50

i myself am not good at explaining ideas

[prisma flax]

So id have to literally draw it to make you see what I see

[modest brook]

i was thinking you could make a rough geographic map of nodes on a network by meassuring latency

[prisma flax]

I thought of it too which would actually be cool because it would automatically mark slow nodes as absent or less important

[modest brook]

and then using springs from physics, forces determined by latencies to essentially draw out a map

[modest brook]

eya

yea

[prisma flax]

This requires some fancy calculations though

[modest brook]

yeaaaa

take inspiration from other algorithms

[prisma flax]

And you have to do it in every direction pretty often to keep the map updated

[modest brook]

ipv4, quicksort, merge sort

[prisma flax]

Isnt that too heavy?

[modest brook]

they're all fast because of divide and conquer

[modest brook]

true

[prisma flax]

Isnt that the british?

[modest brook]

indeed is, but makes for a cool visual

[modest brook]

lol

but i mean

[prisma flax]

Good point

[modest brook]

its very much a great strat

ipv4 works by dividing stuff in a tree like structure afterall

[prisma flax]

I have to go see how Ipv4 works

What about ipv6?

[modest brook]

i haven't looked into ipv6 much at all

but the way ipv4 divides stuff you can get rough geographic location based on it

[prisma flax]

Also I wanted each node to know the least possible about the rest of the mesh

Only knowing what's necessary

[modest brook]

but to have good routing information, you need geographical data

in a system where things are determined by coordinates, the coordinates are acting like that data

[prisma flax]

You also have to remember that the mesh would keep a somewhat round (or whatever its called in higher dimensions) shape

Like any weird shape wouldnt last really long

[modest brook]

infact the network prioritises close friends to join together so essentially doxxing out people and their closeness

[modest brook]

hypersphere

[prisma flax]

Well I mean you both could just join the wrong one to fool any kind of scraping like that

Also you could very well end up in totally opposite sides of the mesh

[modest brook]

yea, but once again the network can monitor the packets you send and at what addresses they go

[prisma flax]

"the network"? Who is the network? No one knows that much

[modest brook]

ngl, i just thought of something

[prisma flax]

When you relay a message you just know "i have to send it topright". thats it

[modest brook]

yea

[prisma flax]

You dont know where it should end up

Only the receiver knows

[modest brook]

in that case how do you make the rectangles work

how does it know when to turn the flow of packet

without knowing the target coordinate

[prisma flax]

If you have to send it topright then you know that you have to send it top and right. Thats it.

Wait I think i know what you mean

[modest brook]

yea

if you broadcast in all top+right directions, you end up with the problem of wasting compute resources

[prisma flax]

Maybe you could tell them how much to go right or top but since you dont need to tell them both they can only guess one of the coordinates

[modest brook]

you would have to tell them both of them

[prisma flax]

Nope

[modest brook]

essentially revealing coordinates

how so

[prisma flax]

To the one on the right of you you tell them "topright, right only 7 times", and the one top of you "topright, top only 3 times"

[modest brook]

ahh that

but how does the recipient know when to recieve a packet

it would be encrypted w their pubkey?

[prisma flax]

When you get a message you check if it was encrypted with your pubkey, if it was its yours and thats it

[modest brook]

i see

[prisma flax]

I think we went over most of the possible issues didnt we?

[modest brook]

not a lot of cybersecurity issues

[prisma flax]

Is there still anything important (or even minor, it can't have any slightly important flaws)

[modest brook]

yes'

[prisma flax]

Please do tell me

[modest brook]

i am not sure how

buy how do you enforce the nodes to form a certain order

[prisma flax]

Like what?

[modest brook]

like ensure that they enter and occupy a certain coordinate

[prisma flax]

Well they get placed by someone there, and that's who they're connected with

They physically cannot "go" anywhere else

[modest brook]

but once they're connected, they can place all subsequent nodes to their liking

[prisma flax]

I thought of that too, and I actually don't know

How does blockchain deal with this?

[modest brook]

blockchain doesn't have a coordinate system

i work in blockchain, they have a lot of globally distributed central/authority nodes

they cross check each other

and the authority is decided by how much crypto currency they have invested

[prisma flax]

I mean, the main thing that comes to mind is: coordinates aren't really occupied: if someone claims (10,4) is occupied then they should tell its supposed connections to connect to them

[modest brook]

if wrongdoing is found, the invested cryptocurrency is burnt

its all financial incentive

[prisma flax]

If that doesnt happen then the supposed connections will just not mark it as occupied and fill it whenever they can

[prisma flax]

Not for bitcoin

[modest brook]

bitcoin is different

[prisma flax]

Wait wait let's get back to the problem not blockchain

[modest brook]

it works by the fact that you would need to be able to insert more than just a few blocks to bbe able to maliciously inject your motive

cool

back to problem

[prisma flax]

This

[prisma flax]

And this

Imagine I'm at 9,4 and the guy at 10,3 wants to act like 10,4 is occupied

Usually when someone connects you'd tell all of his neighbours to connect to him

[modest brook]

are connections allowed in diagonal manner

[prisma flax]

But if I at 9,4 dont get that connection from 10,3 then Ill just think 10,4 is not occupied

[prisma flax]

Nope

I mean they could but I don't see why

[prisma flax]

And if I think it's not occupied I'll fill it with a real connection as soon as I can

[modest brook]

wait

instead of graphs

[prisma flax]

And if all the people around the node agree to fake its existence (which gets way harder in higher dimensions)

[modest brook]

and fixed spaces

why not work in a tree like structure anyways

since the entire thing is pretty much tree like

[prisma flax]

Give me an example of how it would work

And how it would be decentralized

[modest brook]

i am thinking about it

[prisma flax]

sure

[modest brook]

the problem is how do you define addresses

[prisma flax]

Well let's say the start of the tree is 0

[modest brook]

how can you think of an address in such a way that it contains information as to where you need to send the packet to

[prisma flax]

Above it there is 1/0 and 1/1 maybe?

Then 2/0, 2/1, 2/2, 2/3

[modest brook]

not exactly tree like then

more like a graph

[prisma flax]

No

[modest brook]

a meshed graph

[prisma flax]

2/0 and 2/1 are connected to 1/0

2/2 and 2/3 are connected to 1/1

[modest brook]

yea that's a tree

what i mean

is if we just create a meshed graph

not a fully dense mesh, but a mesh none the less

[prisma flax]

Whats the difference from my original idea?

[modest brook]

no clue, i think we're nearly a round circle near to reinventing the internet

[prisma flax]

Yeah I think we might

[modest brook]

and the internet is not decentralized either

[prisma flax]

I mean I don't wanna be stubborn but I just don't see what's wrong with my original idea

[modest brook]

the only way to have a truly anonymous platform is if every node transmits into every direction

[modest brook]

i feel its doable, but robustness isn't there

[prisma flax]

In what is it lacking robustness?

[modest brook]

like, you could come up with very creative ways to sabotage the network

[prisma flax]

I always dreamed of having a team of white hat hackers reporting to me like "we were able to sabotage all communication with this method" and then I go fix it

[modest brook]

the initial problem statement of finding a permanent address to a node

and then establishing a route to the said node

[prisma flax]

Wait like how?

[modest brook]

haha, i dreamt of learning ethical hacking once, didn't go much deep into it

[prisma flax]

I feel like I could do it but I prefer creating stuff directly

[modest brook]

that's the million dollar question, how do you find the geometrical address from permanent addreess (pubkey)

[modest brook]

yea, that's cool

[prisma flax]

So the problem is knowing the pubkey and the coordinates linked in some way

[modest brook]

yea

[prisma flax]

Now we could just make it so that the users have to share them directly but thats not very good

[modest brook]

yess

and since its a decentralized network, can't rely on routers and switches

[prisma flax]

Maybe we could make it so that the username+coordinates somehow result in the pubkey

So youd have to know the coordinates

But also a username chosen by the person

[modest brook]

that's not doable

[prisma flax]

And maybe this can be worked on more

[prisma flax]

Why not?

[modest brook]

i am not sure