#networks

1 messages · Page 37 of 1

ember ledge
#

I used a urllib web scrapper to take the html from google.com and I made it where it puts the html code in a Flask server. It works but it doesn't display 100% of the page since it is only taking HTML from that page, not from other pages, so images and links don't 100% work

frigid escarp
#

Hi ust wondering if anyone has used tcpdump before? Thank you

#

I need to filter out data packets on their source and port and then save this to a new file and run md5ssum

#

I am trying the command: tcpdump -tttt -ni -igb1 port 80 src host [ip address] -r [filename] but I am getting syntax errors!!

#

Thank you very much!

tidal garnet
#

Hi! Does anyone have any expreience with aiohttp's websocket library?

#

Basically my websocket handler doesn't get notified if a conneciton is lost (not properly closed). And if I turn on heartbeat it will throw a Cannot write to closing transport error, so it failes to send the next ping message and not even notifing that it didn't got a pong response

wind oriole
#

Quiet in here so I'll throw this in... I'm building a very respectful crawler as a learning exercise, not for full deployment. It currently stores both the html and the images it finds. It crawls the html with requests, extremely quickly finds and cleans image urls, and then aiohttp switches up headers and fetches the image files with 5 semaphores. Even faster than in a browser. As I see it, this is even less load than me visiting the page, because no css or js are being loaded. Like I said, I want to be respectful of peoples sites, can anyone tell me if that assumption is correct?

#

Or if I'm going to end up setting off bad status codes and upsetting site owners

hasty quest
#

But that doesn't matter much

#

The main question isn't what content you load, but how quickly

#

When you visit a page in browser, you might fire off 20 requests at once to load everything on it. So 20 requests ok to send at once for scraping?

#

Nope

#

Because when you visit a page in browser, you usually stay on it for a bit.

#

You don't make 20 requests and then switch to a different page 0.3 seconds later. You spend a while on it

wind oriole
#

oh for sure, I never visit the same domain more than once every 8 seconds

hasty quest
#

that sounds fine then. 1 request per host per second is considered a pretty standard respectful spidering rate.

wind oriole
#

and 5 semaphores is fewer concurrent connections than most web browsers

#

i figured since it's within a session, it shouldn't be an issue. it's not like I'm opening 50 sessions per visit

hasty quest
#

if you're sending only one request per host per 8 seconds, then your load is definitely negligible

#

👍

wind oriole
#

well no, I visit AND grab images once every 8

hasty quest
#

How many images?

wind oriole
#

however many the browser would load

#

whatevers in the html

hasty quest
#

if that's 8000 then you'll have problems doing it every 8 seconds.

wind oriole
#

i time from crawl completion

hasty quest
#

A better approach would be to put all the image sources + A refs + etc you find for one host into a queue and only pull targets from the queue at a specific speed.

#

As is, it could be fine, or very not fine. Depends on the site

#

Nothing, pretty much. It all boils down to automatically fetching stuff

#

But there are a lot of reasons you might do that

wind oriole
#

well there's a big project going on right now to scrape all the images from commoncrawl and pair them with their attributes to make the largest annotated image dataset in the world

#

or how the Internet Archive does it

hasty quest
#

I scraped a webcomic site so I could have an archive in case the site goes away

wind oriole
#

building crawlers becomes an addiction

hasty quest
#

The most recent strip was posted in 2004 lol

#

i think it's probably just running on a rack and the owner may not even remember it still exists...

wind oriole
#

there's a discord server for archiving youtube too

hasty quest
#

I hope they're stored on tape nice

#

Fun fact, tape is still legitimately the best way to archive stuff. The random access is awful, but sequential access is probably better than most top grade hard drives.

#

And the density is ridiculous

#

You can get like 100tib for $100 I think. Main problem is that the reading and writing equipment is expensive.

#

Good question nice

#

Google's answer is pretty much "Uhhh, because."

#

nah that package exists. It's just in violation of Google's terms of use.

#

Ofc it is

#

Just because it's against their terms of use to use it doesn't mean they have the authority to wipe its existence from the internet.

#

They can deny you the use of their service if they catch you

#

In practice all that happens is the video won't load if they think you're using it.

#

that's why I mean by "deny use of the service"

#

They might have the legal right to, but I can't see them doing it without it being a complete shitshow.

#

If they were willing to do that, you'd definitely have heard of it.

#

Like can you imagine the shitshow that would be?

#

That's why you "need" youtube premium

#

Downloading stuff via that isn't a TOU breach.

#

yup

wind oriole
#

terabytes of youtube 😄

wind oriole
#

all national us news media video coverage of covid since it started

#

plus webinars, press conferences, roundtables, media briefings, speeches, etc

#

and a lot of hockey 😊

eager mango
#

What will these headers do? Will this completely load the source code returned in the response?

#

My webdriver using proxy that is my local socket server```py
from selenium import webdriver
from selenium.webdriver.common.proxy import Proxy, ProxyType

proxy_ip_port = 'localhost:12345'

proxy = Proxy()
proxy.proxy_type = ProxyType.MANUAL
proxy.http_proxy = proxy_ip_port
proxy.ssl_proxy = proxy_ip_port

capabilities = webdriver.DesiredCapabilities.CHROME
proxy.add_to_capabilities(capabilities)

driver = webdriver.Chrome(desired_capabilities=capabilities)

driver.get('https://www.google.com')```

#

My socket server```py
import socket
import requests

s = socket.socket()
print ("Socket successfully created")

port = 12345

s.bind(('', port))
print ("socket binded to %s" %(port))

s.listen(5)
print ("socket is listening")

while True:
c, addr = s.accept()
print ('Got connection from', addr )
precious_data = c.recv(4026).decode()
print(precious_data)

r = requests.get("https://www.google.com")

c.send(r.text.encode())```
iron fjord
#

would it be possible to setup a WAN server that can't be traced back to me?

#

like i wouldn't use my actual IP

hasty quest
#

Can't by who?

#

Your next door neighbor? The NSA? God?

#

Sure, I'm pretty much only interested in hearing their threat model.

iron fjord
#

i mean is there a way to setup a server at home without giving my IP address for people to connect to, lets say, my website? This seems like a really dumb question

#

and it is right?

hasty quest
#

Not a dumb question, but a very different question from "Can't be traced back to me"

iron fjord
#

haha yes, that probably seemed very suspicious

hasty quest
#

My answer to "Can I set up X without it being traced back to me" would be "If you have to ask, then no."

#

But what you're asking is quite realistic, I'll leave that up to NS though

iron fjord
#

ye, that always fascinated me: the idea of TOR.

hasty quest
#

If all you want is to not expose your home network to random people visiting your page, both of those solutions are overkill imo

#

exactly

#

If for some reason you do need TOR, you have to know how TOR works. Nothing is less secure than a false sense of security.

#

Fantastic tool which I love

#

Just ask Jeremy Hammond

iron fjord
#

if I'm not mistaken, TOR works by bouncing your packet around multiple servers around the world right?

hasty quest
#

, along with a VPN to encrypt data that won't be encrypted.
No, no no. Please no. You've just wrecked your security.

#

It's worse than that.

#

The model of TOR is based around the distribution of trust.

#

Trusting your VPN provider isn't good enough, because "attacking" (from the perspective of deanonymization) a VPN provider is easier than attacking TOR.

#

You use TOR because you can't trust single authorities.

#

If you're willing to place trust in a single authority at all, even if you think they are trustworthy, you don't need TOR.

#

And if you do need TOR, you don't place trust in a single authority.

iron fjord
#

I always thought of TOR when i was like 7 as one of the scariest things alive

#

👏

hasty quest
#

Me too, like facebook.

#

facebookwkhpilnemxj7asaniu7vnjjbiltxjqhye3mhbshg7kx5tfyd.onion

iron fjord
#

does facebook actually have an onion route?

hasty quest
#

It's... complicated

#

You can host a hidden service without using an onion address

#

Onion addresses are just a way to provide location of services similar to DNS

#

Well, it's more complicated than that.

#

Anyways

iron fjord
#

feels good to talk to people who don't ridicule me for asking a question

hasty quest
#

An onion address is derived from the public key of the service itself. That's why they look the way they do, so that they can be self-verifying.

iron fjord
#

whats the worst someone could do with your ip? DDOS and reverse SSH?

hasty quest
#

When you connect to a hidden service via its onion address, you can be sure that the party you're connected to is actually the maintainer of that hidden service - and not someone else pretending to be them.

iron fjord
#

ah

hasty quest
#

The gist of how an onion address works is that the hidden service picks rendezvous points within the TOR network and publishes those points encrypted with the private key of the service.

#

A user then looks up the published records and makes a connection to one of the rendezvous points.

#

They establish a TOR circuit through that point. And the client can verify the integrity of the connection by making the hidden service prove that it owns the private key of the service.

#

The client ofc knows the public key of the service, since it's public. And since the onion address is derived from the publickey, there's no need for an authority to verify the integrity of the public key like with TLS.

iron fjord
#

how could you be de-anonymized with TOR?

hasty quest
#

A lot lot lot of ways

iron fjord
#

oh yeah, ofc

#

haha

hasty quest
#

Millions of people around the world use Tor every day to protect themselves from surveillance and censorship. While most people use Tor to reach ordinary websites more safely, a tiny fraction of Tor traffic makes up what overhyped journalists like to call the "dark web". Tor onion services (formerly known as Tor hidden services) let people run I...

▶ Play video
#

^ Talk given by one of the TOR project maintainers

#

Very worth a listen if you want to understand the tech more

iron fjord
#

same here

#

i thought about downloading Tails OS but i spend 99% of my time on kali

#

ye, i tried setting that up

hasty quest
iron fjord
#

ye ik, i don't do much on my pc except code. I'm learning ethical hacking to see if its a career path i would like to go down or stick to Software Engineering. I mainly use PopOS for day to day use

#

i always wanted to learn how to "hack" ever since i was like 7

#

i originally started to show off to friends but then i realized, i really don't have any and thats a stupid reason to learn

#

roblox?

#

thats also how i started

#

didn't understand it and went onto unity

#

i was always top of my class with computers but my parents couldn't really afford a decent one so it took awhile for me to actually enjoy using a computer.

#

i learned how to make a couple of things but i didn't go far and very soon decided to learn python as it was required for my Computer Science course for GCSE at my school

#

I've only done python, C# and JS

#

ngl i was trying to see if it was possible for my to hide my listener IP in a payload so it can't be traced back to me

#

or more of a backdoor

#

I'm not a malicious person but i thought if i asked this it would seem like i was trying to be unethical

#

ok, amazing

wind narwhal
ember ledge
#
class shush:

    def delete(channel, message):
        r = requests.put(url=f"https://discord.com/api/v9/channels/{channel}/messages/{message}", headers=headers)
        if r.status_code == 200 or r.status_code == 201 or r.status_code == 204:
            print(f"{Fore.BLUE}>{Fore.RESET} Deleted {message}{Fore.RESET}")



@client.command()
async def massd(ctx):
    await ctx.message.delete()
    ids = open("ids.txt","r").read().splitlines()
    for message in ids:
        del = functools.partial(shush.delete, ctx.channel.id, message)
        threading.Thread(target=del).start()

im trying to make my bot delete messages with requests but it doesnt work and i get no errors any ideas?

wind oriole
#

you getting an error?

ember ledge
#

nope

ember ledge
#

Guys so ssh is more secure then telnet and serial, right?

light zealot
#

iirc yes

crimson peak
#

who uses pycharm?

ember ledge
#

your mum

crystal current
hasty quest
hoary sorrel
#

fail2ban only goes so far in the age of botnets, when people can bring a virtually unlimited number of IP addresses to bear on the brute forcing

crystal current
#

Yeah, fail2ban can help but I really wouldn't want to rely on it

reef sierra
#

Is this the right place to ask socket programming questions?

reef sierra
#

I am wondering if anyone knows how to make a 16 signed integer in python?

#

I tried googling it and my best guess is that struct.pack might be able to do it.

#

Since struct.unpack can convert those to normal integers

#

But I don’t know what format to feed it to make it work and I could find anything on the subject by searching google or stack overflow.

reef sierra
#

After a bit of working on it I figured out by close reading of the struct official docs.

hasty quest
reef sierra
#

Yeah

#

Although the first page there I looked in didn’t have much of use.

#

But I managed to find another page in its internal search that wasn’t listed on google that had it

tidal skiff
#

hello ! Does someone know a library to make udp client server for streaming with a packet loss management ?

#

webcam streaming*

haughty panther
#

Hi, do you guys know how to find hidden API’s with Fiddler Everywhere? Not sure if this is the right channel to ask

prisma cobalt
slate briar
#

How do I find out if I have an eavesropper on my internet provider?

slate briar
#

Endless paranoia

#

I don’t feel safe online

#

Big brother vibes

prisma cobalt
#

its free to use and not illegal (like some people think)

#

just expect slower loading times

slate briar
prisma cobalt
#

if your really paranoid you can use a VPN and then TOR

slate briar
# prisma cobalt yes, they will see your **using** TOR, but not what your using TOR for, and it s...

A research study by the Open Observatory of Network Interference (OONI) and the Association for Freedom of Thought and Expression (AFTE). To download a full copy of the report in PDF format click here Egypt-Internet-Censorship-AFTE-OONI-2018-07 Table of contents Key Findings Introduction Background Network landscape and internet penetration Lega...

austere gale
#

Hi guys can someone help me out with smtplib. I’m stuck since 4+ hours now

#

Sending email through smtplib works fine on my local machine but when my flask code is deployed on cPanel server it doesn’t work and gives error saying address family not supported by protocol

lavish umbra
#

yo

prisma cobalt
#

ello

lavish umbra
#

so i got this

#

the tcp is my python script i believe

prisma cobalt
#

can i see your code please

lavish umbra
#

import socket
import threading

HEADER = 64
PORT = 6050
SERVER = socket.gethostbyname(socket.gethostname())
ADDR = (SERVER, PORT)
FORMAT = 'utf-8'
DISCONNECT_MESSAGE = "!DISCONNECT"

server = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
server.bind(ADDR)

def handle_client(conn, addr):
print(f"[NEW CONNECTION] {addr} connected.")

connected = True
while connected:
    msg_length = conn.recv(HEADER).decode(FORMAT)
    if msg_length:
        msg_length = int(msg_length)
        msg = conn.recv(msg_length).decode(FORMAT)
        if msg == DISCONNECT_MESSAGE:
            connected = False

        print(f"[{addr}] {msg}")
        conn.send("Msg received".encode(FORMAT))

conn.close()

def start():
server.listen()
print(f"[LISTENING] Server is listening on {SERVER}")
while True:
conn, addr = server.accept()
thread = threading.Thread(target=handle_client, args=(conn, addr))
thread.start()
print(f"[ACTIVE CONNECTIONS] {threading.activeCount() - 1}")

print("[STARTING] server is starting...")
start()

#

import socket
import threading

HEADER = 64
PORT = 6050
SERVER = socket.gethostbyname(socket.gethostname())
ADDR = (SERVER, PORT)
FORMAT = 'utf-8'
DISCONNECT_MESSAGE = "!DISCONNECT"

server = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
server.bind(ADDR)

def handle_client(conn, addr):
print(f"[NEW CONNECTION] {addr} connected.")

connected = True
while connected:
    msg_length = conn.recv(HEADER).decode(FORMAT)
    if msg_length:
        msg_length = int(msg_length)
        msg = conn.recv(msg_length).decode(FORMAT)
        if msg == DISCONNECT_MESSAGE:
            connected = False

        print(f"[{addr}] {msg}")
        conn.send("Msg received".encode(FORMAT))

conn.close()

def start():
server.listen()
print(f"[LISTENING] Server is listening on {SERVER}")
while True:
conn, addr = server.accept()
thread = threading.Thread(target=handle_client, args=(conn, addr))
thread.start()
print(f"[ACTIVE CONNECTIONS] {threading.activeCount() - 1}")

print("[STARTING] server is starting...")
start()

prisma cobalt
#

so your using port 6050?

lavish umbra
#

yh my bad

#

6050 has none

prisma cobalt
#

youve pasted the same code twice

lavish umbra
#

ohhh sorry

#

`import socket

HEADER = 64
PORT = 6050
FORMAT = 'utf-8'
DISCONNECT_MESSAGE = "!DISCONNECT"
SERVER = socket.gethostbyname(socket.gethostname())
ADDR = (SERVER, PORT)

client = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
client.connect(ADDR)

def send(msg):
message = msg.encode(FORMAT)
msg_length = len(message)
send_length = str(msg_length).encode(FORMAT)
send_length += b' ' * (HEADER - len(send_length))
client.send(send_length)
client.send(message)
print(client.recv(2048).decode(FORMAT))

send("Hello World!")
input()
send("Hello Everyone!")
input()
send("Hello Tim!")

send(DISCONNECT_MESSAGE)`

#

btw this is copied from a video as i thought i would get working code

#

to make sure its def my pc

#

and not my code

#

this is after i run the server so thats the python script

prisma cobalt
#

your connecting to your own address so just use "localhost" as the address

lavish umbra
#

how do i do that

#

replace SERVER?

prisma cobalt
#

ADDR = ("localhost", PORT)

lavish umbra
#

yooo thx so much

#

that worked

#

😄

#

just a quick question so how would this work if the client was on another machine

prisma cobalt
lavish umbra
#

outside the network

#

im trying to make a peer to peer network

#

might be a bit complicated cos ive never done networking b4

prisma cobalt
#

you need to implement a port forwarding rule

#

also assuming you have a dynamic ip, you would need to change the server ip in every client script every so often or use noip or suchlike

lavish umbra
#

that sounds pretty complicated

#

ill look more into it but thx alot for the help

prisma cobalt
#

easier said then done, his ISP might not support IPv6 and implementing it currently for that size of a project is a waste of time

#

thats... dangerous, what he should do is implement a port forwarding rule to direct traffic from outside his network to his server

lavish umbra
prisma cobalt
left trellis
#

I wanna ask, i accidentally use pkill /? on my vps. i forgot in linux using man to get help.
I already restart my vps and the ssh service is up. but i still cant access my vps.

#

from the dashboard i guess. well, this vps is gift from my friend. and he told me the ssh is alive on server.

#

well, i want to help him figure it out since it my fault it happend

#

ok. i'll try

unborn dock
#

Hi, I want to be able to view the output of a command over the web.
Similar to being able to view the output of a command from github actions.

How would I achieve this? I tried websockets, but it seems it just doesn't want to work in browsers anymore

mint plank
#

how do i send a request with headers AND form data?

hasty quest
#

Afaik you can't send form data without headers in recent versions of HTTP.

#

the Host header is a required header now.

mint plank
#

ya i need both

hasty quest
#

Right. So you just seperate them with the normal HTTP separator.

mint plank
#

i dont think i ever used that

#

do u have any examples?

hasty quest
#
POST / HTTP/1.1
Host: localhost:9001
User-Agent: curl/7.79.0
Accept: */*
Content-Length: 7
Content-Type: application/x-www-form-urlencoded

foo=bar```
#

The real question though is why you're hand rolling http at all

#

just use requests

#

or urllib

#

HTTP is a complicated protocol. Let a library handle it.

mint plank
#

im trying it in requests

hasty quest
#

Then just pass headers and data

mint plank
#

where data is form data?

hasty quest
#

Yea

mint plank
#

ok ill try that

hasty quest
#

The data param is for request body data, the headers param is for headers

mint plank
#

ok ill try that

left trellis
#

So, here what i got.
Sorry for very late response. i just realize to do it on cmd. i try on putty before but it doesnt work.

ember ledge
#

Delete - p 22

#

And also your server îs not up and running

#

Tell your friend to restart it again

left trellis
#

my friend already restart it. here ss from the dashboard server

#

Note: this is happen after i use pkill /? on ssh. i though it was to get command help

#

probably there's some service that i need to turn on again?

quasi juniper
#

does anyone know a good ansible discord?

crystal current
real snow
#

Hello everyone, I would like to ask you a question as I am struggling to complete a uni exercise, I would appreciate your help.
I have created a server-side and client-side that are connected with each other. What are the next steps to create a new file that is able to monitor the traffic between server and clients? Also, consider that may help if we can include a mechanism that recognizes if the client has connected 2nd time

prisma cobalt
#

!rule 5

errant bayBOT
#

5. Do not provide or request help on projects that may break laws, breach terms of services, or are malicious or inappropriate.

prisma cobalt
#

monitoring a network without permission is most likely not okay with your local laws/TOS

#

if its your own project then you can implement it server side

real snow
#

included in the main server file ?

#

so that can keep records and share it with others as well

prisma cobalt
#

yeah since your server is shared between all your clients, its reasonable to implement logging and stuff inside the server since it recvieves and sends all the data in your project

real snow
#

indeed, and if I would like to create a mechanism that recognize the same client for the second time after his disconnection , would be easier to store lets say a nickname into array and then look up if the nickname exists

#

what's your thought ?

iron fjord
#

who controls the tor proxies?

hasty quest
#

TOR is a system based on distribution of trust instead of authorities, so the more random people running relays, the more secure it is.

iron fjord
hasty quest
#

maybe? idk

#

vaguely remember discussing cryptography but not anything since

iron fjord
#

i came up with the idea to encrypt send messages with the servers public key then relaying it to everyone else with their public key

#

if i made sense

#

is this secure and are there any flaws you can see?

hasty quest
#

sounds fine

#

mostly anyways

#

In that system, the server has everyone's public keys and relays messages encrypted with everyone's public key

#

that's secure if the server is secure, but it's not e2e and the server sees the message or could modify it

#

If everyone has everyone else's public key, then you can encrypt with everyone else's public key and have the server relay each of those to each person

#

Either way, there is one thing you should add regardless:

#

when sending a message, a user should encrypt that message with their private key

#

The receiver decrypts it with the sender's public key

#

This ensures the authenticity of the sender.

iron fjord
#

who is the reciever in this case? the server or other clients connected?

hasty quest
#

the other clients

#

the server can be a receiver too

#

but the clients shouldn't trust the server to relay the message accurately

#

therefore encrypt with sender privkey

iron fjord
#

ok so, person A wants to send a message ("hello") to person B and person C. "hello" is encrypted with person A's private key and then sent to the server which relays it to person B and person C which they decrypt with person A's public key?

#

this is done to make sure the message is from person A and not an intruder or something else.

hasty quest
#

yea

iron fjord
#

@hasty quest did you study cryptography?

#

but isn't this unsafe due to the public key being... public?

hasty quest
#

Why would it be unsafe?

#

the whole point of the public key is that it is public. It decrypts any data encrypted with the private key, and vice versa

#

The sender's public key verifies the authenticity of the sender, and the receiver's public key verifies the authenticity of the receiver.

#

Ideally you use both.

#

Sender encrypts with their private key, then encrypts with the receiver's public key. Sends to receiver

#

Receiver decrypts with their private key and then decrypts with the sender's public key

iron fjord
#

I'm sorry if I'm frustrating you with what I'm sure are simple questions

hasty quest
#

Public is public

#

You can send the public key to any number of people. that's what makes it the public key.

#

The private key you send to nobody, including the server. Because it's private.

iron fjord
#

alright I'm fairly certain i got this now, can you do a public and private key with AES?

hasty quest
#

nope

#

AES is symmetric encryption

iron fjord
#

is RSA the best option?

hasty quest
iron fjord
#

thanks

hasty quest
#

Unless you have a reason to choose the algorithms yourself, just use the pynacl defaults

iron fjord
#

alright

iron fjord
#

@hasty quest I'm confused on how my origional idea wasn't e2e and wouldn't it be easy to decrypt messages since your using public keys to decrypt. I thought the whole idea was public was to encrypt and private was to decrypt.

hasty quest
#

The public key only verifies the authenticity of the sender

#

You still need to decrypt with the recipient private key to prevent decryption in transit.

#

That's why I keep saying to use both the sender privkey and the recipient pubkey.

#

i explained this already.

#

Sender encrypts with their private key, then encrypts with the receiver's public key. Sends to receiver
Receiver decrypts with their private key and then decrypts with the sender's public key

#

having the recipient decrypt with their private key prevents encryption in transit, but doesn't authenticate the sender, since everyone has access to the recipient public key.

#

having the recipient decrypt with the sender's public key authenticates the sender, but doesn't prevent encryption in transit since everyone has access to the sender public key.

#

Combine the two, and you have a protocol which both authenticates the sender and prevents encryption in transit

iron fjord
#

alright thanks, i also got where i went wrong with the sender encrypting with private key stuff.

#

I appreciate all your help with my simple questions

hasty quest
#

👍

undone cloak
#

Is router port forwarding required to enable a local socket server to work with ngrok’s tcp connection ?

ashen copper
#

local socket
What does your router have to do with a local socket? It's local.

undone cloak
frozen iris
#

I am implementing the WebSocket protocol using WSProto and I just want to verify, after sending a closing message then receiving one.. is it safe to close the TCP connection?

#
Initiate closing frame ->
            <- Send closing message
Reply and respond with close ->
            <- ?????
*close TCP connection?*
cedar forum
hasty quest
#

ye, it's a portable NAT punchthrough

#

Wouldn't be much of a NAT punchthrough if it didn't punch through NAT.

undone cloak
undone cloak
cedar forum
#

ngrok doesn't do TCP

#

or, well, it does, but you have to tell it to

#

what ngrok command are you running?

undone cloak
#

ngrok tcp XXXX whats that command then

#

im pinging the url they give me, use it with the given port at client side. and then "127.0.0.1" with the port specified in the command on server side

cedar forum
#

and you signed up?

undone cloak
#

yes

#

wouldnt get the auth token otherwise

cedar forum
#

working here

#

just proxied postgres through it

#

oh wait, 127.0.0.1 with the port they specified?

#

you need to use the hostname and port they specify

undone cloak
#

Client side:
host = "6.tcp.ngrok.io"
port = Their port
Server side:
server_ip = "127.0.0.1"
port = 5050 (my port)

#

thats my setup

cedar forum
#

right, and you don't get any connection?

#

do you see a new conn in the ngrok interface?

undone cloak
#

Connections ttl opn rt1 rt5 p50 p90 0 0 0.00 0.00 0.00 0.00

cedar forum
#

hmmm seems to be client side then

#

what's your client code, if you connect with netcat do you get through?

undone cloak
#

the weird thing is,
Client side:

s.connect((host,port))
print("Connected To The Server")

the print gets executed

#

Server side:

s.listen()
conn, addr = s.accept()
print("Connected")

the print doesnt get executed

undone cloak
cedar forum
#

hmmmm

undone cloak
#
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
host = "3.140.223.7"
port = 12949
s.connect((host,port))
print("Connected To The Server")


class MyThd1(Thread):
    def run(self):
        while True:
            s_msg = input("Enter Your Msg : ")
            s_msg = s_msg.encode()
            s.send(s_msg)
            print("sent successfully")

class MyThd2(Thread):
    def run(self):
        while True:
            r_msg = s.recv(1024)
            r_msg = r_msg.decode()
            print("Received Msg : " + r_msg)

t1 = MyThd1()
t2 = MyThd2()
t1.start()
t2.start()
undone cloak
#

can you do that from ur end ? the netcat thing

cedar forum
#

if you've got nc on your system it's just nc 6.tcp.ngrok.io [port] and see if the conn goes through

undone cloak
#

is netcat preinstalled on kali ?

#

if that, ill launch the vm

#

i actually believe the problem is not with the server or client, but in the local host server. idk do firewalls affect it ?

undone cloak
cedar forum
#

yeah someone has proxied an ssh conn then

#

so that means it's probably on your server

undone cloak
#

im not really into networking

#

any fixes ?

#

root@kali:~# netcat 0.tcp.ngrok.io 15780
root@kali:~#
does this mean the connection failed to ngrok ?

#

or to my localhost

ember ledge
mint plank
#

using requests i get this responce back from the site:

{"code": 50035, "errors": {"avatar": {"_errors": [{"code": "AVATAR_RATE_LIMIT", "message": "You are changing your avatar too fast. Try again later."}]}}, "message": "Invalid Form Body"}```and i have this to take out to just the part i want: ```py
  data = r.json()
  message = data['message']
  print(message)
``` and it sort of works. it gets the one that says invalid form body. but i want the one that tells me it changes my avatar/profile too quick. how do i get that one instead?
ember ledge
#

i think you have to adjust your indexes

#

data["errors"]["avatar"]["_errors"][0]["message"]

ember ledge
#

https://github.com/mcha-me (my github and website you can find some useful stuff on java, javascript, html and css)

halcyon oasis
#

Hey guys, m new to this field of python, can someone recommend me what module I should learn for networking?

hasty quest
ember ledge
#

start with high level, the library you want is requests, or in case by some chance you are required to write asyncronous code then aiohttp is what you need

#

after that move onto lower level, like socket to understand networking "closer to the wire"

halcyon oasis
#

@ember ledge @hasty quest thank u both of u🥰

rigid breach
#

anyone ever see this kind of stack trace when running uvicorn?

h11._util.RemoteProtocolError: illegal request line
WARNING:  Invalid HTTP request received.
Traceback (most recent call last):
  File "/Users/foobar/.pyenv/versions/3.8.5/Python.framework/Versions/3.8/lib/python3.8/site-packages/uvicorn/protocols/http/h11_impl.py", line 136, in handle_events
    event = self.conn.next_event()
  File "/Users/foobar/.pyenv/versions/3.8.5/Python.framework/Versions/3.8/lib/python3.8/site-packages/h11/_connection.py", line 443, in next_event
    exc._reraise_as_remote_protocol_error()
  File "/Users/foobar/.pyenv/versions/3.8.5/Python.framework/Versions/3.8/lib/python3.8/site-packages/h11/_util.py", line 76, in _reraise_as_remote_protocol_error
    raise self
  File "/Users/foobar/.pyenv/versions/3.8.5/Python.framework/Versions/3.8/lib/python3.8/site-packages/h11/_connection.py", line 425, in next_event
    event = self._extract_next_receive_event()
  File "/Users/foobar/.pyenv/versions/3.8.5/Python.framework/Versions/3.8/lib/python3.8/site-packages/h11/_connection.py", line 367, in _extract_next_receive_event
    event = self._reader(self._receive_buffer)
  File "/Users/foobar/.pyenv/versions/3.8.5/Python.framework/Versions/3.8/lib/python3.8/site-packages/h11/_readers.py", line 68, in maybe_read_from_IDLE_client
    raise LocalProtocolError("illegal request line")
h11._util.RemoteProtocolError: illegal request line
hasty quest
#

First line is the request line:

GET / HTTP/1.1
Host: localhost:9090
User-Agent: curl/7.79.0
Accept: */*

#

the GET / HTTP/1.1

#

dump it out and see what it is.

#

if it's coming from a random public client, ignore it

#

if it's coming from your own code, then whatever you're using to issue the request is at fault

grave cloak
#

does anyone know if its possible to block a website via etc/hosts or something

#

but only the website, not entire domain

#
GitHub

The Python programming language. Contribute to python/cpython development by creating an account on GitHub.

GitHub

Repositories related to the Python Programming language - Python

GitHub

The Python programming language. Contribute to python/cpython development by creating an account on GitHub.

severe ridge
#

hey can someone answer a couple questions so my family has been having issues with wifi constantly and i am wondering if hosting a proxy server off of a pc would help with that a little? our wifi is slow and always cutting out

#

nvm figured out it won’t

#

anybody got solutions tho?

wind oriole
#

When simulating a browser visit to a webpage, since browsers use ~6 simultaneous connections, would it be more correct to use 6 async requests in one session or 6 threads?

foggy quiver
wind oriole
#

ok, trying to make sure I don't upset any site owners

#

visiting each page with a healthy pause in between but downloading all resources referenced in the html once i do

#

appreciate the feedback

next pollen
#

hi anyone feeling super lonely post-college grad? I'm a data scientist in industry that just graduated college about a year ago and I want to create an online community where young data scientists can collaborate and learn from one another. Similar to a college data science club or a really engaged discussion group that you would have in a class. the pandemic has made things so lonely.... Some things we could do for example are like a fast-ai study session (I really cannot get myself to learn deep learning...). Ping me and let's network and set up something amazing!

steady sage
#

How can i capture ip of source and destination on eth or lan using python.
Any help appreciated 🙂

#

And put capture ip in a table*

placid solstice
plucky junco
#

hi

#

i need help in improving a simple game that I made related to socket programming.

cedar forum
#

because that's not how /etc/hosts works

#

/etc/hosts is a local override of DNS lookups, when you browse to github.com/python/cpython then you only actually lookup github.com to get the location of github, once you've got that you ask the github server for /python/cpython

#

you can only override that initial bit with /etc/hosts, you can redirect requests to all github.com pages elsewhere, but not outright blacklist pages

#

for that sort of thing, you'd need a proxy/firewall which is capable of seeing inside TLS traffic, there are several solutions that do that

#

but you'd need to install certificates on the machine and run a firewall, so it's not as cheap as /etc/hosts

#

you might also find a browser extension that can do it if for whatever reason you are only trying to blacklist sites on your own machine

iron fjord
#

what's a good or standard way to recv data one after another? I keep getting the issue that half the data is recieved in the last sock.recv() call

#

if I'm making sense

#

I only want data 123 and 456 to go separately so i can do something like ```py
username = socket.recv(BUFFER_SIZE)
something_else = sokcket.recv(BUFFER_SIZE)

#

but my issue is 123456 would be put into username when i want it to be in something_else

ember ledge
#

In your client.py you can put a time.sleep() between server.send(b'123') and server.send(b'456')

foggy prawn
#

i need help

ember ledge
#

@foggy prawn maybe ask your question so we can help you?

foggy prawn
#

ok

#

i have trouble making a multiplayer script

#

im using the JSON library to send variables through a server script

ember ledge
#

post your script

foggy prawn
#

let me find it i have it on my other computer

#

server script

errant bayBOT
#

Hey @foggy prawn!

Uh-oh! It looks like your message got zapped by our spam filter. We currently don't allow .txt attachments, so here are some tips to help you travel safely:

• If you attempted to send a message longer than 2000 characters, try shortening your message to fit within the character limit or use a pasting service (see below)

• If you tried to show someone your code, you can use codeblocks
(run !code-blocks in #bot-commands for more information) or use a pasting service like:

https://paste.pythondiscord.com

foggy prawn
#

it doesnt work

ember ledge
#

Read

foggy prawn
#

i cant post it

ember ledge
#

You have to post your script

foggy prawn
#

ok

ember ledge
#

after you posted it, press ctrl + s

#

then paste the link

foggy prawn
#

i have pated my script in the website

ember ledge
#

yeah

#

press ctrl + s

#

copy link

#

and send it here

foggy prawn
#

here

#

this is the server

#

this is the network script for my game

ember ledge
#

I see

#

What's the error?

foggy prawn
#

i cant send the list voxel_info

#

when i search for the list in the tuple it doesnt recognise the keyword

ember ledge
#

then just do a class

#

and set voxel_info as a class variable with self

foggy prawn
#

i have tried to do that but still doesnt work

ember ledge
#

impossible

foggy prawn
#

i have tried in two weeks now to fix the problem

#

i can try again to make the class

#

thanks for the suggestion

#

bye

foggy prawn
#

i need help please

#

do anyone know an easy way to host your python scripts online?

ember ledge
ember ledge
#

heyy i have a little problem with this
https://www.strava.cz/foxisapi/foxisapi.dll/istravne.istravne.process?xmljidelnickyA&&zarizeni=3148
it somehow wont work with Curl and so on, does someone know what to do with the &? i know that its the problem

foggy prawn
#

thx

next canyon
#

@ember ledge Add me as a friend (te rog)

ember ledge
#

Also, if you use command line or any bash cmd

#

& will be recognise as a next command

#

like echo a & echo b

#

will output ```
a
b

#

So, i ll suggest you to put the link in ""

curl "https://www.strava.cz/foxisapi/foxisapi.dll/istravne.istravne.process?xmljidelnickyA&&zarizeni=3148"
ember ledge
#

if you want a .xml file as an output

#

just put curl "link" -O file.xml

#

yep

ember ledge
#

curl will paste the response in the file

#

with -O arg

#

if it is allowed

ember ledge
#

Im trying to find a module that supports RSA, trying to encrypt my server-client model traffic, anyone know a reliable module for this?

#

an asymmetric one used to encrypt traffic

errant bayBOT
crimson violet
#

Hello, I would like to make a proxy that allows each user to help host this one, do you know how I can do so that the user cannot access the request that the proxy does nor modify the code? A bit like Tor

ashen copper
ember ledge
#

Can anyone send me some documentation on multiprocessing

true bear
#

I have a .net core service, a rabbitmq and a machine learning python service classifying images.

What is the best way to transfer the images from the .net core to python.
Is it ok to shove the whole image to rabbitmq?

storm saffron
#

What kind of size

#

like 16x16 greyscale or 8k raw?

queen ether
#

what do you guys think of networking a top down approach? (book)

#

i want to learn some of the different terms related to networking so i know what im doing with http, ip and all these different things

crystal current
#

And if you don't know what the OSI network model is, start by looking that up :) just knowing that alone can help a lot

queen ether
#

thanks c:

frozen seal
#

does anyone know how s.sendfile() works and how to recieve it? im refering to the socket module.

hasty zodiac
#

Hey, if I have a vps and a domain on it and I want to create an ssh tunnel from my computer to the vps so I can reach my computer through the domain, how would the ssh command look like

crystal current
#

Basically you enable sshd, generate and configure certificates, open firewall port, that's about it

#

Er... That's just setting up ssh... Not sure I follow what you mean about reaching your computer through the domain, might need point to point VPN for that with OpenVPN or something

crimson violet
#

You know where I can find courses to learn how to make a proxy with python? I admit to seeing things but I absolutely do not understand what I am doing.

clear bobcat
#

I did something similar few years ago with OpenVPN as well as WireGuard

clear bobcat
crimson violet
#

Is it for learn^^

clear bobcat
# crimson violet Is it for learn^^

Start then with creating a simple server which will accept incoming requests. You can setup browser to use your proxy, but I would prefer to start with curl.
You can print and debug incoming packets and see how they are structured. You should read from the packets informations about destination server - just send data to this server and reply with response.

hoary forum
#

I'm using the built in socket library to ping an mc server and get data back, and i'm threading it to run this ping thousands of times a second. Problem is, when i do this without a time.sleep between starting the threads my system starts yelling at me about to many files open. With the sleep it gets slowed down a lot. Is there any way to make it not open files for a socket?

hasty quest
#

tfw you try to write a ping flood DOS but DOS your own system kek

#

absolute gold

#

also <@&831776746206265384>

hoary forum
#

wait what did i do

#

i'm just searching for minecraft servers lol.

#

by randomly generating ips and looking through all of them 🤷‍♂️

#

bruh

#

that's not what i'm doing

#

i can legit show you my code

#

it's not one host, each ping is to a different host

#

and every second it's a different set of hosts

#

so a host only gets pinged once per run

#

hypothetically all public ipv4 addresses until i have all mc server ips. ideally all of them only once.
yeah thinking about it that probably does sound kind of sus

#

Yup

hasty quest
#

"they" being the ISP?

hoary forum
#

stops and rm -rf's the project dir

#

I was trying to basically make a clone of the Coppenheimer project (scanned the entire internet for mc servers but is not oss so i can't use it), and use my clone to look for a specific server

ember ledge
#

How can I find all the Bluetooth devices near me with Pybluez even if it's connected? Right now Pybluez only finds the ones that aren't connected.

hoary forum
#

i'll just uh destroy all the evidence rq

#

only did like 200k IPs...

#

ya know only

#

yeah i stopped the script

#

idk why it would be illegal though Thonk

hasty quest
#

I don't see how it possibly could be illegal, plenty of legitimate orgs do it.

#

That said your ISP may not approve and you may get a lot of angry letters from people

#

Including TLO

hoary forum
#

i was thinking about running it on a droplet but that would prob get shut down

#

and if it is sort of illegal then how did the other project get away with it

ember ledge
#

Yea. Not a lawyer, but I'm not sure it's illegal. Your ISP, VPN, or VPS provider might care a lot.

hoary forum
#

i'll just quit now then

#

probably a good idea dogeKek

ember ledge
#

Yea. I think a droplet at DO would get shut down quick

hasty quest
#

There are vague arguments under which it could be called unauthorized access under the CFAA...
but so could sending literally any network traffic anywhere, so

hoary forum
#

I at least got 2 ip's out of that

clear bobcat
hoary forum
#

I'm trying to find a specific not listed one

#

Not that I could join it bc it's white listed but I could see when people log in and log out

hoary forum
ember ledge
hoary forum
#

I just want to see when people log in and log out. MC servers publicly provide who is online

clear bobcat
hoary forum
#

Basically I want to find the ip of the Hermitcraft server so I can see how long people are logged in for, and see the average time spent on the server in a session

#

Literally nothing. I'm just obsessed with Hermitcraft

#

How would it be spying though

#

Well it's a private server which is why i have to find the ip in the first place, invite only etc. The players online data you can get from the mc server list in mc too

#

I'm not going to do it anymore.

#

I'm also trying to figure out how the inspiration for this got away with it tho lol

#

Anyways thanks for telling me not to do that. I'll go enjoy my non-terminated internet

limpid bay
#

I'm trying to create a RabbitMQ handler for logging and I've got a class that goes like this:

class RabbitMQHandler(logging.Handler):
    def __init__(self, url, exchange):
        logging.Handler.__init__(self)
        print("Connecting")
        connection = pika.BlockingConnection(pika.URLParameters(url))
        self.channel = connection.channel()
        self.exchange = exchange

    def emit(self, record):
        print("Sending")
        self.channel.basic_publish(
            exchange=self.exchange,
            routing_key="",
            body=record.msg,
            properties=pika.BasicProperties(delivery_mode=2))


logger = logging.getLogger()
logger.setLevel(10)
handler = RabbitMQHandler(RABBITMQ_PATH, exchange="logs")
logger.addHandler(handler)

print("Before sending")
logger.info("Test")
print("After sending")

This hangs at logger.info("Test") and "Sending" keeps getting printed without the message being published to the exchange. What am I doing wrong? The connection works I can confirm that but I'm unsure how to debug the publishing part

west burrow
#

does anyone have a socket server where i could test on?

#

yea but id like it to be out of my network

#

bc i want to test my swarmer without 1ms ping

#

ill ask someone else

hasty quest
lean burrow
#

hey guys can anyone here help with UDP socket programming. I'm looking for a program to transmit images between a client and server. I've manage to done it via TCP and but i'm having a hard time to figure it out UDP

proud arch
#

import socket target_host = "www.imageDestination.com" target_port = 102 client = socket.socket(socket.AF_INET, socket.SOCK_STREAM) client.connect((target_host, target_port)) client.send(image_data) response = client.recv(some_amount_of_bytes)

#

I couldn't get the lines to indent

#

But those are some helpful lines of code to look at when inspecting socket programming with python

#

target_host = "www.imageDestination.com"
target_port = 102
client = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
client.connect((target_host, target_port))
client.send(image_data) 
response = client.recv(some_amount_of_bytes) ```
#

There that is better.

lean burrow
#

@proud arch Thanks, but you used socket.SOCK_STREAM I believe that is used for TCP and I'm looking for UDP socket that uses socket.SOCK_DGRAM

#

This this the code I've done for TCP

#

HEADER = 2048
PORT = 5050
DISCONNECT_MSG = """I used the image to transfer the image.
but finally the work is done, it always will be!!"""
SERVER = socket.gethostbyname(socket.gethostname())
ADDR = (SERVER, PORT)

client = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
client.connect(ADDR)

file = open('Test.jpg', 'rb')
image_data = file.read(HEADER)

while image_data:
    client.send(image_data)
    image_data = file.read(HEADER)



print(f"{DISCONNECT_MSG}")```
#

This is for client side.

#


HEADER = 2048
PORT = 5050
SERVER = socket.gethostbyname(socket.gethostname())
ADDR = (SERVER, PORT)

server = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
server.bind(ADDR)
server.listen()

c_socket, c_address = server.accept()

file = open('result_image.jpg', "wb")
img_data = c_socket.recv(2048)  # stream-based protocol

while img_data:
    file.write(img_data)
    img_data = c_socket.recv(2048)

file.close()
c_socket.close()```
#

and this is for server side

#

This is for TCP.

hasty quest
#

UDP is connectionless

#

To transmit UDP between two hosts, both hosts need to be listening for UDP

#

The UDP source port is optional, unlike TCP source port.

#

All you need to send UDP is this:

#

Server listening for UDP traffic on port P

#

Client sending UDP traffic to port P

#

The most basic UDP setup doesn't allow for reply traffic

#

but the client can open a UDP port and send that port as the "source port" field to the server

#

If it does this, the server is expected to send any replies to the client on that port.

pearl solar
#

is any one working with Nornir?? need some help.

frozen seal
#
while True:

    msg = s.recv(1024)
    msg = msg.decode("utf-8")
    print(msg)

    filename = filedialog.askopenfilename()
    print(filename)
    if filename == "Cancel":
        break
    s.send(bytes(filename,"utf-8"))
    
    file_data = filename.read(1024)

    while file_data:
        s.send(f"{file_data}".encode())
        file_data = filename.read(1024)```

im getting this error

file_data = filename.read(1024)

AttributeError: 'str' object has no attribute 'read'```

clear bobcat
fiery mountain
#

If I go on something like canyouseeme.org and I put in the port 80, but it says that the port is closed, how am I able to browse the web?

clear bobcat
fiery mountain
#

Or am I confused on how it works

clear bobcat
#

So servers need to have open ports to accept clients' connections

fiery mountain
#

Oh I see

fiery mountain
#

@clear bobcatNow if my ISP is blocking all of my ports and I want to connect/have something connect to me through a port, are there any other options I can use?

crystal current
clear bobcat
fiery mountain
fiery mountain
#

Oh, and one different websites it shows my IP as being different

clear bobcat
# fiery mountain What do you mean?

Public IP address is such kind of address which allows to connect directly to your device, when you are behind NAT without public address there is no way to connect with your device

fiery mountain
#

How do I disable a "NAT"?

#

Also how do I know if I am behind a nat?

clear bobcat
#

You cannot, this is mechanism used by your ISP

clear bobcat
#

You should start with basics about networks like what are private addresses, how packets go from your computer to destination server and so on

fiery mountain
#

Is there any work around?

#

Like can I use a VPN

#

Or something

#

Or like an RDP, like is there any way I could do it?

#

Like I'm trying to port forward, but I can't do so cause it says all of the ports are closed

clear bobcat
#

Do you want to connect with your computer from the Internet?

fiery mountain
#

I want to port forward so I can host a Minecraft server

#

@clear bobcat

clear bobcat
fiery mountain
#

Wait, what do you mean?

#

Like the VPN will need to connect to my server?

clear bobcat
#

But this still requires any publicly available entrypoint

fiery mountain
#

So there is no option for me?

clear bobcat
#

The easiest way is to buy VPS to host your server

fiery mountain
#

Oh, would I be able to port forward on a VPS?

#

Like does a vps have its own internet connect and isp etc.

clear bobcat
#

VPSes are basically available publicly so you just need to open chosen port and run your server

fiery mountain
#

Oh awesome!!!\

#

Thank you so much!

clear bobcat
#

You're welcome

proven quarry
#

not sure if i should do this here or in a help channel, but can anyone tell my why this doesnt work?

import requests


proxy = {
    "http" : "http://193.168.147.179:8080"
}
url = "https://api.ipify.org?format=json"

a = requests.get(url, proxies=proxy)
print(a.json())

it sends using my ip address and not the proxy

hasty quest
#

@proven quarry The server respects X-Forwarded-For and uses that IP.

#

Your proxy is relaying XFF info to the server, which the server then spits back out to you.

proven quarry
#

so like

#

is there a way to fix it

hasty quest
#

Just don't send xff at the proxy

proven quarry
#

but how?

hasty quest
#

Is it a proxy you control?

proven quarry
#

no i was just testing to see if it was actually possible, but i am going to get some soon

hasty quest
#

If you don't control the proxy you can't control what it sends. But there are other public IP identifying services which respond with the IP as seen by the server, not XFF

proven quarry
#

what about it?

hasty quest
#

It uses the IP as seen by the server

#

It doesn't use the XFF

wide frigate
#

anyone know why this wont work

#
import requests

req = input("~~#curl ")
url = (req)
payload = dict(key1='value1', key2='value2')
res = requests.post(url, data=payload)

print(res.text)
#

ping me

ember ledge
#

hey can anyone suggest me what to do to learn networking in python

hasty quest
#

My mind reader is in the shop

steady horizon
ember ledge
steady horizon
#

Try tech with team or look at other resources online, maybe even read the docs of one of the libraries.

ember ledge
#

oh

#

ok

#

thansl

#

thanks

steady horizon
#

👍

ember ledge
#

Hi! I want to list all the tab names of a Web Browser in a Google SpreadSheet. So I thought about Python (Maybe making a webbrowser with it and from there take the names of the tabs?). I wonder which is the best approach and if Python is the way to go.

livid orbit
#

@ember ledge you could use Google's own sheets API in python. We're doing that to read sheet data. It has a function to fetch tab names.

ember ledge
#

thanks!

steady sage
#

How to get mac address of network if i know ip , using socket module

crystal current
#

Don't ask me any questions though 😉

hasty quest
#

you don't need to write a sniffer just make an arp request lol

#

For linux you can fetch entries from the ARP table using netlink

#

If you don't want to dig into netlink, there's arpreq on pypi for that

indigo crater
#

i love to network through people

prisma cobalt
#

you should leak delicate infomation like that

hasty quest
ember ledge
#

hey guys! I'm 16 and I was just wondering what degree u guys got to become a network security analyst, network engineer, etc?

#

computer science?

cedar forum
#

compsci and if you can get some industry experience in the degree

#

also worth looking at CCNA, which is a cisco networking exam that is pretty much the standard networking qualification to hold

#

this is the networking module on my regular degree, we cover earlier networking stuff as part of our mandatory systems modules

ember ledge
#

Right now I’m doing the intro to networks 1

#

I’m gonna take that exam next year

grizzled goblet
#

I have a "Networking question" 🤔 I but it has nothing to do with Python is it fine if I ask it here?

grizzled goblet
#

So I just enable "Block http (port 80 and 443)" on my WIFI does that mean if anyone tries to DoS that port on my WIFI it will not go through?

fierce pivot
#

Work on both windows and linux 👀 👀 👀 👀

soft silo
#

any idea how to make a sound and screen sharing thing in python
i don't know where to start off

#

do i send a video with short duration everytime?

#

image + audio data seems large to me

versed spindle
#

I ran into a few minor issues when trying to implement a server to receive HTTP requests using sockets.
I sent a POST request to the server via Requests. In the synchronous case, it was fine, but when I enabled multi-threaded concurrent operations, the server received data missing the request body.

#

Can someone look at it for me?

gritty island
#

how would i send packet like "fe 01" as hex to a ip and port

uncut plover
#

Anyone learning python with some good automation experience?? /Europe?

hasty quest
#

do you want them to be able to reply?

gritty island
hasty quest
gritty island
#

so

#
sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
try:

    print(f'sending')
    sent = sock.sendto(b'\fe\01', address)

    print(f'waiting to receive')
    data, server = sock.recvfrom(4096)
    print(f'received')

finally:
    print(f'closing socket')
    sock.close()```
#

i found that online but idk if it works

hasty quest
#

You didn't tell the socket where to send the data to

gritty island
#

its the tuple address

hasty quest
#

Oh nvm I see the sendto

#

try it

#

see if it works

gritty island
#

i did

#

been here for 5mins

hasty quest
#

then the server didn't send anything back

#

or it couldn't reach you

gritty island
#

if it helps

#

i send the same packet in packer sender

#

and it works there

hasty quest
#

If the server is responding to one input but not another, then they aren't the same

#

Either the server isn't responding because something is different, or it is responding but not reaching you

#

Might be the latter.

#

Is there a source port in whatever you sent before?

#

Source port is optional in UDP. Might be that your python isn't sending a source port

#

To make the socket send a source port so it can get a reply back, bind() it

gritty island
#

so bind(address)?

hasty quest
#

no

#

You bind to the address you want to receive data at

#

just use ('0.0.0.0', 0)

#

Binding on port 0 will just use whatever port is available

gritty island
#

where do i put this

#

the start?

hasty quest
#

It's a method of socket

#

Call it on your socket object

#

Binding on ipv4 0.0.0.0 will use any IP that the machine is reachable at

gritty island
#
    sock.bind(("0.0.0.0", 58180))
    print(f'waiting to receive')
    data, server = sock.recvfrom(4096)
    print(f'received')```like this?
#

im assuming it sends to port 58180 since its what packer sender says

hasty quest
#

no

#

You have no idea whether that port is available or not.

#

In fact, you know specifically that something else might already be using it.

#

That's why I said to use port 0, which will use whatever port happens to be available.

#

And you have to do it before you send the initial data.

#

If you don't bind the port before you send the data to the server, how can the server know what port to reply to?

#

You bind first, so that the server knows what port to reply on.

gritty island
#

still doesnt print recieved

hot torrent
#

I am hosting a private linux server which only has an IP. No domain name. Is it possible to implement a Cloufflare/Akamai like service on my server to protect it from DDoS and other attacks ? If yes, how acn I do it ?

hasty quest
#

If the cloudflare server at 103.21.244.0 sees this request, and doesn't have the page cached, it knows to fetch the page from my server.

GET / HTTP/1.1
Host: mypage.tld
#

If the cloudflare serverjust sees this:

GET / HTTP/1.1

Then how does it know which site is being requested?

#

You need a domain. Get one for like $5 a year on namecheap

near delta
#

hi there
I have some csv online
but it make me download
for example here :https://www.cophieu68.vn/export/excelfull.php?id=AAA
i want ro read it online without download
so i make a request to this site but it need to login
i've tried put username and password in requests params
i also tried use HTTPBasicAuth and HTTPDigestAuth
but i still can't log in
can anyone help me

late birch
# near delta hi there I have some csv online but it make me download for example here :https:...

have you tried to capture the HTTP request that logs in to the user? you could use the dev tools from chrome or firefox to inspect the request sent, which will give you the necessary information you are looking for on how to construct the request. I am guessing you are not using the write fields in the HTTP request. Keep in mind that most of the time user login is done with a POST method (sending form data to the server)

ebon isle
#

hi
im trying to make like a chat on socket
and i got into a problem when i try to remove the client from the clients list on his disconnect

def logout(client, address):

    clients.remove(client)
    client.close()
    print(f"[LOGOUT] {address} Disconnected.")
    broadcast(f"[LOGOUT] {address} Disconnected.")

this is the function im calling when he gets disconnected

#

i get this error:

ValueError: list.remove(x): x not in list

ember ledge
#
import twisted
import socket

"""
Create a web app using Socket and Twisted.
Twisted is known as the web.
"""
web = twisted.Client()

while True:
  twisted.addClient(web)
  if clients > 5:
    break
left geode
ebon isle
#

but why is that happening?

left geode
#

Wait, you removed the for loop from your original code, right?

#

And you still have the same error ?

jovial python
#

@hasty quest

#

hi

#

so yea, as I was saying, my aim is to provide software for users to run on their PC's, the software then makes their device a server, and then uses python network automation to open router ports so their buddies can connect to their server

#

and I was hoping there would be a default style meaning that 80% of the time the software follows default steps to open the ports, and if it fails (due to the router not following the same format as most) then the person trying to use the software has to manually open the ports on the router config

#

but you say that router brands all have very different automation styles so it is unlikely to follow a similar pattern to open ports accross all router brands

#

@hasty quest

hasty quest
#

@jovial python there are basically two ways I can think of to do what you want with different drawbacks

#

option 1) implement automatic forwarding for the most common models of consumer routers. The downside of this option is it takes more time to set up initially and limits you to supporting specific devices.

#

option 2) set up something kinda similar to ngrok, basically working like this: customer connects their server to your server - customer's clients connect to your server and you forward the data on that connection to the customer's server

#

The downside of this is you need dedicated hardware to do the forwarding and enough bandwidth to handle all that traffic. It's not the most expensive option ever because it's cheap on compute, bandwidth is much more of a factor than compute in that situation.

#

So that'd be an alternative to supporting various consumer routers. Instead they just connect to you and you forward all the info, this works regardless of their router model because it's just TCP which everything supports ofc.

#

Faster to get up and running and works with everything, but costs money to maintain

#

The easiest way I can think of to set up the latter would be to set up your server as a VPN gateway and have the client machine connect to your server as a VPN client. Then your server is basically the router, and port forwards everything to their client.

jovial python
#

wow

#

ill read that

#

rn

#

lmao tysm

iron fjord
#

Katy likes to write essays when helping others

hasty quest
#

oh right, one more issue is you have to have a different port for every customer machine hooked up to your server, unless you have multiple servers.

#

So two different customers can't both have port 80 on that setup lol, unless you have some way to multiplex the port to multiple backends.

#

which you could do but would require different setups for different services like http

jovial python
#

I think im just going to provide extremely well explained documentation to the users who want to host servers, on how to perform port forwarding!

proud arch
#

@hasty quest Couldn't you control the meta data and share the port? Like a multi-threading situation? Especially if one costumer is sending less requests, I would think about like sequence packets and the nodal queue to be handling this circumstance. BUT I am a total novice so please help! xD

prisma cobalt
#

oops wrong reply

#

well same person, wrong message

hasty quest
#

Right, but the problem with that is that the client has to support it, in order to ident the service they're trying to reach.

#

I don't mean the client as in the customer paying to have forwarding set up, I mean the person trying to connect to the service the customer is hosting

#

It is doable for specific services, since things like HTTP support service multiplexing already.

prisma cobalt
#

im confused, isnt every client running the same script and that script would contain the code that supports indentification

hasty quest
#

No

#

There are two clients involved

#

There's the customer's client which connects to the forwarding service

#

And there's the client trying to connect to that service.

#

the customer's client, you have control over and can set up however you want.

prisma cobalt
#

oh so theres communication between the custom script and the unknown script?

hasty quest
#

No

#

I will draw you a diagram

prisma cobalt
#

ahh hang on, i misread the question 😳

#

they are trying to make the generic person running the script a server

#

for other clients/services

hasty quest
#

They're trying to provide a service that lets the customer have a service be reached without having to configure port forwarding.

#

One way to do it is to have the customer connect out to an external server, and have the service be reached there. The external server then forwards that information over the connection made by the customer to the external server.

#

The arrows show the direction of connection. this option works by having the customer's machine connect OUT, instead of being connected INTO.

#

Connecting into NAT requires port forwarding at the NAT gateway. Connecting out does not.

#

The issue with this configuration is that if the external server is just proxying TCP, two customer services cannot run on the same port.

#

There'd be no way for the external server to know whether a connection coming in on port 80 should go to Customer A or Customer B.

#

You can multiplex and run multiple HTTP backends through the same port, but you have to parse HTTP to know which backend to go to. At that point you're working on the application layer, not the transport layer.

#

And you'd have to configure this forwarding for every application layer you want to support.

jovial python
#

wow

#

nice diagram

#

lmao

#

ill read all this rn

glacial stone
#

Hi i have a question:
Say there are 2 computers connected to the router with a hub, so they both get each others messages. How does my computer know what messages are meant for him and what he can ignore (i guess by the dest ip), and how he ignores the irrelevant messages?

hasty quest
#

IP is a network layer concept and happens in software. Filtering is done by the firmware on the network card itself at the data link layer

#

that's why you have a mac address. Mac addresses are used to indicate network devices on a network.

#

A network card which receives a packet destined for a different mac, can freely ignore it, which saves performance.

#

If it sees a packet destined for my card, only then is that packet transmitted to the CPU to be processed by the operating system.

#

Ip addresses have more to do with routes.

#

The IP address is mainly relevant when sending traffic through a sequence of several machines, rather than from one network card to another network card.

#

At any rate, the IP address is purely a software concept, whereas packet filtering can be done in firmware and without the aid of the CPU or the OS.

glacial stone
#

Thanks🙏🙏

#

That makes sense

#

Also i guess because IP addresses change all the time filtering by mac would be more relevant

hasty quest
#

It's less because they change and more because they're a software thing

#

You can chance macs as frequently as you please

#

They're not actually baked into the hardware in most cases. All common network cards let you set whatever mac you want

#

But it is ideal if you can avoid sending any useless data to the operating system.

#

Also, filtering at the network layer means you have to write filtering for every single network protocol you want to use

#

In reality that's like 2, but still.

#

Filtering at the data link layer lets you have filtering for every protocol which runs on data link

cloud root
#

k, so where do you lecture, I'm signing up @hasty quest

ember ledge
#

anyone knows how to do port forwarding using ddns?

shadow loom
#

can someone help me im getting a error

#

**print("Im test")
print("Welcome to my survey")
first_name = input("Whats your first name?:")
print("Hi " + first_name)
last_name = input("Whats your last name?:")
full_name = first_name + last_name
print("1.Hello There " + full_name)
print("2.How are you feeling today?:")
print("3.Select your mood 🠗")
print("4.Happy 😊")
print("5.Cool")
print("6.Just right 🙉")
print("7.Mad 😡")
custom = input("8.Custom:")

while True:
# take input from the user
choice = input("Enter choice(1/2/3/4): ")

# check if choice is one of the four options
if choice in ('1', '2', '3', '4'):**
#

whats the problem with it

ember ledge
austere veldt
flint timber
#

Hey guys what does a layout look like with IP address, subnet and internet? I dont understand the concept of network much. Can someone help?

devout zephyr
#

Can sending big data requests to API's break them or will most API's ignore the request/truncate it?

hasty quest
#

if you look at boto3 for example it has interfaces for "chunking" requests and responses.

#

if you need a fuckton of data, the response will ship with a field you include in the next request to get the next piece of data

devout zephyr
#

ok thanks

dapper pulsar
#

to have PyPI as fallback and source of truth for hashes but to pull the packages from users that are close by if possible

#

now, one of the main issues i'm struggling with is how discovery without central coordination could work

#

i was thinking that github could be used as a "white board" to seed the network initially and then i was wondering maybe there's a feature in ipv6 that could help spread data through the network?

#

since we have PyPI as a working fallback, there's no need to be backwards compatible with ipv4 or other outdated tech

#

any ideas that come to mind?

zealous badge
#

Ive seen in chat apps tht u need to connect to servers using sockets but I have tried to connect to public ips with sockets but its doesnt work, how to connect to public ips

#

Like I can only connect to devices in my network

leaden cypress
#

Find out if the organizations behind those public ips have an API you can use to connect

zealous badge
leaden cypress
#

You setup a server

#

Then you can look at hole-punching

#

Or, you can use peer-to-peer technologies

vernal jackal
#

Any idea how to bypass CSPF?

#

I am trying to make a request and I get block from a page says smt about CSPF

ember ledge
#

CSRF token is usually in your first response

#

to the website

#

so what i would do is intercept all the requests to your website of choice using Burp Suite or other

#

and then track down which response contains that generated CSRF token

#

then repeat that request

#

whenever you get an error about CSRF

vernal jackal
wanton ether
ember ledge
#

that requests a CSRF token from you

mortal onyx
#

hello do you know a liblary that analyse the network trafic

vernal jackal
# ember ledge is it your website

I just make a call on a website I wanna get some data from and when I make a request, as response I receive an HTML saying Forbidden 403. CSFR Verification failed

prisma cobalt
thorn valley
#

not a bot btw

ember ledge
#

it still stands

cedar dust
#

hello I have a socket programming problem regarding socket.close(). When I close the socket, the client prints endless newlines. Can anyone help me?

visual mountain
#

hey, I am trying to make a chat room application using tkinter and socket but I don't know how to connect with a remote host as I am hosting on locally how can someone connect to me

woeful charm
#

oh

steady horizon
woeful charm
#

Can you please elaborate

steady horizon
#

Your router doesn't open its ports for you and the programs you are hosting inside your local network. You need to forward the port so the router opens a port that connects to your server's port. That way, any client that connects to your router with its public IP and the port you chose can communicate with the program the port is connected to

#

Basically if your program uses port X, you need to forward it to port Y (or X, doesn't matter) of the router.

#

And your clients need to connect to the router's public IP address and the port you chose for obvious reasons

#

The process for forwarding a port is different for every router, but it usually involves putting your router's gateway IP address into a web browser, logging in and going to advanced/port-forwarding settings

woven blaze
#

oh

#

ok

ember ledge
#

Hello is there an HTTP/3 Python library in python at the moment

cursive hedge
ember ledge
ember ledge
#

Burp Suite

#

is a program

#

that basically acts as a middle man

#

between your browser

#

and a website

#

so with it, you can intercept all requests sent to the website

#

and then find one that generated CSRF token (aka had it in the response)

prisma cobalt
prisma cobalt
#

wait HTTP3 isnt even in effect yet

ember ledge
#

well Google, Discord, and Facebook all use it

prisma cobalt
#

urrr

ember ledge
#

OK, my fault, HTTP/3 is not yet an RFC (but the underlying QUIC is RFC9000 now)

prisma cobalt
#

there wont be a python module for it

#

but you could make your own client with sockets and learning the HTTP syntax

#

but HTTP3 hasnt been finalized so expect needing to change the code to match the current spec

prisma cobalt
#

if you want help building a custom HTTP client the ping me @ember ledge
i have experience with custom HTTP servers and clients based on just low level sockets (from scratch)

ember ledge
#

@prisma cobalt

#

do you have any sample code

#

for HTTP2 implementation

#

by any chance

prisma cobalt
#

not for HTTP/2 sorry @ember ledge

deft hornet
#

hey

#

anyone there ?

cedar forum
ember ledge
#

hey i just want a simple register an login system i tried many times but i failed can u give me a prototype code ?

ember ledge
#

I've read through like first 6 chapters on RFC

#

seems doable

prisma cobalt
ember ledge
#

sockets and TLS

ember ledge
#

hey

#

i need help with sockets

#

i am working on an educational purpose python botnet

#

and i am currently making the architecture getter function

#

but i cant seem to do it

#

can anyone help?

prisma cobalt
prisma cobalt
jolly cape
#

hey, does anyone have any experience running a sql server on google cloud and running requests from python?

rigid edge
#

can a esp8266 , be made secure running micro python ?

ember ledge
#

just ask away

jolly cape
# ember ledge which part are you unsure about

so i read the google documentation for their bigquery api (I assume thats the API i need to run SQL commands to a my SQL server). I was wondering how i am supposed to verify that the request is coming from my program rather than another program that may of found the .json file in the program directory? Also, does the SQL query i send to the server return a record as an array if thats what my query wants?

ember ledge
mint plank
#

how hard would it be to make a simple browser and its hosted inside a website. it would proxy all of the stuff its loading thru the server its running on and not the users internet connection and then it will relay back what the website sends to the server

jolly cape
jolly cape
ember ledge
jolly cape
proven vale
#

hiii is there a way to make something like .NET's TcpClients and TcpListeners
in python?
and set up a multi client server chat room and send and receive data?

hasty quest
#

the socket function from that module supports tcp/ip, with address family AF_INET / AF_INET6 and type SOCK_STREAM.

proven vale
#

Oh okay thanks

stark jolt
#

I'm starting to develop a proxy similar to https://github.com/haveachin/infrared/, but for ALL TCP traffic. Am I able to detect a hostname from the headers to decide what backend server to proxy to?

thorn ether
#

Hi, my application need to acess to an API from a website and the code return me this error. Can someone help me pls ? (i'm on windows and the code work perfectly on other PC)

hasty quest
#

That's why infrared only works for minecraft. It looks at the server name in the application layer data.

#

That's also how a reverse proxy for HTTP works. It looks at the Host header from the HTTP request, which is application layer data.

#

TCP on IP only carries the source address, source port, destination address, and destination port.

#

There is a protocol for proxying arbitrary TCP traffic - the trick is that the client has to connect to your server using that protocol.

#

So it won't work for just any random person without special configuration

#

SOCKS is the protocol

#

Opening a connection over SOCKS involves the client sending information about what they're trying to connect to - so the server can then look at it and connect appropriately

finite dock
#

I have been unable to find an SNMP method to capture data, can anyone ping me with a method or tutorial? I've tried easysnmp and pysnmp but like I said I cannot seem to direct the output anywhere, just prints in every tutorial I've seen.. I need to save the data.
Thanks.

finite dock
#

fml.. I found the answer I needed.. redirecting stdout as a switch off/on

#

was only on lime the 30th google page, so prolly a few years old

deft hornet
#

hey bro

#

anyone ?

hasty quest
crystal current
thorn valley
#

Has anyone here worked with ANN models? If so please ping me!

#

i really need help aaa

steady sage
#

How can i print source ip, mac and destination ip, mac from arp in python

deft hornet
#

do you know ngrok people ?

queen ether
#

could someone correct me if im wrong?

deft hornet
#

hey

#

know ngrok ?

queen ether
#

access ISP's connect to the regional ISP then the regional ISP connects to the tier 1 ISP

hasty quest
deft hornet
#

hi dude

hasty quest
#

It's gonna take 1000 years to get anywhere if all you do is reply later to people who replied later to you, just sayin

ember ledge
hasty quest
#

But if you just come out of the gate with your real question, a single reply later might be all it takes.

deft hornet
queen ether
cedar forum
#

it obviously depends on who your ISP is

#

for example, it's generally not as long as that

#

and, you rarely connect to one tier 1 ISP

#

this is the diagram of my current connection from my University campus, so JANet (AS786) is my ISP so I connect to them and then they have a direct connection to 5 tier 1 ISPs

#

that means that when I go to an address Janet could route me through any of those 5 connections depending on which it deems the most suitable

#

similarly, this is the network for the ISP that I run, AS211224. When you connect to that network you will either upstream to Vultr or another one of my providers, then they will upstream to the tier 1s, I do that because it's considerably cheaper than buying transit from a T1 directly, whereas Janet can obviously afford htat

#

you can see which ISPs your traffic is going through with a tool like mtr, mtr [ip] and then hit z will display the ASN for each hop. You can see that when I ping to github.com my packets start at JANet (AS786), then they go through AS1299, which is Telia, a tier 1 network, and then they end up at AS36459, which is GitHub

#

(whois AS211224 will show what the ISP is, or just googling the AS number)

cedar forum
#

it's important to note here that "internet users" have many choices for how they get their connection to the internet. you could go through a tier 3 network, you could go directly to a tier 2 or you could even go straight to a tier 1 if you wanted but it would be considerably harder

nimble breach
#

I really need help to make this simple networking programing work

#

All the help channels are fulll

#

the nondormant ones atleast, because i cant access the dormant ones

frozen drum
cedar forum
frozen drum
#

Yes it is GÉANT lemon_wink

cedar forum
#

think we're using the on campus eduroam for COVID tracking this year which is quite cool

tidal skiff
#

Do you know a library with packets loss management for caméra streaming with cv2 for example ?

twin heath
#

anyone know how to reset a sequence number once it reaches max sequence

wicked ocean
#

i got a local network with two devices

#

one that has port 443 and 80 open, preferrably running caddy

#

and one with port 8000 open only to local LAN, running django