#networks
1 messages · Page 40 of 1
Note: I am using Google Collab
Note: Initially I did not have ngrok on my PC
After running the last code block I got the url but ngrok wanted me to create an account. I created an account, extracted ngrok on Windows, authenticated my ngrok agent and restarted my computer. I ran the last block again and got an error "Your account may not run more than 2 tunnels over a single ngrok client session." So I killed the process and ran all code blocks. However, after gettiing the URl and running it in the browser it tells to to register again.
Note: There is a warning when the URL is generated 't=2022-01-22T22:40:55+0000 lvl=warn msg="can't bind default web address, trying alternatives" obj=web addr=127.0.0.1:4040'
Why is this hapenning?
Can someone tell me what I should to do in order to solve the situation?
tried to change port to 5040 that has listening state but then it shows a different error 'The connection to URL was successfully tunneled to your ngrok client, but the client failed to establish a connection to the local address localhost:5040.'
How do I resolve this error?
OSError: [WinError 10013] An attempt was made to access a socket in a way forbidden by its access permissions
What threw this?
@earnest blaze When I try to use .sendall(packet)
I'm trying to make a custom ping message
So icreate a packet using stuct.pack
And then I attempt to send it using an ICMP socket
Is the socket connected to a peer?
what's a peer?
icmpSocket = socket.socket(socket.AF_INET,socket.SOCK_RAW,socket.IPPROTO_ICMP)
This is how i create my socket
I'd assume you didn't call connect ?
I have called connect
I created a tuple: ipAddress, port
I set port to 0 since its a ping and doesn't need a port from my understanding
hold on what's wrong with it? I couldn't connect otherwise and making it a tuple was the only way to make it work
Whats a binding?
You said you set the port to 0, what exactly do you mean by that?
icmpSocket.connect(myTuple)```
like this
since conncet required a tuple to be passed
so its the ip address followed by the port number
atleaset that's how i understoond how connect works/requires
Your wording makes it sound like the first thing you did while learning about networking is jumping straight into raw sockets.
Try running it with elevated privileges @visual raptor
I guess so lol
As in my windows privallages? or running my code with the sudo keyword?
Are you using both linux and windows?
Just do whatever it is you do on your platform
Ah okay, running windows currently but I'd also want this to run on linux
I've just realized I didn't ask you if this was a server or client 😩
Guess would be a client since I'm only sending messages to websites
But would want to turn it into a server so it can also recieve messages
Right now just sending
@earnest blaze upon changing my privallges it has worked
If I run it as an admin in powershell
im here @radiant garden
okay one sec
If I'm trying to crawl with some serious speed, how many semaphores is reasonable in aiohttp on a VPS? I want to get moving faster but I'm already at 100 semaphores and need serious speed improvements. Barebones crawler here, no fancy/expensive operations, just crawling. I just don't want to have my VPS company put a hit out on me.
or is this a better Q for #async-and-concurrency ?
feel like it's more of a connection question
imap or pop3.
how they work?
It's an rfc, read it.
Like how can imap can make us see those emails in our browser?
Because Outlook interacts with IMAP or POP3 in ways via csharp or something; i dunno.
Google or other such search engines will answer your broadstroke questions. For details, seek here. But research first.
If anything, setup your own domain and create a mail server.
Learn how it works from both sides of the equation.
Run your own Nameserver.
Understand that which makes the everyday things you take for granted.
guys im running a service which requires a connection between the client and my server, the problem is, my IPv4 address changes every few days, is there a naming service i can use?
async def connect():
while True:
try:
reader, writer = await asyncio.open_connection("notahost", 1)
except OSError as e:
pass
is this a good way to reconnect a connection or do i use up multiple file descriptors with this?
ok so apparently this does NOT cause a problem with file descriptors but is a memory leak
Not that I would be an expert on this topic, but I can't imagine where that memory leak would be.
apparently there is one 🤷♂️ https://bugs.python.org/issue44697
Am I the only person having issues with that link?
sorry, double subdomain, fixed it
Indeed, there send to be an unfixed memory leak. I wonder why it has not been touched for months. Really surprises me.
Yes, following the comments on there this might be even older which is quite odd. I'm thinking about switching to threading instead now to circumvent this since I'm not exactly worried about performance.
I'm curious if you have been able to reproduce the issue or just read about it. Can you verify it still exists?
I actually found out about the issue myself because I checked the file descriptors being used, I had the ram increase on every reconnect and thought I just didn't implement it right.
What a shame. Thanks!
anyone knows how i could connect to the internet with my home wifi's ip without being home like a proxy or something like that
@opaque gulch ssh would do that for you.
What is "request initiator chain"?
thanks man really appreciated
heya i need help with networking
so i need to have 10 conections onto the upper floor of my house
is there a way to bring 1 cable from the switch downstairs, and like parrel wire to the 10 connections
or do i need to grab an other switch?
@steep kite thanks man much better now
is the socket module encrypted by default ? like can messages sent inside of the sockets be clearly visible if someone was sniffing packets ?
No it's not
Is it possible to emulate a connection loss? Basically close() but in a way that an exception gets triggered?
is port forwarding necessory for sockets?
I am in need of a job can someone help me out.
@odd mesa depends on the networking architecture. As example in a server client infrastructure where the server is wide open - no. If the server is behind router with many hosts on same IP then yes.
There is also something called reverse connection which opens the networking backwards
Heya, is there a way to ssh to a computer where the IP changes constantly?
I have a computer with only 4g internet
Yes there is (Pontus) this would be a reverse ssh connection
I can baely do a normal ssh aha
Read up on Reverse SSH Tunnel
It could even be automatically setup through Python for example lol
And cron/task
If you want to get extra crafty, use logic to where if the connection drops it auto tries again
this won't be of any use
if both machines are on residential IPs
with each having a dynamic IP address and also ISP preventing any ingress traffic
if both machines are different regular ISP customers then your only option is to find a middle man @tacit matrix
that will receive/pass the traffic from one machine to the other
with this for example
you can either run your own middle man ("proxy server" as mentioned in that repo) in the cloud
or otherwise you can use the server of the guy behind this repo
which is the default setting, you dont have to tweak anything, but that option is risky because your SSH terminal commands could be seen by that middle man
so running your own proxy server in the cloud is what i am doing personally
Is there a way to extract certain infromation from a byte struct?
For example I'm receiving a packet and I want to extract the arrival time
Define "arrival time"
Sorry not arrival time but time to live
I understand that its the 8th byte but not sure how to extract it from a b* packet
struct*
Well for one, you'd need to not strip out the header.
recv does that by default, are you using a raw socket?
yes
mmmm maybe i could using something like myPacket[20:28]
Something like that should select only the time to live
what does the 20 : 28 signify here?
The ttl data is the 9th byte
Well in an ipv4 header at least.
So I have a socket listening for an echo return
b'E\x00\x00\x1c\x00\x00\x00\x004\x01\x9fP\xac\xd9\x10\xee\n<\x1f\x8e\x00\x00\xef\xff\x00\x00\x10\x00'
This is what i recive
So that means that 20:28 should get the time to live
Actually maybe I'd want to do [8:9] since this would get the 8th byte
Which would be the time to live @earnest blaze
This is a great resource, thanks for sharing! 🔥
you should probably use the struct module for that.
for anything "below" a single byte, like the version and IHL part (and the flags), you need to extract a full byte (or more) and do bitmask operations.
i keep getting these errors when trying to import requests Traceback (most recent call last): import requests from .packages.six.moves.http_client import IncompleteRead as httplib_IncompleteRead from importlib.util import spec_from_loader
Actually reverse SSH tunneling works perfectly fine on its own. I was assuming individual accesses via cloud VPS jump box or knows how to use port forwarding. If he doesn’t then he probably shouldn’t be doing ssh reverse tunnels at all
But sure, you can add a middleman cloud proxy instance if you want - I think that is further complicating the task at hand though. 100 ways to skin a cat
Depends on if the client will be able to connect outside your wifi or not
he doesnt use cloud
he just wants to connect 2 regular ISP customers that are in different locations
5. Do not provide or request help on projects that may break laws, breach terms of services, or are malicious or inappropriate.
I get this error: ```Traceback (most recent call last):
File "/home/neonje/Desktop/TCPClient.py", line 9, in <module>
clientsocket.connect(str(host, port))
TypeError: decoding str is not supported
This is my code: ```import socket
clientsocket = socket.socket(socket.AF_INET,socket.SOCK_STREAM)
host = input("Enter the host you want to test: ")
port = input("Enter the port you want to test: ")
clientsocket.connect(str(host, port))
message = clientsocket.recv(1024)
clientsocket.close()
print(message.decode('ascii'))```
how could one implement a global bandwidth limit for an app that uses aiohttp?
@versed vortex I feel like your issue is the fact your trying to pass both host and port as a string when it's expecting port to be an int. I'd change the port variable to be casted as an integer.
So what would be the correct way?
how would i change the cipher suite on aiohttp
Anyone familiar with scapy
port=int(input ())
actually
Delete your type casting to str for host and post in the connect call
serversocket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
#host = '192.168.50.34'
host = socket.gethostname()
port = int (input("Enter the port you want to scan: "))
serversocket.bind((host, port)) #it gives me an error in this line
serversocket.listen(1)
while True:
clientsocket, address = serversocket.accept()
print ("received connection from " + {address})
message = 'Hello! Thank you for connecting to the server' + "\r\n"
clientsocket.send(message.encode('ascii'))
clientsocket.close()```
Host name isn't gonna give you your machines address
it does
No it doesn't, just machine/host logical name
what is network automation?
Assuming you want to accept connections from outside your pc, you need to change your host str to "0.0.0.0"
This accepts any/all ipv4 connections
I'm not OP
What's the error?
Perhaps you're trying to bind to an priv'd port and thus raises and error upon the bind call.
Ports >= 1024 are considered to be privileged ports.
Just bind to the doom port
i'm somewhat familiar with scapy and are using it for several projects, what do you need?
Hi guys! At my job, we have a bunch of Python-jobs running using cron/task-scheduler on three different servers. They run daily, weekly or monthly, performing data processing and database operations. Simple stuff.
But if one of them fails, we don't really see it until later. So I'm hoping someone can point me towards a tool I can use to monitor these. Maybe a simple web interface that the script can connect to and set its status to "OK", "Failed" or something similar.
Oh my bad, replied to the wrong message
We (and many others) use Splunk
Thank you! My company is moving into Azure (my group doesn't to DevOps/build stuff) - is Splunk compatible / duplicating this?
Splunk is (primarily) a query-based tool for parsing, exploring, filtering, pivoting and visualizing unstructured data
We use it to parse web logs and alert on errors, as well as many other things
No BS piece of software
Good for the resume if you work in dev, BI or IT
My organization has Splunk as well, maybe I'll ask for a demo and see if it fits our use case. I really just want to see if our scripts have run and maybe how much data was processed while we prepare for a full rewrite.
(I'm learning a lot as I go - some technological slippage due to personnel loss before I started).
I got a $15 class on udemy that taught me most of what I needed
And splunk has a free version I believe
Wish I could say the same for Tableau
Oh! Maybe I wasn't looking good enough.
Can it also "run"/manage processes? So I can control them from one interface isntead of multiple cronjobs?
Provided they're run through splunk itself, it's not intended to be an OS cron replacement
But you can make your own apps and addons
Just text files really
Cool. I was looking at Airflow at a friends suggestion. But I'm wondering if I just start containerizing them as a way of learning that as well, even if it might be overkill.
Teach me some orchestration, yaml etc.
If it were me I'd look at the title of the next job I want, go find 50 job postings for it, and convince my company to use that one
Whichever is being requested most often
In my town it's all node/kubernetes/microservices based on my job hunt last year.
Ah
I'm not a developer, but my field of responsibility deals a lot with networking and processing of data (I work with geodata).
People like our maps.
POIs and all that?
And we'd like to own more inhouse instead of hiring expensive consultants).
Check out sensu, it can run timed jobs and send notifications on errors /successes
Yeah. And traffic estimation, emissions, pollution, live boat traffic etc.
But like I mentioned, due to re-organization of the organization we lost a few good people in the past year. So now we are doing what we HAVE to. But I'd like to prepare for what we want to do as we rebuild our foundation
Seems wise
Thank you, noted!
Riffed my whole dept except me at the last gig
Employment security is a cruel mistress
All good
Ouch. Good on you (I guess)
Unless it left you in a bad position.
Unfortunately the departure of people at my job, opened up a position for me and I'm so much happier than a year ago. So yeah, employment security is a bitch
can someone here help me getting the access token from a oauth2 login without having a client _secret? ive searched everywhere for it, tho couldnt find anything about it
I'm a little confused as to how to implement the timeout functionality in my ping program that I'm making
Should I have a while loop that constatnly checks the time and see if the waiting time is greater than the timeout?
can you give us a bit more detail to your situation and how it relates to networking?
Yes I can my bad
So I'm trying to make my own version of ping in python, using a RAW_SOCKET
And I want my program to also recive messages, so when I send a ping it will wait for a response
However, from my understanding ping usually has a timeout incase it takes too long for a packet to return
So I'm wondering if I could get some direction as to how I would go about making some timeout functionality
the simplest way
its basically as simple as
from async_timeout import timeout
async def sender(socket)
try:
async with timeout(2):
await socket.send()
await socket.recv()
except asyncio.TimeoutError:
print("Didn't manage to receive data back within 2 seconds")
is it possible for more than one IP address to be connected to the same port on the same server?
i'm asking because i find it weird how my device opens up a random closed port to send/receive data, but when i connect to a server on port 80 for example, the server doesn't switch to an unused port..? all traffic passes through the port which i've specified it seems. unless this is handled internally?
why would it have to switch to another port?
to serve another ip address i assume
but then
The connection is not defined by the destination port. But by the combination of source/dest ip/port
so the same server on port 80 can serve multiple connections at once
that would mean a server can have up to 65k connections, making it easy to DDoS
yeah
i see now
It's pretty normal for servers to handle thousands and tens of thouands of connections
You may also be interested in http://highscalability.com/blog/2013/5/13/the-secret-to-10-million-concurrent-connections-the-kernel-i.html
Note also the date of the article 😉
sweet, thanks for the read
Note however that's it's more for bragging right. You wouldn't expect your typical server to handle millions of concurrent connections
How can I fetch product and vendor name from cpe?
whats cpe
i'm guessing you are referring to Customer Placed Equipment (CPE), often a router or layer-3 switch provided by the ISP to be housed by the customer
most of the time you will not have any access to the cpe at all other then physical access to the hardware it self, usually vendor, model number and serial number is printed somewhere on the device it self, which can be very hard to get at if it's mounted in a rack
what is the best way to send messages to all the clients connected to a tcp server without slowing down significantly pretty fast, I have both an asyncio version and threaded version of a tcp server and they both slow down pretty fast, asyncio version slows down after about 1k connections and the threaded one slows down after 4k connections.
both have something along the lines of
I guess I am also trying to figure out if async is suited for this, but I would assume so since I see a lot of async being used for handling a ton of connections, I also think its the way I am going on about sending messages to everyone, but still the performance loss has me stumped
#all connections are appended to this list
global_connections_list=[]
def add_all(current_client,msg):
for connections in global_connections_list:
if x != current_client:
connections.send(msg.encode())
you could not encode the message every time you send it
i.e. encode it once and put that in a variable outside the loop
but really you need to profile your code to figure out what is taking so long, I would recommend looking into something like https://docs.python.org/3/library/profile.html#module-cProfile
how are you doing the async and threaded versions?
I took a look again since I sent that message while I was on my phone and the message is already encoded, so I was wrong there
this is the async version of the code
import asyncio
#async TCP server
glist = []
# this is the code to send the message to everyone
# and this is what slows down the connections
async def send_all(writer,data):
for w in glist:
if w != writer:
#data is already type byte
w.write(data)
await w.drain()
async def handle_echo(reader, writer):
glist.append(writer)
# while True:
data = await reader.read(100)
message = data.decode()
print(f'data: {data} message: {message}')
#testing sending only 1 message to server and not all
# this does not slow down at all and does it really fast when testing it out with my test client
writer.write(data)
await writer.drain()
# send message to everyone connected to server
# await send_all(writer,data)
async def main():
server = await asyncio.start_server(
handle_echo, '127.0.0.1', 8888)
print(f'Online\nserver:{server.sockets}\n')
async with server:
await server.serve_forever()
asyncio.run(main())
and this is the threaded version of the code
import socket
import threading
import sys
gglist = []
#threade version of TCP server
def new_func(clientsock, msg):
for x in gglist:
if x !=clientsock:
x.send(msg.encode())
def listen(clientsock,addr):
gglist.append(clientsock)
while True:
msg = clientsock.recv(1024).decode()
# I comment out the bottomm function to test only
new_func(clientsock, msg)
#clientsock.send(msg)
serversock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
ip_address = "localhost"
port_number = 8888
address_port = (ip_address,port_number)
serversock.bind(address_port )
serversock.listen()
print('server up')
try:
while True:
client, addr = serversock.accept()
listen_thread = threading.Thread(target = listen, args =(client,addr),daemon = True)
listen_thread.start()
except:
print('\nerror')
sys.exit()
when doing the testing I run it so that the client only sends a message to the server and leaves the connection open, another test is having all the connections being appended onto a list and having the message being sent to the whole list. this function that makes it send the message to all connections is what slows me down significantly
Hello i'm new to the Python networking, and networking in general. I think i know some basics, but is there a mindmap or quick overview on this subject in Python? Short comprehensive understanding of TCP/IP, Sockets, UDP etc.
Maybe instead of sending it to the clients one at a time, you send it to all the clients at once
is there a way to go on about this, because I have tried looking for that and there is nothing. the only mention of sending the a message to all other clients connected was using a loop like the ones that I have used in both the async and threaded version or at least something similar
Well in asyncio I think if you wrote the data to all the sockets first and then awaited them to drain that would accomplish the same thing
Not 100% sure though if that's how asyncio works
Or maybe even draining before you write to the stream
That way you aren't limiting how fast you are sending based on your network io
I will try that out
Lmk how it goes
currently trying to hack myself because I forgot my router password 🙂
And I don't know shit about ssh so I guess this is gonna be a learning experience lmaoo
I know i'm gonna have to use Putty or some shit but damn
If you forgot your router password just reset your router
should be a button on it somewhere
hopefully you shouldn't be able to hack into your router
tyr to chek the unique number written on the router or on paper or try to login to the router adminpage
Hello guys
I’m a network Tech who needs advice on learning Python to get prepared to be a network engineer.
I’m considering a boot camp but until I can afford the classes. What’s the best resource that is a directly related to Network Engineering?
Is there any resources on here I can use besides this chat?
Hi all,
very new to python networking. Is it possible to host a flask web server? I want it receive api request from other apps. I created a rest-api with flask but don't know how to "make it" public.
This tutorial will walk you through hosting a python socket server online where it can be accessed and connected to from anywhere. We will setup a headless linux server and transfer our server side code there. After running the code from our server we will connect our clients.
Get a free $20 credit on Linode using the code TWT19: https://www.li...
Appreciate it, thanks
is there something similar to open(fd, closefd=False) for socket.socket(fileno=fd)
i would just use socket.fromfd but that requires family and type
i decided to use socket.detach()
Does anyone here have a way to access a free Linux/Bash Terminal on Windows? I'm doing an Intro to Networking course and kind of need one lol
WSL but it sucks iirc. not much you can do on there, but it exists. you might just wanna install a VM instead to have the full fledged experience, or just install a distro onto a thumbdrive and you can then slap it into any computer's usb port to have access to your system
Scapy is a wonderful library written for Python. If you're playing with network stuff, learn it. I like it so much I've built workshops around it, enjoy:
https://github.com/stryngs/workshops/tree/master/id0p/SNIPs
i'd like to contribute this as well: https://beej.us/guide/bgnet/html
good stuff
the example code is in C though, but python is similar to C so you might wanna pull up a C reference such as https://en.cppreference.com/w/c
django on a LAN PC refuses to let other PCs connect. Any ideas on the culprit?
- I've opened port 8000 for incoming traffic
- I've ran runserver with 0.0.0.0:8000
- telnet pinging to the local network IP with port 8000 hangs
- I've set ALLOWED_HOSTS in settings.py to ['*']
I'm out of ideas
Anyone interested in participating in a network CTF?
did you solve your problem?
I'm gonna take a break and revisit it tomorrow haha. Still not sure what it could be
So guys i have this code here in which Im just seeing that whether a port is open or not using netcat, But when i check for the word "host" in port_response it just returns NO but in the actual response open does exist!
IP = "The_Ip"
PORT = "3333"
word = "open"
port_response = str(subprocess.run("nc -vz"+" "+IP+ " " + PORT, shell=True))
if word in port_response:
print("ok")
else :
print("NO") ```
The response :
The_Ip : inverse host lookup failed: Unknown host
(UNKNOWN) [The_Ip] 3333 (?) open
NO
As you can see the word "open" does exist here!
Hello! Can anyone help me find solutions to implement this?
GPIO python script works fine locally on RPi, but I need to input variables and start it from anywhere in the world. I guess making the web form is not a big deal, but connecting it to python GPIO script seems mystery to me
Hello, I’m looking for a backend developer for a web3 related project
me too bro, let's cooperate
What you want?
Subprocess.run returns a dict data type i belive, try port_response.output
!d subprocess.run
subprocess.run(args, *, stdin=None, input=None, stdout=None, stderr=None, capture_output=False, shell=False, cwd=None, timeout=None, check=False, encoding=None, errors=None, ...)```
Run the command described by *args*. Wait for command to complete, then return a [`CompletedProcess`](https://docs.python.org/3/library/subprocess.html#subprocess.CompletedProcess "subprocess.CompletedProcess") instance.
The arguments shown above are merely the most common ones, described below in [Frequently Used Arguments](https://docs.python.org/3/library/subprocess.html#frequently-used-arguments) (hence the use of keyword-only notation in the abbreviated signature). The full function signature is largely the same as that of the [`Popen`](https://docs.python.org/3/library/subprocess.html#subprocess.Popen "subprocess.Popen") constructor - most of the arguments to this function are passed through to that interface. (*timeout*, *input*, *check*, and *capture\_output* are not.)
Also make sure capture output flag îs set
would you kindly check my question as well mate?:)
if a header starts with X does it mean its a custom made header ?
@bleak stratus if what you're expecting is html, it'll be in 'content' or 'text'.```py
import requests
resp = requests.get('https://torproject.org')
html = resp.content # Or resp.text```
No, there is documentation on the Chrome site for these headers
If I have a single PC running on Windows, can I use OpenVPN to host my own VPN or do I have to go through a server to do it?
!voiceverify
yes, though some have unofficial commonly accepted purposes
there's a list on Wikipedia
from what i understand it's more of a convention to name custom headers this way
thanks
hi guys, where does the DHCP gets its ip ranges from like where does it get the ip database to redistribute to the local computers
In Internet networking, a private network is a computer network that uses a private address space of IP addresses. These addresses are commonly used for local area networks (LANs) in residential, office, and enterprise environments. Both the IPv4 and the IPv6 specifications define private IP address ranges.Private network addresses are not alloc...
got a question of my own: when a packet arrives on port x, is the IP header processed and stripped by the kernel or is it and it's inner layers handed directly to the listening software?
kernel
well, soooort of depends on how you open the socket and stuff
but yes, IP is normally long gone by the time the packet enters userland, and normally TCP/UDP is as well
if you use raw sockets however, you can implement TCP/IP in userland
and using tun/tap interfaces, you can implement IP handling and even ethernet frame handling in userland (though should never do this for prod really, lol)
i see. thanks for the amazing explanation lol
👍
it's one of those things where it's like
90% of the time yeah, but it depends on how you're doing it
it's a config option on the router
you'll specify a CIDR and the DHCP server will allocate from that
so if you set it to 10.0.0.0/8 then it'll allocate ips starting 10.0.0.1 and so on
client sends a discovery message, server says "hi i'm here!", client requests an IP, server says "alright, take 10.x.x.x"
I'm trying to make a custom proxy server and this is killing me. I'm trying to use the low level socket libary so that's most likely why but does anyone know any guides/resources i can use to achieve this with python? Is there a raw to send the raw http request?
@iron fjord, just in case you're not aware, https://mitmproxy.org/ is in python and you can write a custom proxy based on that. Depends on what you want to do with it.
Thanks i'll most likely be using this
A solution is to provide the state and the state changes via the webserver as a websocket. And then have each pi connect the the websocket to receive updates. If you use FastAPI as the web server, it already comes with websocket support: https://fastapi.tiangolo.com/advanced/websockets/
FastAPI framework, high performance, easy to learn, fast to code, ready for production
Does it replace Flask or it can work together with it?
Looks lika alternative
Hello all, looking for recommendations on simple graphical packages to plot network devices and their physical connections on.
I've written a script that can log into any number of devices and pull their CDP/connection data. This tells me a few key pieces of info: remote/local port, remote/local hostname, model, etc. I am currently saving all of this data into a nice dictionary/yaml format and would like to know what my options are to graph this automatically
I aim to automatically map L2/physical topologies with this data
Hi. I'm facing issue using requests module when streaming data from file.
summaryCsv = tempfile.NamedTemporaryFile(mode="w", delete=False)
...
summaryCsv.close()
with open(summaryCsv.name) as f:
requests.post(url, headers=headers, data=f)
Any idea why ?
For anyone curious, I found a sweet module called 'N2G' that helped me with this task.
A good reference example I found was at https://bitstudio.dev/drawing-network-topologies-in-python/
i'm trying to create an http post that uses HTTPDigestAuth and i see you pass in the username/password, but how do i set the realm?
For most use cases it replaces Flask, but you can combine both. For your scenario, you should be able to do all in FastAPI. BTW, Flask does web sockets, too, (you just need to install an add-on, like described here: https://blog.miguelgrinberg.com/post/add-a-websocket-route-to-your-flask-2-x-application) so you can use Flask if you like.
Well ppl say fastAPI is fresher and faster, so I'll probably give it a try
It's new and awesome and focuses on the creation of APIs, not so much web sites. And it comes with great docs. Flask is a good choice for websites where the user interface is important (and so is Django). For my own projects: If it's mainly an API with maybe a simple form, I chose FastAPI. For a website I chose Flask. And for a real world project, you can chose to have both.
How can i recieve time to live exceptions in python?
where to start networking
hey, what do you mean?
bunch of ressources on youtube/google
depends what you want to do specifically
Not quite sure where to put this one so I'l ltry here - I'm using Ansible to configure Cisco devices and I need to enter a URL with params that contains a ? - if you type it normally it shows the help, so you have to escape it with Ctrl + V when entering it via the CLI. But I can't get it working with ansible. Is there a way to use Jinja or something to send Ctrl + V ?
This is my question about requests and xhr
Hey @ashen mantle!
You either uploaded a .txt file or entered a message that was too long. Please use our paste bin instead.
hello, i wrote a multiplayer tic-tac-toe but the problem is sometimes when i send something i miss the pack and dont recv it. I use TCP so it should correct the packet which get lost. after sending it the game "freeze" because the server didnt get the packet and is waiting for the packet. I will post my code if anyone can find the error it would help me a lot.
How to reproduce the problem?
Just play the game with 2 clients
Sometimes it happen after 4 moves sometimes a whole game finish and i need to start a new game
hello guys. i am create a coolest http-server, and create instructions for him, video and photo of the work is attached. please rate my project - https://github.com/srgwrtg0342/The-Best-Python-HTTP-Server
Hello, I dont know whether anyone here can help with me, but Im creating a new game using python sockets.
I am running it multithreaded, but some times messages and socket is still blocked
i dont know how to explain it better
will do
Hey @rocky dagger!
You either uploaded a .txt file or entered a message that was too long. Please use our paste bin instead.
@ember ledge I dont know whether its related, but also whenever I closed the server by stopping the program and running it again, the client cannot connect to the server.
If you need my client code I can paste it as well
Does anyone have any sample code or anything useful for rtp ejection
I send the base64 text to my server over HTTP.
The correct base64 looks like this (its fragment):
/9j/4AAQSkZJRgABAQAAAQABAAD/2wBDAAEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQE
but after print this base64 on the server using print(request.data) I get something like this
%2F9j%2F4AAQSkZJRgABAQAAAQABAAD%2F2wBDAAEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEBAQEB
Is there a converter that can convert these strange '%2F9' characters into the characters they should be?
I tried everything.
from flask import Flask, jsonify, render_template, request
# from rozpoznaj_litere import *
app = Flask(__name__)
@app.route('/', methods=['GET', 'POST'])
def getdata():
print(request.data)
data = { "test":"test" }
return jsonify(data)
if __name__ == "__main__":
app.run(debug=True,host="127.0.0.1", port="8081", threaded=True)
@crimson pier urllib.parse.unquote
Can someone help me understand why download managers like JDownloader are able to successfully retrieve files that I cannot with requests? Same home IP, correctly-formed browser headers, session, no proxies...
What is the difference between their request and a basic one using the requests library?
@wind oriole
it depends on the website
that is hosting the file
for example if you are using jdownloader to fetch data from mega then mega is closely integrated with the browser
and therefore it will be hard to achieve the same merely from the python context with requests
websites generally do a lot of tricks and dont just simply give you a link to the file you want to download
its not that simple always, sometimes the data is sent in many chunks over multiple connections, sometimes it may require some javascript trickery
at the front-end level
some websites actively make it hard to just download files from them, you can think of Netflix as an example, they wouldnt want their content to be easily downloadable so they take steps to prevent that
Hi Guys , Is there anyone in this chat that works as a network engineer or a network automation engineer?
Id like to ask a few questions in regards to networking & automation and how a standard network engineer would shift towards automation in networking
If you work at an ISP/MSP even better as thats where i currently work at
This may be a better spot to ask then in general, but is it possible with sockets to define which domains to try and resolve hostnames against?
Did u find the problem/solution?
I didn't check, but I can do now
Yoy're receiving from two different threads, who's to say which thread gets what data?
The data ends up effectively lost.
Additionally you're receiving arbitrary amounts, that's just a bad idea, due to nagle's algorithm,you'll end up reading the data of many sends in on recv since it will be sent out as one tcp segment.
And even if you disable the algorithm this doesn't make it any less bad of an idea.
Yes, that tutorial you read or watched where they just hammer 1024 in every recv call is popularizing a bad practice.
(At least for tcp, for udp it's a different story)
Also the two problems feed off each other, both are bad enough on their own, but the thread won't terminate unless it receives something that when decoded is equal to a particular string and also , however that isn't guaranteed since you're probably reading more than you need to, and also the other thread is already running and fighting for data which in turn and in turn leads to more fighting.
Normally to fix this is (with with minimal effort, i.e continue using threads instead of switching to asynchrony)
you'd need to make a one and only centralized receival thread, which will receive all the data and then dispatch them to the other threads waiting on it with a queue.
However, that still doesn't fix the second issue, and a more important glaring one, which I haven't talked about yet.
You don't need a second thread at all.
Just look at clientListen, even if it worked correctly what is it going to accomplish?
It's a loop,that breaks immediately upon receiving something, so what's the point?
Even more importantly that thing could only be one thing and that's the start signal, which is what'll essentially start up the whole game's logic, so why is it even in a thread?
You could just put that part before the logic loop of the game and tell the user to wait for another player to connect, and then just wait. And when you receive the signal, start the game.
Hey @frosty thunder!
It looks like you tried to attach a Python file - please use a code-pasting service such as https://paste.pythondiscord.com
import json
import requests
os.chdir('D:/Files')
i = 1
while i <= 666:
link = f" ipfs://QmTfFB7SxEQBb9g18LsX5zs5xab2JfTHFSmt1pHCefbafB/{str(i)}.json"
r = requests.get(link)
with open( str(i) + '.json', 'wb') as f:
f.write(r.content)
i += 1```
this is not working
Plus how can I add loop into threading?```import os
import json
import requests
import threading
os.chdir('D:/Metadata/test')
def function(i):
link = f" ipfs://QmTfFB7SxEQBb9g18LsX5zs5xab2JfTHFSmt1pHCefbafB/{str(i)}.json"
r = requests.get(link)
with open( str(i) + '.json', 'wb') as f:
f.write(r.content)
function(1)
thread = threading.Thread(target= function)
thread.start()```
Super educational, thanks for taking the time!
hey guys, I have this question in some assessment, had trouble finding answers online, can someone help?
What is the recommended maximum number of connections per Linux server (based on the lowest common denominator limit in a Linux TCP configuration)? Suppose we have quadrupled the number of transactions per second, please provide some load balancing and auto-scaling solutions.
It relies on the hardware and your application type. Is it a web server?
Depending on the weight of your app you can handle 1M+ connections
But it could be bottlenecked by other resources such cpu/ram usage
Scripting for networks in python is fun
hey guys I'm studying for cisco devnet, and I'm searching for a mentor for when I get stuck.
How to "catch" the requests a site makes with python?
Like what I can see in network dev tools in chrome i want to grab with python
if you are using the python requests library, requests aren't made without your own doing.
If you are using selenium to automate requests in a browser you can look up DesiredCapabilities
From a pure syntax POV -
You should use async/threading in your case for receiving the requests (don't use them synchronously in a loop); along with a for loop instead of a while loop, and separate the stages.
e.g. don't generate the links and call the site while you're opening/creating files.
So I have a web page. But I want it to also return a JSON, so for example. When I go to the page on my browser. It shows the page. But when I do a get request like an api, that it will give me a json back, even if I get the webpage as well that doesn’t matter.
why use the same route?
the requests are basically the same wether they're made from a browser or from a script, but i guess you could look into headers
like user-agent but i think its just a lot of struggle for nothing
you could also use parameters
you could make it so a GET request returns the page and a POST request would return your data
i know you said you still want it too be a GET request but... why? it overcomplicates things
if you do want a GET request for both your page AND your JSON response then either use different routes (reccomended) or do some clunky header validation like flexflower said
or or or since you dont mind getting the webpage back as well with your GET requests, you could put the JSON in an HTML comment in your webpage and then parse it out but this is inefficient to say the least
does urllib/yarl have any standard to create urls with a query like ?a&b=1 where a just straight up doesn't have a value?
How to get the href from a bs4.element.Tag ?
I got this
<class 'bs4.element.Tag'>
When I print
print(tag)
print(type(tag))
Maybe I could regex or something, but there isnt any property? If there is I cannot find it 😔
Okey
So print(point.__dict__) says that there it is point.href...
point['href'] this works while point.href dosent
🤔
It’s because I need it for my app. The api I’m using has a special way of generating tokens and codes. So I can get it to redirect to one page. But it has to be the same as the regen..
if anyone could help me in #help-pear would be really appreciated its concerning sockets
how would i ping something in python and get the actual ping data?
"actual ping data":
Can anybody give me code for a Router Software
How do i return with flask a list of dictionarys when i do a get request
The list looks like
[{
"dateandtitle": "13/02/2022 - The",
"Contents":
"content me"
},{"dateandtitle":"13/02/2022 - The","Contents":"content me 2"}]
I have it saved to a json file,
Can anyone give me cript for cisco configuration backup
hi so I have a client and a server but I keep getting the following error when running them:
error:
[TheRainMan@fedora networkPy]$ python3 server.py
[STARTING] server is starting...
[NEW CONNECTION] <socket.socket fd=4, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('192.168.227.137', 5050), raddr=('192.168.227.137', 37858)> connected.
Exception in thread Thread-1 (handle_client):
Traceback (most recent call last):
File "/usr/lib64/python3.10/threading.py", line 1009, in _bootstrap_inner
[ACTIVE CONNECTIONS] 1
self.run()
File "/usr/lib64/python3.10/threading.py", line 946, in run
self._target(*self._args, **self._kwargs)
File "/home/TheRainMan/Desktop/networkPy/server.py", line 21, in handle_client
msg_lenght = conn.recv(HEADER).decode(FORMAT)
AttributeError: 'tuple' object has no attribute 'recv'
import socket
HEADER = 64
HOST = '192.168.227.137'
PORT = 5050
FORMAT = 'utf-8'
DISC_MESSAGE = "!DISCONNECT"
ADDR = HOST, PORT
client = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
client.connect(ADDR)
import socket
import threading
HEADER = 64
HOST = socket.gethostbyname(socket.gethostname())
PORT = 5050
ADDR = (HOST, PORT)
FORMAT = 'utf-8'
DISC_MESSAGE = "!DISCONNECT"
server = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
server.bind(ADDR)
def handle_client(addr, conn):
print(f"[NEW CONNECTION] {addr} connected.")
connected = True
while connected:
msg_lenght = conn.recv(HEADER).decode(FORMAT)
if msg_lenght:
msg_lenght = int(msg_lenght)
msg = conn.recv(msg_lenght).decode(FORMAT)
if msg == DISC_MESSAGE:
connection = False
print(f"[{addr}] {msg}")
conn.close()
def start():
server.listen()
while True:
conn, addr = server.accept()
thread = threading.Thread(target=handle_client, args=(conn,addr))
thread.start()
print(f"[ACTIVE CONNECTIONS] {threading.active_count() - 1}")
print("[STARTING] server is starting...")
any help would be appreciated
upon client connection you pass in: args=(conn,addr)
but in your function definition you declare you args as (addr, conn)
you've basically swapped them round and since the addr arg is a tuple of the ip and address, the recv method will fail
thank you 🙂
Does anyone know if when you send a request to a proxy server, is the response all completed in the proxy server before it is send to you or is it real-time and sent in chunks? Also, can proxy-server modify the headers of a https request?
how many web browsers accept Python?
it is not possible run python code in browsers, only javascript code can
except when there is a streaming of data (like when you watch a movie) usually the proxy server send in a chunk the response that received from the server which contained the resource. yes proxy servers can modify the headers of a http request
Okay thanks
there are some projects like brython or pyodide that let you run python in the browser; however none of them are actually widely used
and all require external dependencies
!heljp
could someone please explain to me the difference between a vpn and a proxy
vpn is ptp and proxy is a inbetweener... usually data is also encrypted on vpn
wat
can proxy servers modify the headers of "https" request?
@ember ledge The reason am asking whether a proxy can modify a https request header is because one of the proxy providers I use, I tried sending GET instead of HEAD to save on bandwidth on some given requests, but when I checked the next day to see if I had saved bandwidth, lo and behold, there wasn't any difference. So I don't understand how some of these proxy providers measure there bandwidth, I was just confused and only suspected they either modify headers, so they must be doing man-in-the-middle somewhere... (because the requests are encrypted) Do large proxy provider do this?
how do i remove the firewall so that my sockets server can be accessed by multiple clients?
Why would you even remove a firewall? Open the right ports for it....
Removing the firewall and the server becoming accessible by multiple clients are unrelated.
well how would you open the right ports?
It depends on your OS and firewall you are using. Which OS are you using?
linux mint
Anyone used scapy? I'm getting lost in the docs to find what I need.
this
Router Security: The Benefits of Closing Unnecessary Or Unused Ports Leaving ports open can leave your router – and with it, your computer - vulnerable to hackers and malware attacks. A port functions just like a real world port. It’s a place where your network can communicate with the outside world (the intern
and what kind of firewall are you using? ufw or just iptables ?
be specific when asking questions like this 😄
"just" iptables lol
Actually got by with psutil.net_io_counters
ok... "just" isn't the right wording... 😂
I mean... iptables is already there... ufw needs to be installed.... :p
im using ufw i think
Rip bozo
Hi guys, may I know where can I get started to learn network automation. I am a beginner.
Mininet: An Instant Virtual Network on Your Laptop (or Other PC) Mininet creates a realistic virtual network, running
real kernel, switch and …
that can give you a virtual network topology on your PC, which you can configure using python
you can create virtual routers in that topology and build on that (example: https://github.com/mininet/mininet/blob/master/examples/linuxrouter.py)
@hardy coyote pinging so you see it 👍
this looks good!, thank you so much
Hello everyone 🙂
I'm working on a assignment for CIDR subtraction, in Python is anyone here familiar with this?
I’m not sorry
How do i return a json file (like an api) and a html file from a get request on the same directory?
So i am returning a html page on a flask site, how do i customise the headers as well?
So I'm getting a constant 5-10mb input at all times on my server which is really bogging down my server speed and was wondering if there's a way to log only the IP's with tcpdump so I can check-host the IP's and rate-limit their connections with iptables?
hi, im currently doing my undergrad and ive been learning ethical hacking (nmap,metaspoilt) and i would really like to learn more formally and get my certs in cybersecurity or ethical hacking and i was wondering if going for networking for my postgrad would help me get learn more about ethical hacking to get the certs
thank you in advance
hey, is it possible to make a multiplayer app in which ppl can create a lobby, and other ppl who have the lobby code can join them, kinda like that the unity photon pun apps
in python in mean
replit.com does this already
oo
u mean like it allows us to create applications like that?
if yes then could u send some example pls 
It’s a IDE with collaboration built in, it also has built in flask support so no localhost, u get a url you can use it’s pretty diverse ide, some limited things about it but I use it all the time
ik what replit is, i meant to ask if i can make an a multiplayer game sorta thing
using pygame or something
Oh a game I thought you were talking about a multiplayer code editor
Looking for an experienced dev to do some work. PM if interested. Good pay. Easy task.
please don't advertise jobs like this here
what is it you want done?
!rule 9
Hello. I'm brand new. I don't know a lick of python yet but I can learn! I'm trying to figure out how to run a particular GCcolab project on my local machine, I think it's possible in this case. I have Anaconda and Atom... If any kind soul is willing to help or just chat a little, answer a few questions, that would be awesome! Thanks!
I promise I'm not as dumb as I sound...
Personally I would help but i don't know much about your project sorry!
Is anyone here down to help me on a project if you want too dm me.
It's kind of vague, it might be easier to get help if you can ask a more specific question
is the project open source?
because otherwise we wont really be able to help
ohhk
can someone explain me what is SQL?
can i ask docker question
What is the difference between incoming-outgoing traffic and inbound and outbound traffic?
programming language for creating/managing databases
inbound is the traffic that comes to you from the outside, outbound is the traffic that you send out into the network
oh, i think i misunderstood your question, there is no difference
incoming, inbound and ingress can all mean the same thing
You can ddos urself 🤷🏼♂️
!rule 5
5. Do not provide or request help on projects that may break laws, breach terms of services, or are malicious or inappropriate.
hmm okay, how obout http proxy?
i am trying to use sockets to make a server and a client on two different computers on the same local area network, when i try to make the client connect to the server, it doesnt work, no error, no nothing. just doesnt connect, any ideas?
they are not on the same lan, or one blocks inbound connections
ty
I'm working on a bot that sends a request to the server and it returns with a variable named result:[] and inside there is a line variable line:"" and there are multiple of them
how can i read the line variable with json?
Maybe just run the tcpdump command in the background and redirect the stdoutput to a file which could be parsed and then appropriately rate limited
Can't you just iterate over through the result variable and store the value in dict/json format?
I've tried that but i nede to parse it because the owner is forcing me to do so
request's Response object has .json() method. After you have it, the object will be accessible like normal python nested dicts and lists
As you said you get a result key with a list:
r = requests.post(...) #or get, or whatever
data = r.json()
for elem in data['result']:
print(elem)
i actually love u
i love u
THANK U
BRB
I'm taken
Lol haha
Look into throttling connections
Likely the more suitable wording would be "firewall interface" since ufw is just a wrapper around iptables
Hey there.
I want to be able to connect to my work machine through web browser.
So basically,
I have a server running on my work, I want to be able to connect it through a web broswer (how can I set the domain and stuff).
basic, thing I did is to route traffic to aws server then to my machine (I don't want to do that).
Also, I don't want to use (open-ssh server), then connect via the shell from my home machine.
Any ideas?
Basics:
server (running for example jupyterlab, any program on some port).
multiple machine want to connect to that server via some domain
you can use ssh to build a SOCKS server, or you can do port forwarding with ssh and it'll route specific local ports to whatever remote port you want. another solution is to use VSCode's remote development + ipynbs. I know you said you don't want to youse open-ssh, but these are probably the easiest ways to bypass a firewall. otherwise, there's not really a domain setting you can do if you're not on that network
@golden escarp
for django
i have this get view thingy
and when i try to send a get request with axios
it always has the broken pipe error:
does anyone know a good solution to this...
what does something like curl respond with?
It means the client didn't wait for response and send another request. If I am not wrong.
yeah
DDoSing yourself isn't against the law if you have permission to do so.
Never said it is, you can hack anyone with their permission
I agree, but someone showed !rule5
Or whatever it is.
That shouldn't of been showed just wanted to back you up.
It's totally legal if you have permission.
Dw bout them, they cant do anything
Yes, especially if you hack urself It’ s not illegal 😂
Exactly
Hello all, is there a tableau expert in this room?
Hey guys guys trying to make a a script that pings the url address entered, how do i make it so that when the user enters the url address it pings what ever the user inputed, here's the code
import socket
import threading
import time
print('Neighbours Wifi V.7')
print('––––––––––––––––')
print('Enter a selected ip address:')
import os
x = input()
time.sleep(4)
def myping(host):
response = os.system("ping -c 1 " + host)
if response == 0:
return True
else:
return False
print(myping( + x))
I'm trying to make it so that the last code is working properly but I can't seem to figure it out.
If you can help dm me or let me know through this channel!
@ember ledge might be better suited to #❓|how-to-get-help ..
anyone ever ran into a issue where when u copy the requests code from postman, it doesn't work, but the http.client does?
Both with the exact same headers and payload
First, protocols are just a set of rules and how-to instructions, for example in English language, you can't just say gibberish to an english speaker without conforming to basic rules of English and expect some kind of response. You have to conform to those basic rules of English in order for the person you are speaking with to understand you, and give you a valid reply. It's the same thing with protocols in computer science.
HTTP protocol is used when you access the websites on the internet, what it means is that your computer sends a message to some website that must be in the format specified by HTTP protocol, otherwise the website won't be able to serve you anything. Now, HTTP proxy is just some computer on the internet which acts as a "middle man" between you and the website. So instead of you sending your HTTP request directly to the website, you instead send it to the proxy and the proxy will pass your HTTP request further to the website, without modifying it in any way.
you can just pass x to myping() when calling it, no need for + x
what is this request? why does it happen so often? is this normal?
hi
i have quastion about socekts
How can two programs find each other on the network and send messages without building a server and a client?
the top one is an HTTP GET request for the /generate_204 endpoint of the HTTP server
the one below it is the HTTP response with the status code of 204 meaning its worked
Hey guys
I recently made a menu for python, the Issue is when I select option 2 the code doesn't work here's the code
def list():
operation = input('''
Select operation:
[1] Add number to the list
[2] Neighbours Wifi V.2
[3] Display list
''')
mylist = []
if operation == '1':
print("Type the number you would like to add to the list: ")
number = int(input())
mylist.append(number)
elif operation == '2':
#option 2
print("Neighbours Wifi V.1")
print('––––––––––––––––')
import socket
import threading
import time
import requests
try:
if requests.get('https://epicgames.com').ok:
print("Loading...")
except:
print("Failed To Load!")
print("--‐-----------")
print("Leaving Programme!")
time.sleep(1)
quit()
time.sleep(5)
print('––––––––––––––––')
print('Neighbours Wifi V.7')
print('––––––––––––––––')
time.sleep(2)
print('developed and coded by qxagw')
print('––––––––––––––––')
time.sleep(2)
print('Make sure to use .com at the end!')
print('––––––––––––––––')
time.sleep(5)
print('Enter a url address:')
print('––––––––––––––––')
x = input()
print('––––––––––––––––')
print('––––––––––––––––')
import os
time.sleep(4)
def myping(host):
response = os.system("ping -c 1 " + host)
if response == 0:
return True
else:
return False
print(myping(x))
print('––––––––––––––––')
time.sleep(2)
print('Thankyou for using this script!')
quit(3)
#End of option 2
number = int(input())
mylist.pop(number)
elif operation == '3':
print(mylist)
else:
print('You have not chosen a valid operator, please run the program again.')
again()
def again():
list_again = input('''
Would you like to see main menu again? (Y/N)
''')
if list_again.upper() == 'Y':
list()
elif list_again.upper() == 'N':
print('Leaving programme!')
else:
again()
list()
Can anyone help?
yo does anyone know the exact security vulnerabilities of a p2p server?
if you have experience with p2p servers and want to help plz dm me
!paste
Pasting large amounts of code
If your code is too long to fit in a codeblock in discord, you can paste your code here:
https://paste.pythondiscord.com/
After pasting your code, save it by clicking the floppy disk icon in the top right, or by typing ctrl + S. After doing that, the URL should change. Copy the URL and post it here so others can see it.
Can you elaborate
@ember ledge yes for exampel : first pc has 2 port server and client and second pc has 2 port server and client if in network app run in this port can lisen and send messenge
thThis is possible؟
but my ports are dynamic
thank you so much undrestand
anyone know how to regenerate your temporary ipv6 address? currently my sysctl has: net.ipv6.conf.default.temp_prefered_lft = 86400
but I want to know how long is left on my current temporary privacy address and how to force a new one
I've not made any sysctl changes from a stock ubuntu focal system
I also have use_tempaddr = 2
can anyone assist me with some dnspython issues regarding docker?
im trying to get the dnskey from a website, however it cant seem to establish a connection to the dns server from docker but runs locally, does anyone have any suggestions and or has dealt with this directly?
I am having an issue with post requests on socket
when i run this i get error 308
my code works for get requests but does not work when i use post
@ember ledge the http response status code of 308 means "permanent redirect", the response they give should contain the new uri for you to request to
now im getting code 400
my headers are correct for the api im using i just dont know if my content is correctly formatted
or just gmni in general
Hello everyone, I'm having a weird error with apprise (v0.9.7), the same command works on my machine but not in a GCP instance, here's the command an the output
❯ apprise -vvv -t "I'm just playing around" -b "Ping" "mailtos://notifications:****@trackwebpage.com?user=notifications@trackwebpage.com&smtp=mail.trackwebpage.com&port=465&to=notifications@trackwebpage.com"
2022-02-25 19:01:23,738 - DEBUG - Loaded E-Mail URL: mailtos://notifications:****@trackwebpage.com/?from=notifications%40trackwebpage.com&mode=starttls&smtp=mail.trackwebpage.com&user=notifications%40trackwebpage.com&format=html&overflow=upstream&rto=4.0&cto=15&verify=yes
2022-02-25 19:01:23,738 - DEBUG - Using selector: EpollSelector
2022-02-25 19:01:23,739 - INFO - Notifying 1 service(s) asynchronously.
2022-02-25 19:01:23,739 - DEBUG - Email From: Apprise Notifications <notifications@trackwebpage.com>
2022-02-25 19:01:23,740 - DEBUG - Email To: notifications@trackwebpage.com
2022-02-25 19:01:23,740 - DEBUG - Login ID: notifications@trackwebpage.com
2022-02-25 19:01:23,740 - DEBUG - Delivery: mail.trackwebpage.com:587
2022-02-25 19:01:23,740 - DEBUG - Connecting to remote SMTP server...
2022-02-25 19:01:23,867 - DEBUG - Securing connection with STARTTLS...
2022-02-25 19:01:24,108 - DEBUG - Applying user credentials...
2022-02-25 19:01:24,528 - INFO - Sent Email notification to "notifications@trackwebpage.com".
I tried to add firewall rules to allow all ingress and egress for all protocols and ports but still having the same issue
[Errno 104] Connection reset by peer
Does anyone know why I'm getting this error? I made a game which works fine when I use my local ip address and multiple clients can connect without any issues. But when I tried hosting the server online, the clients connect to it without any errors but when I try to send data back to the clients this error occurs.
On the client side I changed the ip address to the online server's one so that shouldn't be the issue
On the server I changed my local ip address to "" as mentioned in a youtube tutorial
Soo, I installed nginx and have once the website (vhost on nginx) liquid-dev.de and the website lqdev.de (a sharex upload server).
Now when I go to lqdev.de the website comes up, for whatever reason. My configs are the following:
server {
server_name lqdev.de;
location / {
proxy_pass http://localhost:port;
proxy_set_header Host $http_host;
proxy_set_header X-Real-IP $remote_addr;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
}
}
und
server {
server_name liquid-dev.de;
location / {
proxy_pass http://localhost:andererport;
}
}
~~So I'm reading about implementing TLS using the websockets package and the example for setting up encryption uses a file localhost.pem that contains a private key and a cert. I'm assuming that the key and cert used in the example are not to be used production? I will have to generate my own .pem file using the ssl module correct? The localhost.pem file is found here https://websockets.readthedocs.io/en/stable/intro/quickstart.html#encrypt-connections~~ (Okkkk this file is for localhost use only lmao im a dummy)
Here are a few examples to get you started quickly with websockets. Say “Hello world!”: Here’s a WebSocket server. It receives a name from the client, sends a greeting, and closes the connection. s...
That doesn't have anything to do with the error, a connection reset error would require the connection to have been established to begin with.
It typically occurs if one end suddenly came down hard. Like an exception being raised. Or killing the process.
You would firstly need to check everyhting else functions properly, and then check how you're handeling disconnections, both the ones that adhere to the procedure and unexpected ones.
Any1 knows how to "stream incoming events"? Thought it is websocket or smth, but doesn't work, how to implement this? The code I tried py session = aiohttp.ClientSession(headers=_HEADERS) async with session.ws_connect('https://lichess.org/api/stream/event') as ws: async for msg in ws: print(msg.data)
API reference https://lichess.org/api#operation/apiStreamEvent
The error is py WSServerHandshakeError( aiohttp.client_exceptions.WSServerHandshakeError: 200, message='Invalid response status', url=URL('https://lichess.org/api/stream/event')
Probably I am doing something wrong
Maybe this is not even websockets
Help me someone
Seems like I found a method and it is not a websocket definitely
r = await session.get(...)
async for data in r.content.iter_any():
print(data)```
Is this correct?
does anyone know how to use proxies with socket to do get requests? i can not seem to figure it out for the life of me. Any help would be appreciated
If you could, the socket documentation would DEFINITELY have a section on that. Have you checked it?
It doesn’t
It’s not like requests where you can set it you have to connect and send your request through it
I’m getting code 400 when I do
Pasting large amounts of code
If your code is too long to fit in a codeblock in discord, you can paste your code here:
https://paste.pythondiscord.com/
After pasting your code, save it by clicking the floppy disk icon in the top right, or by typing ctrl + S. After doing that, the URL should change. Copy the URL and post it here so others can see it.
remove the sensitive parts
but generally
import requests
proxies = {
"http": "http://1.1.1.1:9040",
"https": "https://5.6.7.8:1001"
}
r = requests.get("http://www.example.com", proxies=proxies)
I’ll post it when I’m home thanks so much
Isn’t this for the requests package?
edited
I know that
I’m asking specifically for socket
oh
for sockets
its simple
if you use http proxies
you just connect to the proxy server
and then in the actual HTTP payload you send, you modify Host
header to point to the final website
Ah I figured that
But I’m doing ssl
Does that affect anything
no i don't believe so
Ah ok I did that and got code 400
I’ll post code when im home
400 is bad request
so you most likely have malformed payload
can you
post the actual payload
you send
over the socket
?
remove sensitive stuff
So, I’m an idiot, absolutely no idea what I’m doing with sockets, I have a basic understanding of python. I’m trying to make a simple chat system with sockets that works over the internet to talk with friends but any guide or blog or video I follow doesn’t even work locally. Please someone help.
British timezone by the way I may not respond immediately
is this how you would want it formatted ?
atm im getting an ssl wrong version number
whats the actual traceback
back o the sock.do_handshake()
?
what is the error message
are you positive
that this proxy server supports HTTPS
yes
actually
let me attempt a connection without https
i cant even ping this
server
on my end
try ping 209.127.191.180
from command line
this proxy server is down
oooooooof
wait
if you get any particular error, just paste it here
let me try doing this using requests
and see what happens
the proxy server
you are trying to connect to
seems to be down
bro
you can't connect to it
no
its working
i just used requests and it worked fine
you just cant ping it
Mostly errno 111 (connection refused) although a moment ago I got errno 61 instead. I can’t do any trouble shooting or anything right now as my laptop is turned off and I have to go to sleep soon so
i used the requests library not cmd
also
u wouldnt be able to because its ip authorization
over the internet to talk with friends - do you have a server in the cloud
fair enough
are you sure i,formatted
these 2 values don't have new line
bytes
or anything
oooooooooops
no
ignore those
wdym ignore those, they are embedded into raw HTTP payload
Uhhhhh, I don’t know, I have a raspi that I have my own basic onion site running on, I could use that but for the time being no
string= ("GET /v1/assets/63993845/resellers HTTP/1.1\r\n"
"Host: economy.roblox.com\r\n"
"Content-Type: text/html\r\n"
"X-CSRF-TOKEN: {}\r\n"
"cookie: {}\r\n"
"\r\n".format(i,formatted))```
thats the actual string
i know, but the values of i and formatted will be included
those headers were for my comparison in latency to requests
in the string
are you positive those don't have any
unexpected
bytes
yes
i tested without proxies and it works
sock = ssl_context.wrap_socket(
sock,
server_side=False,
do_handshake_on_connect=False,
suppress_ragged_eofs=False,
server_hostname="209.127.191.180")
for server host name do i put the roblox endpoint in there or the proxy endpoint
then it means
i would assume the proxy
the issue is with raw HTTP payload
you are manually setting
when using sockets
considering your usual request.get connecting to the same exact endpoint
worked just fine
yes and using socket without proxies also worked fine for those headers
im pretty sure roblox requires ssl as well
considering those cookies are sensitive data
@ember ledge when using requests
did you also provide
csrf
and cookie
headers
the head variable is what i use in requests
and you are positive you used
proxy
with requests
and didn't just send a request directly to roblox
O
i did the proxy thing on a different wwebsite
so you didn't actually
receive any data
from this proxy server
yet
other than 400
status code
regardless whether with requests or socket
im trying it now
yes
it works
with those headers and with the same proxy
i think i have a problem with ssl and the proxy
SSL does not care about
about the actual
raw HTTP request
the SSL server that does all the "verifications" can't even see
the HTTP payload
hm
can you post the code
you used for requests
I gotta go offline now, just incase you missed it, here was my answer to the server question
for i in range(20):
t0 = time.time()
response = requests.get(f"https://economy.roblox.com/v1/assets/{item_id}/resellers",headers=head,proxies=proxies)
t1 = time.time()
elapsed.append(t1-t0)
time.sleep(1)
# print(response.content)
there is one difference in the headers
the content type for requests is json while it is text for socket
here is the rule, residential ISP clients in one location can not communicate with residential ISP clients in another location
only when there is some kind of intermediary server (with static IP and also that allows ingress traffic)
so running your own hidden service would fall into that condition, because between you (server) and your friends there is a tor network that acts as that "intermediary server"
but if for the time being no
then no you won't be able to directly talk with your friends directly when both of you are regular residential ISP clients
string= ("GET www.example.com HTTP/1.1\r\n"
"Host: example.com\r\n")
conn = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
conn.connect(("209.127.191.180",9279))
conn.sendall(str.encode(string))
item=conn.recv(4096)
print(item)
print("hi")
here is something else i tried
got a code 400
oop
forgot an extra /r/n
"GET www.example.com HTTP/1.1\r\n"
still a bad request tho
you mean't "GET / HTTP/1.1\r\n"
?
or /v1/assets/63993845/resellers
oh
for example.com /
this is just trying on a different website
will do
i know
but 400 error
is justified
because in first line
of request
you put the wrong value for path
o
502 bad gateway
its a webshare error
my proxy provider
i ran the same code and got autherization error
after fixing the path
you have to be ip verified with those proxies
or have the user and pass
i can give you the authorization
no need
what do you get
from running this
after fixing the path
502
here
and adding extra \r\n
502 means proxy
HTTP/1.1 502 Bad Gateway\r\nProxy-Connection: close\r\nX-Webshare-Error: 502\r\nX-Webshare-Reason: target_connect_refused\r\nDate: Mon, 28 Feb 2022 00:05:51 GMT\r\nContent-Length: 19\r\nContent-Type: text/plain; charset=utf-8\r\nConnection: close\r\n\r\nBad gateway error: '
hi
<ssl.SSLSocket fd=464, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('10.0.0.12', 25820), raddr=('209.127.191.180', 9279)>
issue
yeah
it says webshare error
fair enough
you are using a private proxy
that you paid for
and foolish of me for trying to replicate
now back to the 400
that error is most definitely raised when the actual HTTP request is malformed
yes
but im fairly certain there is nothing wrong with the headers
or anything like that
you are including Authorization header
too right?
for proxies?
no
its ip authorized
alright
well
how would i do that
anyways
have you not looked into the docs?
wait nvm i don't think they even showcase how to actually use a proxy but rather how to interact with your account
@ember ledge once again
they show how to use it for requests
lets go back to the simpler example
with example.com
with example.com?
yeah
correct
uh
ok
so about this
what does the proxies
?
proxies = {
"http": "http://209.127.191.180:9279",
"https": "http://209.127.191.180:9279"
}
no ssl
actually
i sent https to go through http
wait a minute
OK
okokok
u were right
i dont think these proxies support https
LOL
heres what i ran
elapsed=[]
proxies = {
"https": "https://209.127.191.180:9279"
}
time.sleep(1)
for i in range(20):
t0 = time.time()
response = requests.get(f"https://economy.roblox.com/v1/assets/{item_id}/resellers",headers=head,proxies=proxies)
t1 = time.time()
elapsed.append(t1-t0)
time.sleep(1)
# print(response.content)
print(sum(elapsed)/20)
print(elapsed)
when i forced it to use https
it broke
One second, I have internet issues
with this error
no worries
look like again
ok
its good now
correct
the rule is
inside of proxies
variable
both the key
and value
should start with the same protocol
"https": "https://209.127.191.180:9279"
ok so the reason i did that
is i copied the thing
from another project
and this is wrong: "https": "http://209.127.191.180:9279"
where it works
i forgot that i did that
until i looked at it
because it breaks the request
okok
so now i just should remove the ssl stuff
verify=False
try adding this parameter
to your .get()
that wouldnt work
ive tried that before
but ill try it now
should i remove proxies?
no
leave everything as it is
all the SSL errors you get are
errors when you establish a connection with your proxy
via SSL
the roblox domain does not play any part here yet
ik
right
so
elapsed=[]
proxies = {
"https": "https://209.127.191.180:9279"
}
time.sleep(1)
for i in range(20):
t0 = time.time()
response = requests.get(f"https://economy.roblox.com/v1/assets/{item_id}/resellers",verify=False, headers=head,proxies=proxies)
t1 = time.time()
elapsed.append(t1-t0)
time.sleep(1)
# print(response.content)
print(sum(elapsed)/20)
print(elapsed)
same error
should add http proxy instead
and it would work
but setting verify to false makes roblox reject the request
i
stand corrected
aparently roblox does not care about verify=False
maybe only for post requests
only if your proxy only supports http
and not https
then sure
i did that
elapsed=[]
proxies = {
"http": "http://209.127.191.180:9279/",
"https": "http://209.127.191.180:9279/"
}
time.sleep(1)
for i in range(20):
t0 = time.time()
response = requests.get(f"https://economy.roblox.com/v1/assets/{item_id}/resellers",verify=False, headers=head,proxies=proxies)
t1 = time.time()
elapsed.append(t1-t0)
time.sleep(1)
print(response.content)
print(sum(elapsed)/20)
print(elapsed)```
this worked
verify=False
"http": "http://209.127.191.180:9279/",
just leave this one
here
the conclusion from this is that your proxy does not support SSL
once again this is probably mentioned
got this fun little thing
but it still works
yes
how you didn't know that prior
is a wonder
its probably mentioned
in your dashboard
ikr
where you bought the proxies
i didnt buy them
its just free for now
does this break
the code
or just a warning
nope
just a warning
now back to socket
the issue with http proxies is that the actual request you send
to the proxy
is in cleartext
and anyone on the wire can listen on it
yeahhhhhh
but
tbh
it doesnt matter
so there must be proxies supporting https on webshare thingy
alright
the cookies i am sending are on alt accounts so no problems there
this
after you've fixed the path
and added \r\n
you get 502
right?
yup
conn.sendall(str.encode(string))
string= ("GET / HTTP/1.1\r\n"
"Host: example.com\r\n"
"\r\n")
conn = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
conn.connect(("209.127.191.180",9279))
conn.sendall(str.encode(string))
item=conn.recv(4096)
print(item)
print("hi")```
did you mean conn.sendall(string.encode("utf-8"))
whoopsie
but yeah still 502
post the code
again
string= ("GET / HTTP/1.1\r\n"
"Host: example.com\r\n"
"\r\n")
conn = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
conn.connect(("209.127.191.180",9279))
conn.sendall(string.encode("utf-8"))
item=conn.recv(4096)
print(item)
print("hi")```
any body
in response?
like error message
along with 502 status
and also
i tried this adjustment on the roblox example
i also got 502
once again
headers
that you used with requests
did they contain anything that webshare
requires
for webshare
for authentication
uh
no
lets try doing it using user and password
how would i do that with socket
okay hold on
can't believe i missed that
you can't forward https traffic
http?
through an http proxy
so if that code worked
its because it ignored
the proxy
i can try ip api
elapsed=[]
# No reason to provide "https" one because our proxy doesn't support https
proxies = {
"http": "http://209.127.191.180:9279/",
}
time.sleep(1)
for i in range(20):
t0 = time.time()
response = requests.get(f"https://economy.roblox.com/v1/assets/{item_id}/resellers",verify=False, headers=head,proxies=proxies) # trying to access https website via http proxy, the 'proxies' parameter is ignored.
t1 = time.time()
elapsed.append(t1-t0)
time.sleep(1)
print(response.content)
print(sum(elapsed)/20)
print(elapsed)
and this would still return a valid 200 status code
giving an impression that a proxy was used but it wasn't
i assure u it works
you would directly send a request to roblox
i know it works
but it doesn't use proxy
behind the scenes
i just tried it
on ipify.org
here you access http website
here you access https website
2 different things
it would give me an error
