#networks | Python | Page 41

ember ledge Feb 28, 2022, 12:31 AM

#

here

#

lets try it

#

no it wouldnt

#

i just verified it on my machine with a simple test script

#

#

ember ledge Feb 28, 2022, 12:32 AM

#

ember ledge

second line

#

actually

#

uh

#

oh right

#

wait

#

god this is confusing

#

one sec

#

ikr

#

Like

#

idk why this works but it does for all my other projects

#

where using a proxy is necessary

#

ok nvm it is possible to access https website

#

via http proxy

#

its just

#

harder

#

than just sending an HTTP request directly

#

wdym harder

#

more latency?

#

so i just pretended to be a proxy

#

on my machine

#

import requests

proxies = {
    "https": "http://127.0.0.1:4444"
    }

r = requests.get("https://www.example.com", proxies=proxies)
print(r.status_code)

#

and in the other terminal a simple HTTP listener on port 4444

#

right

#

if i try accessing http://www.examle.com here

#

then

#

thats what i get on the http listener side

#

#

just simple http request as it is (from the proxy's perspective)

#

but

#

if i try accessing https://www.examle.com

#

https://developer.mozilla.org/en-US/docs/Web/HTTP/Methods/CONNECT

CONNECT - HTTP | MDN

The HTTP CONNECT method starts two-way communications
with the requested resource. It can be used to open a tunnel.

#

forgot about that one

#

god this stuff is confusing

#

so uh as a note, if you want to use sockets (go the hard way)

#

i need to use sockets ;-;l

#

and connect to https

#

final endpoint

#

then its not as simple as just

#

sending HTTP request directly

#

does that make sense

#

yeah

ember ledge Feb 28, 2022, 12:37 AM

#

ember ledge i need to use sockets ;-;l

are you 100% sure you need to

#

yes

#

there is a 30ms latency drop when i do from requests.get

#

and a lil less if i use request sessions

#

which in my case matters

#

https://github.com/juancarlospaco/faster-than-requests

GitHub

GitHub - juancarlospaco/faster-than-requests: Faster requests on Py...

Faster requests on Python 3. Contribute to juancarlospaco/faster-than-requests development by creating an account on GitHub.

#

uh

#

have you considered

#

this

#

i have not

#

uh

#

try

#

maybe

#

see whats the latency like

#

with it

#

alr ill compare rn

#

string= ("GET http://www.example.com/ HTTP/1.1\r\n"
"Host: example.com\r\n"
"Connection: close\r\n"
"\r\n")

conn = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
conn.connect(("209.127.191.180",9279))
conn.send(string.encode())
item=conn.recv(4096)
print(item)
print("hi")

#

maybe Connection request header was mandatory, try and see if this gives anything other than 502

#

nope

#

still 502

#

edited

ember ledge Feb 28, 2022, 12:42 AM

#

ember ledge ``` string= ("GET http://www.example.com/ HTTP/1.1\r\n" "Host: example.com\r\n" ...

try again

#

what did u edit?

#

changed the path

#

to the one i originally thought was invalid

#

trolled

#

OOOOOOOOOOOOO

#

O

#

O

#

OOOO

#

it worked

#

SHIT

#

so

#

only proxies accept this kind of path, usually it should always start with /

#

m y dumbass

#

as per HTTP documentation

#

so uh

#

thats something i forgot/didnt know about

#

tried ssl on http proxies 💀

#

LOL

#

regular servers would give you 400 if you set path as:
GET http://www.example.com/ HTTP/1.1

instead of valid
GET / HTTP/1.1

#

but yeah proxies are not just regular servers

#

ok

#

so whats the issue now

ember ledge Feb 28, 2022, 12:44 AM

#

ember ledge tried ssl on http proxies 💀

uh

#

ok

#

its possible

#

but once again

#

hard

#

yeah

#

the actual first request you send

#

to the proxy

#

is in cleartext

#

wait

#

yeah the CONNECT one

#

so

#

now time to try on roblopx

#

ok

#

so i have 502 on roblox

#

wait

#

try https

#

with example.com

#

ok

#

string= ("GET https://www.example.com/ HTTP/1.1\r\n"
"Host: example.com\r\n"
"Connection: close\r\n"
"\r\n")

conn = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
conn.connect(("209.127.191.180",9279))
conn.send(string.encode())
item=conn.recv(4096)
print(item)
print("hi")

#

it worked

#

B)

#

well

#

will using https instead of http increase latency

#

thats all that matters

#

thats why im using sendall instead of send

#

i htink

#

uh

#

i dont think it would

#

thats really the only thing that matters

ember ledge Feb 28, 2022, 12:47 AM

#

ember ledge thats why im using sendall instead of send

i mean sendall

#

is same thing as calling send()

#

in a loop

#

isnt sendall tcp and send udp

#

but its for long

#

requests

#

only

#

you can just use send()

#

as your HTTP requests are small

ember ledge Feb 28, 2022, 12:48 AM

#

ember ledge isnt sendall tcp and send udp

no

#

fucking hell

#

where did i read that

#

https and http

#

only use TCP

#

yeah

#

ok

#

thats what i thought

#

idk where i saw that

#

ok

#

so https and httpo work

ember ledge Feb 28, 2022, 12:49 AM

#

ember ledge ``` string= ("GET https://www.example.com/ HTTP/1.1\r\n" "Host: example.com\r\n"...

uh

#

so requests.get

#

would first send a CONNECT

#

request

#

and only then

#

start sending actual request for that path you are interested in

#

unsure how this actually works

#

yeah

#

so

#

for my situation

#

i would connect once

#

and then send

#

instead of using requests.get to connect everytime

#

i could use request sessions

#

but that is still not as fast

#

and

#

idk abuot faster than requests

#

ill try it later

#

i think the only way to check if GET https://economy.roblox.com/v1/ works or not

#

is by sending

#

such a request

#

because basically it depends how proxy server

#

behaves

#

when it sees our HTTP request

ember ledge Feb 28, 2022, 12:52 AM

#

ember ledge i would connect once

maybe

#

even without CONNECT it would work

#

once again there are no guidelines for using webshare proxies directly via SOCKETS

#

so you can just try and see

ember ledge Feb 28, 2022, 12:53 AM

#

ember ledge ``` string= ("GET https://www.example.com/ HTTP/1.1\r\n" "Host: example.com\r\n"...

try this but for roblox via https

#

is basically what i am saying, this might work

#

wait

#

blocksize = 1024 ** 2
# ssl_context = ssl.SSLContext(ssl.PROTOCOL_SSLv23)

sock = socket.socket()
sock.settimeout(5)
sock.connect(("209.127.191.180", 9279))

# sock = ssl_context.wrap_socket(
#     sock,
#     server_side=False,
#     do_handshake_on_connect=False,
#     suppress_ragged_eofs=False,
#     server_hostname="209.127.191.180")
# sock.do_handshake()
string= ("GET /v1/assets/63993845/resellers HTTP/1.1\r\n"
"Host: economy.roblox.com\r\n"
"Content-Type: text/html\r\n"
"X-CSRF-TOKEN: {}\r\n"
"cookie: {}\r\n"
"\r\n".format(i,formatted))
print(string)
elapsed=[]
for _ in range(20):
    t0 = time.time()
    sock.sendall(
        bytearray(string,"utf-8"))
    response, body = sock \
        .recv(blocksize) \
        .split(b"\r\n\r\n", 1)
    print(response)
    content_length = int(response \
        .split(b"content-length: ", 1)[1] \
        .split(b"\r\n", 1)[0])
    while content_length > len(body):
        body += sock.recv(blocksize)
    # print(body)
    # print(response)
    t1 = time.time()
    elapsed.append(t1-t0)
    time.sleep(1)```

#

GET https://economy.roblox.com/v1/assets/63993845/resellers HTTP/1.1\r\n"

#

try that

#

ok

#

got it

#

yessir

#

now

#

@ember ledge

#

lets say

#

i only wanted to receive the first blocksize

#

for every time i got a request from the server

#

how would i do that

#

like i just want the first block of a request

#

define block

#

just N number of bytes?

#

or first row?

#

yes

ember ledge Feb 28, 2022, 1:08 AM

#

ember ledge just N number of bytes?

e

#

recv(N)

#

?

#

ik but i have a loop

#

then don't have a loop

#

and then when i recv more

#

troll

#

i get the next part of the list

#

when i only want the first part

ember ledge Feb 28, 2022, 1:08 AM

#

ember ledge then don't have a loop

true

#

True

ember ledge Feb 28, 2022, 1:09 AM

#

ember ledge when i only want the first part

can you give

#

me

#

example response

#

so i get better understanding

#

ok

#

there

#

thats an example response

#

hi```

#

thats the first block

#

that i defined

#

blocksize = 1024 ** 2

#

is my blocksize

#

i dont reallly need the next blocks of data

#

well

#

i could just make my block size big enough to get all the data?

#

you want

#

to get the first dictionary

#

ending with }

#

correct

#

{"userAssetId":2348375210,"seller":{"id":2024206996,"type":"User","name":"BIoodgoons"},"price":6990,"serialNumber":null}

#

just this honestly

#

the top seller

#

first_block = ""

while True:
  c = conn.recv(1)
  if c == "}":
      break

  first_block = first_block + c

#

something like this?

ember ledge Feb 28, 2022, 1:17 AM

#

ember ledge ```{"previousPageCursor":null,"nextPageCursor":"eyJzdGFydEluZGV4IjoxMCwiZGlzY3Jp...

the response seems to be in json format though

#

yes

#

so you can retrieve this object

#

with resp.json()

#

oh

#

lol

#

forgot we arent using requests

#

hahah

ember ledge Feb 28, 2022, 1:18 AM

#

ember ledge ``` first_block = "" while True: c = conn.recv(1) if c == "}": break ...

ok so this

#

and then json.loads(first_block) which should return a dictionary

ember ledge Feb 28, 2022, 1:19 AM

#

ember ledge `GET https://economy.roblox.com/v1/assets/63993845/resellers HTTP/1.1\r\n"`

so does this work?

#

then

#

i assume

#

yes it does

#

wouldnt .recv 1 at a time be slower?

#

https://stackoverflow.com/questions/17501488/efficiency-of-python-tcp-socket-recv1-vs-recvn

Stack Overflow

Efficiency of Python TCP socket recv(1) vs recv(n)

I'm building a TCP application which uses a newline \n to separate messages of unknown length (but typically < 64 bytes). I'm finding this article to be very useful.

Is the best (i.e. most

#

seemingly

#

you can read like 100 bytes at once

#

and then just check if the ending curly bracket

#

is in those 100 bytes

#

if so then strip everything after, and append everything before to first_chunk

#

kind of thing

#

ok bet

#

lemme try that

#

i gotta go sleep now

#

ping me tmrw in case you need any more help

#

alr bro thanks for the help

#

big help

#

BASED

ember ledge Feb 28, 2022, 2:25 AM

#

ill need your help tomorrow for a few other things i need and some oddities like why my program breaks when using a proxy when it works fine not and other thigns

#

but

#

thats tomorrows problem

ocean wraith Feb 28, 2022, 10:17 AM

#

Hello I have a networking issue.

I have on my network:
my PC - 192.168.1.100
my Camera Dashboard - 192.168.8.199

subnet: 255.255.0.0

My camera and my pc does show up as connected on my router , however my camera dashboard is not responding
does anyone know if these settings are correct or whether I should change something?

ripe igloo Feb 28, 2022, 2:53 PM

#

ocean wraith Hello I have a networking issue. I have on my network: my PC - 192.168.1.100 my...

Your configuration looks correct and you should be able to access 192.168.8.199 from 192.168.1.100 even without a default gateway. However, your broadcast domain is too big, using a /16 subnet is not best practice. If you want your 3rd octect to be different you should do segmentation using smaller broadcast domains like a /24.

ember ledge Feb 28, 2022, 8:53 PM

#

ember ledge ill need your help tomorrow for a few other things i need and some oddities like...

uh

#

im here

ember ledge Feb 28, 2022, 9:03 PM

#

ember ledge im here

ok

#

lets get started >:)

#

blocksize = 1024 ** 2
# ssl_context = ssl.SSLContext(ssl.PROTOCOL_SSLv23)

sock = socket.socket()
sock.settimeout(5)
sock.connect(("209.127.191.180", 9279))

# sock = ssl_context.wrap_socket(
#     sock,
#     server_side=False,
#     do_handshake_on_connect=False,
#     suppress_ragged_eofs=False,
#     server_hostname="209.127.191.180")
# sock.do_handshake()
string= ("GET https://economy.roblox.com/v1/assets/63993845/resellers HTTP/1.1\r\n"
"Host: economy.roblox.com\r\n"
"Content-Type: text/html\r\n"
"X-CSRF-TOKEN: {}\r\n"
"cookie: {}\r\n"
"\r\n".format(i,formatted))
print(string)
elapsed=[]
for _ in range(20):
    t0 = time.time()
    sock.sendall(
        bytearray(string,"utf-8"))
    response, body = sock \
        .recv(blocksize) \
        .split(b"\r\n\r\n", 1)
    # receive rest of body
    # print(response)
    content_length=int(response.split(b"Content-Length: ")[1].split(b"\r\n")[0])
    print(content_length)
    while content_length > len(body):
        body += sock.recv(blocksize)
    # print(body)
    # print(response)
    t1 = time.time()
    elapsed.append(t1-t0)
    time.sleep(1)```

#

this code works without proxies

#

but gets this error on the proxy

#

@ember ledge

ember ledge Feb 28, 2022, 9:22 PM

#

GET https://economy.roblox.com/v1/assets/63993845/resellers HTTP/1.1\r\n"

#

this should only work with proxies

#

no idea how this works when you directly send a request to roblox

#

as to the error

#

    response, body = sock \
        .recv(blocksize) \
        .split(b"\r\n\r\n", 1)

#

i don't like that

#

just save the value of recv()

#

and print it

#

and only then split

#

to get more insight about the cause of the issue

prisma cobalt Feb 28, 2022, 9:27 PM

#

ember ledge ```python blocksize = 1024 ** 2 # ssl_context = ssl.SSLContext(ssl.PROTOCOL_SSLv...

IP LEAK!!!

#

wait, not yours, dw 😂

ember ledge Feb 28, 2022, 9:28 PM

#

ember ledge this should only work with proxies

i changed that part as well

#

didnt mentino that

ember ledge Feb 28, 2022, 10:04 PM

#

@ember ledge

#

ok well my suggestions are above

#

but just remember that GET https://economy.roblox.com/v1/assets/63993845/resellers HTTP/1.1\r\n" is only for proxy

#

if you talk directly to roblox or any other website

#

GET /v1/assets/63993845/resellers HTTP/1.1\r\n"

#

is the only format websites accept

ember ledge Feb 28, 2022, 10:20 PM

#

ember ledge is the only format websites accept

yeah i understand that

#

right so heres my problem

#

 string= ("GET https://economy.roblox.com/v1/assets/63993845/resellers HTTP/1.1\r\n"
"Host: economy.roblox.com\r\n"
"Content-Type: text/html\r\n"
"X-CSRF-TOKEN: {}\r\n"
"cookie: {}\r\n"
"\r\n".format(i,formatted))
print(string)
elapsed=[]
for _ in range(20):
    t0 = time.time()
    sock.sendall(
        bytearray(string,"utf-8"))
    response=sock.recv(blocksize)
    # receive rest of body
    print(response)
    # content_length=int(response.split(b"Content-Length: ")[1].split(b"\r\n")[0])
    # print(content_length)
    # print(body)
    # print(response)
    t1 = time.time()
    elapsed.append(t1-t0)
    time.sleep(1)

print(sum(elapsed)/20)
print(elapsed)

#

this is all i really need

#

but when i go through the loop

#

i just get the rest of the data

#

i just want to farm the first block size

#

yk what i mean?

#

would i have to reconnect in that case?

ember ledge Feb 28, 2022, 10:42 PM

#

so its one request and one response normally

#

now to read the response you use recv(N)

#

in order to not read the whole response and just first 100 bytes for example

#

you just recv(100) and then say "im out" and start with .connect() again

#

its one request and one response per one TCP connection

#

now you can send more than one requests and receives more than one response

#

on a single TCP connection

#

with http pipelining

#

or by setting Connection

#

header to keep-alive

#

import socket

payload = "GET / HTTP/1.1\r\nHost: www.example.com\r\nConnection: keep-alive\r\n\r\n".encode()
s = socket.socket()
s.connect(("www.example.com", 80))

# First request
s.send(payload)
print(s.recv(4086))

print()

# Second request
s.send(payload)
print(s.recv(4086))

#

@ember ledge

#

so it is possible to send multiple requests over one TCP connection, but you have to read the whole response

#

which doesn't fit your use case

ember ledge Feb 28, 2022, 10:52 PM

#

ember ledge would i have to reconnect in that case?

so yes, considering you don't want to read the whole response

ember ledge Feb 28, 2022, 10:53 PM

#

ember ledge which doesn't fit your use case

i can read the whole response

#

the problem is it doesnt let me get another request over the same connection

ember ledge Feb 28, 2022, 11:00 PM

#

ember ledge ``` import socket payload = "GET / HTTP/1.1\r\nHost: www.example.com\r\nConnect...

if you can read whole response then something like this should work

ember ledge Feb 28, 2022, 11:38 PM

#

ill try that out

viral sundial Mar 1, 2022, 3:04 AM

#

can somehelp me with xmlrpc

sleek sand Mar 1, 2022, 8:56 AM

#

i can't connect to my lapptop using tcp over the internet

#

send help

vestal epoch Mar 1, 2022, 9:34 AM

#

try unplugging it and plugging it back in

ember ledge Mar 1, 2022, 9:53 AM

#

sleek sand i can't connect to my lapptop using tcp over the internet

how are you connecting to it, are you using some server in the cloud?

sleek sand Mar 1, 2022, 9:59 AM

#

no TCP

ember ledge Mar 1, 2022, 10:01 AM

#

so both machines

#

you are trying to connect together

#

are regular ISP clients

humble siren Mar 1, 2022, 12:29 PM

#

guys can someone help me a bit to create a list with a bunch of specific ips so i will use them to call a curl request ?

ember ledge Mar 1, 2022, 4:13 PM

#

Frick

ember ledge Mar 1, 2022, 9:14 PM

#

@ember ledge

#

#


sock = socket.socket()
sock.settimeout(5)
sock.connect(("209.127.191.180", 9279))

# sock = ssl_context.wrap_socket(
#     sock,
#     server_side=False,
#     do_handshake_on_connect=False,
#     suppress_ragged_eofs=False,
#     server_hostname="209.127.191.180")
# sock.do_handshake()
string= ("GET https://economy.roblox.com/v1/assets/63993845/resellers HTTP/1.1\r\n"
"Host: economy.roblox.com\r\n"
"Content-Type: text/html\r\n"
"X-CSRF-TOKEN: {}\r\n"
"cookie: {}\r\n"
"\r\n".format(i,formatted))
print(string)
elapsed=[]
for _ in range(20):

    t0 = time.time()
    sock.send(
        bytearray(string,"utf-8"))
    response=sock.recv(4096)+sock.recv(4096)
    # receive rest of body
    print(response)
    # content_length=int(response.split(b"Content-Length: ")[1].split(b"\r\n")[0])
    # print(content_length)
    # print(body)
    # print(response)
    t1 = time.time()
    elapsed.append(t1-t0)
    time.sleep(1)```

bronze umbra Mar 1, 2022, 9:29 PM

#

m 15 for nudes dm me in insta id dare_devil8t4w

ember ledge Mar 1, 2022, 9:34 PM

#

@ember ledge its working

#

and latency is pretty solid too

#

for i in range(20):
    sock = socket.socket()
    sock.settimeout(5)
    sock.connect(("209.127.191.180", 9279))

    # sock = ssl_context.wrap_socket(
    #     sock,
    #     server_side=False,
    #     do_handshake_on_connect=False,
    #     suppress_ragged_eofs=False,
    #     server_hostname="209.127.191.180")
    # sock.do_handshake()

    # print(string)
    t0 = time.time()
    sock.send(
        bytearray(string,"utf-8"))
    response=sock.recv(4096)+sock.recv(4096)
    # receive rest of body
    # print(response)
    # content_length=int(response.split(b"Content-Length: ")[1].split(b"\r\n")[0])
    # print(content_length)
    # print(body)
    # print(response)
    t1 = time.time()
    elapsed.append(t1-t0)
    time.sleep(1)
    try:
        sock.shutdown(2)
    except OSError:
        pass
    sock.close()```

#

anything i should chagne?

#

and also

#

im having trouble installing faster than requests

#

thge problem is i have to reconnect every time

#

which isnt good

#

and actually

#

its slower

#

i forgot to move the first time thing

blazing trail Mar 1, 2022, 9:43 PM

#

ember ledge ```python for i in range(20): sock = socket.socket() sock.settimeout(5) ...

i hope that isn't a public ip address...

ember ledge Mar 1, 2022, 9:44 PM

#

it isnt lmao

blazing trail Mar 1, 2022, 9:44 PM

#

good good

ember ledge Mar 1, 2022, 10:02 PM

#

FileNotFoundError: [WinError 2] The system cannot find the file specified
im getting this error when trying to install https://github.com/juancarlospaco/faster-than-requests

GitHub

GitHub - juancarlospaco/faster-than-requests: Faster requests on Py...

Faster requests on Python 3. Contribute to juancarlospaco/faster-than-requests development by creating an account on GitHub.

#

https://github.com/juancarlospaco/faster-than-requests/issues/173

GitHub

FileNotFoundError: [Errno 2] No such file or directory: 'nimble' · ...

This error is raised when running the script for the first time

#

i just dont know how to fix it

#

this isnt a networking issue

#

but the package is networking related ig?

#

@ember ledge does faster-than-requests even support proxies in the first place?

ember ledge Mar 1, 2022, 10:20 PM

#

ember ledge ```python for i in range(20): sock = socket.socket() sock.settimeout(5) ...

so as i said there is 2 options, you can use the same socket and not call connect() multiple times

#

but then you'd have to recv() whole response each time

#

the other option (which is what you are doing right now)

#

is only reading a small number of bytes

#

and then re-connecting again

#

though you are reading 4096 * 2 bytes now? when previously you only wanted like first 200-300 until you meet the first curly bracket?

ember ledge Mar 1, 2022, 10:22 PM

#

ember ledge FileNotFoundError: [WinError 2] The system cannot find the file specified im get...

yes

ember ledge Mar 1, 2022, 10:22 PM

#

ember ledge so as i said there is 2 options, you can use the same socket and not call `conne...

but yeah you should try both options

#

and see which one is quicker

#

once again you are not utilising the main benefit of the second option, only reading N number of first bytes you need, instead of reading ALL from the response

ember ledge Mar 1, 2022, 10:43 PM

#

ember ledge so as i said there is 2 options, you can use the same socket and not call `conne...

Nah

#

I have to reconnect every time

#

Because it’s a proxy

#

I think it’s the proxy because when I do without I don’t have to reconnect

white basalt Mar 1, 2022, 10:45 PM

#

can anyone tell me

#

what it meant by

#

DNS in practice operates with a set of defined resources record types

#

im just learning the basics of DNS

white basalt Mar 2, 2022, 12:29 AM

#

yea

#

and cname

#

i think i get it a little bit

#

its just hard to imagine where this process is going on rn

#

like they didnt explain this to me during the network layer in the OSI model

#

can anyone explain to me what a DHCP is in simple terms

ember ledge Mar 2, 2022, 1:56 AM

#

ember ledge I think it’s the proxy because when I do without I don’t have to reconnect

oh right

#

that might be true

#

but when talking directly to a server this approach should work

#

wow

#

i was just abuot to ping u

ember ledge Mar 2, 2022, 1:56 AM

#

ember ledge I have to reconnect every time

its fine to reconnect every time thats how HTTP is expected

#

to work

#

but why are you reading so many bytes?

#

and not by 100 bytes

#

in a loop

#

until you get }

#

nah ur right

#

i should do it 100 bytes at a time

#

but still the reconnect thing is annoying ngl

#

also

#

@ember ledge do u you happen to know how to install faster-than-requests?

#

it doesnt seem to work

#

or any other alternatives

#

do you think they would compare in speed to socket?

ember ledge Mar 2, 2022, 1:59 AM

#

ember ledge <@456226577798135808> do u you happen to know how to install faster-than-request...

#

?

#

https://github.com/juancarlospaco/faster-than-requests

GitHub

GitHub - juancarlospaco/faster-than-requests: Faster requests on Py...

Faster requests on Python 3. Contribute to juancarlospaco/faster-than-requests development by creating an account on GitHub.

#

oh nah thats broken

#

here

#

well

#

first off it wouldnt even let me do that in pycharm

#

and when i installed it manually it gives me a nim error

#

seems like theres nothing i can do about that

#

im going to try some github issues that were solved to fix it

#

do you have any other recommendations for low latency requests alternatives?

#

ping me tmrw

#

i am heading out now

#

ah ok

cold monolith Mar 2, 2022, 4:36 AM

#

https://github.com/oremanj/python-netfilterqueue

GitHub

GitHub - oremanj/python-netfilterqueue: Python bindings for libnetf...

Python bindings for libnetfilter_queue. Contribute to oremanj/python-netfilterqueue development by creating an account on GitHub.

#

Hi I am working with netfilter in Python in order to filterate the traffic of htts/http and ssl. I have knowledge of Iptables but unable to know where I can start from becuase I want to test the real time packets in order to analyze

#

Please anyone guide me

ember ledge Mar 2, 2022, 7:07 AM

#

can anyone recommend a tutorial to turn my ubuntu server into a proxy server and connect it through python? thanks

ocean wraith Mar 2, 2022, 11:24 AM

#

Hello does anyone know what might be a solution to this problem? I have a camera that has 192.168.8.199 ip setup in it's config and I can access the camera or change this static ip

finite dock Mar 2, 2022, 11:57 AM

#

simplest solution is to check your wifi router to see if you can assign wifi/lan from the same dhcp pool

whole prawn Mar 2, 2022, 2:45 PM

#

Is the sockets module safe?

#

By this I’m asking if it’s naturally encrypted or do I have to do that myself

prisma cobalt Mar 2, 2022, 3:22 PM

#

whole prawn By this I’m asking if it’s naturally encrypted or do I have to do that myself

Nope, it's not encrypted at all. It's for sending data from point A to point B
Luckily, theres a default python library call ssl I believe which should offer standard grade encryption (the type that's currently used for HTTPS which is very common)
https://docs.python.org/3/library/ssl.html

whole prawn Mar 2, 2022, 3:22 PM

#

Alright, thank you

torpid berry Mar 2, 2022, 6:37 PM

#

I have this code

import requests
session = requests.Session()
session.post(link, headers=Headers, data="{}")

What does 'data="{}"' send in the request? I need to emulate the request using C#, but I can't figure what the session sends as data

ember ledge Mar 2, 2022, 9:20 PM

#

@ember ledge

#

im here

ember ledge Mar 2, 2022, 9:24 PM

#

ember ledge <@456226577798135808>

yessir

#

ok

#

u sure i need to connect every time?

#

doing that increases latency compared to requests

#

and also how do i install faster-than-requests

#

it just straight up does not work

ember ledge Mar 2, 2022, 9:27 PM

#

ember ledge u sure i need to connect every time?

im collecting the whole thing and still have the problem

ember ledge Mar 2, 2022, 10:12 PM

#

ember ledge u sure i need to connect every time?

i gave you 2 options

#

hold on

#

about those faster requests library

#

why did you fail to install it?

#

what error message did you get

#

ok

ember ledge Mar 2, 2022, 10:14 PM

#

ember ledge https://github.com/juancarlospaco/faster-than-requests/issues/173

^

#

this

#

actually

#

i cant even pip install it anymore]

#

@ember ledge

#

and on replit

#

i managed to install it

#

but thne i get this error

#

    return Nimporter.import_nim_code(fullname, path, library=False)
  File "/home/runner/SurefootedUnequaledDirectories/venv/lib/python3.8/site-packages/nimporter.py", line 828, in import_nim_code
    NimCompiler.compile_nim_code(
  File "/home/runner/SurefootedUnequaledDirectories/venv/lib/python3.8/site-packages/nimporter.py", line 558, in compile_nim_code
    cls.ensure_nimpy()
  File "/home/runner/SurefootedUnequaledDirectories/venv/lib/python3.8/site-packages/nimporter.py", line 270, in ensure_nimpy
    out, errors, _, _ = cls.invoke_compiler('nimble path nimpy'.split())
  File "/home/runner/SurefootedUnequaledDirectories/venv/lib/python3.8/site-packages/nimporter.py", line 245, in invoke_compiler
    process = subprocess.run(
  File "/nix/store/p21fdyxqb3yqflpim7g8s1mymgpnqiv7-python3-3.8.12/lib/python3.8/subprocess.py", line 493, in run
    with Popen(*popenargs, **kwargs) as process:
  File "/nix/store/p21fdyxqb3yqflpim7g8s1mymgpnqiv7-python3-3.8.12/lib/python3.8/subprocess.py", line 858, in __init__
    self._execute_child(args, executable, preexec_fn, close_fds,
  File "/nix/store/p21fdyxqb3yqflpim7g8s1mymgpnqiv7-python3-3.8.12/lib/python3.8/subprocess.py", line 1704, in _execute_child
    raise child_exception_type(errno_num, err_msg, err_filename)
FileNotFoundError: [Errno 2] No such file or directory: 'nimble'```

#

and when i try to pip install on pycharm i get this

errant bayBOT Mar 2, 2022, 10:19 PM

#

Hey @ember ledge!

You either uploaded a .txt file or entered a message that was too long. Please use our paste bin instead.

ember ledge Mar 2, 2022, 10:19 PM

#

https://paste.pythondiscord.com/beriwapoqu

#

so

#

yeah

ember ledge Mar 3, 2022, 1:51 AM

#

@ember ledge

ember ledge Mar 3, 2022, 2:36 AM

#

Uh

#

Tomorrow when I wake up

#

I’ve been working sry

toxic idol Mar 3, 2022, 4:45 AM

#

ocean wraith Hello does anyone know what might be a solution to this problem? I have a camera...

the cam needs to be within the same subnet, your PC and router are 192.168.0.1, your cam is 192.168.8.1, it needs to be manually changed to 192.168.0.1 with the IP 192.168.0.199 to be accessible

toxic idol Mar 3, 2022, 6:45 AM

#

are you sure they are?

#

I don't trust pictures from someone that can't set up a cam properly, I want to see actual addresses

#

cuz I've attemped to help people with "wireless" printers cuz they bought a WiFi extender that says it works for printers but printer wasn't even connected to it

ocean wraith Mar 3, 2022, 11:05 AM

#

@toxic idol Thanks for help. 255.255.240 is my subnet that I setup on my router. and the ip's on the picture are the same as the ones in my network. But the issue still exists I'm not sure what do you mean by changing the ip manually as I dont have access to dasboard on my camera.

toxic idol Mar 3, 2022, 11:13 AM

#

ocean wraith <@!423728106801594369> Thanks for help. 255.255.240 is my subnet that I setup on...

is it wireless or wired?

ember ledge Mar 3, 2022, 12:02 PM

#

@ember ledgeaight

#

im available now

ember ledge Mar 3, 2022, 12:10 PM

#

ember ledge im available now

Read my messages I sent before

ember ledge Mar 3, 2022, 12:10 PM

#

ember ledge I’ve been working sry

Ur good bro thanks for the help

ember ledge Mar 3, 2022, 12:19 PM

#

ember ledge and on replit

have you considered

#

running

#

the code locally?

#

what machine are you using?

ember ledge Mar 3, 2022, 4:41 PM

#

I’m using my own pc to test atm

#

But am planning on running it in aws

ember ledge Mar 3, 2022, 4:42 PM

#

ember ledge the code locally?

Wdym tho

crude trellis Mar 3, 2022, 7:02 PM

#

keep getting this error, im being assured that our api is up, not overloaded, and not refusing my requests when these happen, only happens about twice a week

#

any idea on how to fix or what would be causing it?

ember ledge Mar 3, 2022, 7:56 PM

#

ember ledge and on replit

uh replit

#

is online thing

#

isn't it

#

like online IDE

#

yeah

#

i tried it locally as well

chilly granite Mar 3, 2022, 7:59 PM

#

help

#

help

ember ledge Mar 3, 2022, 7:59 PM

#

before asking

#

what error did you get locally

#

when installing

#

that module

chilly granite Mar 3, 2022, 7:59 PM

#

got damn help

#

please

ember ledge Mar 3, 2022, 7:59 PM

#

i couldnt install locally

#

@chilly granite stop spamming and ask a question

ember ledge Mar 3, 2022, 7:59 PM

#

ember ledge i couldnt install locally

i know, but what error did you get

#

o

ember ledge Mar 3, 2022, 7:59 PM

#

ember ledge https://paste.pythondiscord.com/beriwapoqu

this

chilly granite Mar 3, 2022, 8:00 PM

#

@sharp crag

#

@sharp crag

sharp crag Mar 3, 2022, 8:03 PM

#

wifi deadshit

ember ledge Mar 3, 2022, 8:08 PM

#

ember ledge this

@ember ledge

ember ledge Mar 3, 2022, 8:31 PM

#

@ember ledge pip install wheel

#

and try installing again

errant bayBOT Mar 3, 2022, 8:32 PM

#

Hey @ember ledge!

You either uploaded a .txt file or entered a message that was too long. Please use our paste bin instead.

ember ledge Mar 3, 2022, 8:33 PM

#

https://paste.pythondiscord.com/lowiqovice

#

@ember ledge

ember ledge Mar 3, 2022, 9:27 PM

#

this is why i dont use windows

#

no idea sry

tribal pewter Mar 3, 2022, 10:06 PM

#

from ip route, can someone break down what this means in detail
10.24.92.0/24 dev eno3 proto kernel scope link src 10.24.92.40

tough pond Mar 3, 2022, 11:33 PM

#

10.24.92.0/24 -> Networking Block that is being installed on the routing table
dev eno3 -> The Network interface on your computer. Dev means device and eno3 means that is the fourth ethernet port (the firsit is eno0) onboard. If it was an offboard adapter, It'll be ens3.
proto Kernel -> Means that this route was placed due to autoconfig from the OS perspective (anything placed directly on a config file, not by a cli command)

#

scope link -> means that the destination is a local network

#

src 10.24.92.40 - > is the gateway to the network range.

tough pond Mar 3, 2022, 11:34 PM

#

tribal pewter from ```ip route```, can someone break down what this means in detail ```10.24.9...

Is this what you need my friend?

tribal pewter Mar 3, 2022, 11:36 PM

#

tough pond Is this what you need my friend?

yeah, what is a networking block?

#

it's a subnet of something?

tough pond Mar 3, 2022, 11:36 PM

#

Is an IP address Range.

#

/24 means that the network starts at 10.24.92.0 and goes to 10.24.92.255

tribal pewter Mar 3, 2022, 11:37 PM

#

can you elaborate on "gateway to the network range"?

tough pond Mar 3, 2022, 11:38 PM

#

Basically, this line says:" To reach anyone inside 10.24.92.0/24, or between 10.24.92.0 to 10.24.92.255, you need to get across 10.24.92.40.

tribal pewter Mar 3, 2022, 11:38 PM

#

ahh I see

tough pond Mar 3, 2022, 11:38 PM

#

Does that make any sense to you?

tribal pewter Mar 3, 2022, 11:38 PM

#

yeah, that makes sense

#

10.24.92.40 is the ip of eno3 and to reach that network range, we need to go through eno3

#

like if we were to ssh, we would go to that ip

#

through eno3

ember ledge Mar 4, 2022, 12:34 AM

#

print

#

print("HI")```

#

@errant bay

#

,run py print("HI")

#

fuck

ember ledge Mar 4, 2022, 1:03 AM

#

@ember ledge

#

!e

errant bayBOT Mar 4, 2022, 1:03 AM

#

Command Help

!eval [code]
Can also use: e

*Run Python code and get the results.

This command supports multiple lines of code, including code wrapped inside a formatted code block. Code can be re-evaluated by editing the original message within 10 seconds and clicking the reaction that subsequently appears.

We've done our best to make this sandboxed, but do let us know if you manage to find an issue with it!*

ember ledge Mar 4, 2022, 1:04 AM

#

!e

print("hey")

errant bayBOT Mar 4, 2022, 1:04 AM

#

@ember ledge :white_check_mark: Your eval job has completed with return code 0.

hey

marble wharf Mar 4, 2022, 1:56 AM

#

!e
print ("Groovy")

ember ledge Mar 4, 2022, 2:28 AM

#

!e

while True:
print("COOL")```

#

facrs

#

!e

while True:
 print("COOL")```

#

😖

toxic karma Mar 4, 2022, 6:24 AM

#

Hi , I wanted some resources/techniques on finding the actual IP addresses behind a proxy/VPN IP address. Its for a project Im working for, not getting much info regarding this.

frozen drum Mar 4, 2022, 9:18 AM

#

toxic karma Hi , I wanted some resources/techniques on finding the actual IP addresses behin...

the whole point of those is to hide the source. You can look up webrtc leaking which was a thing, but it can easily be circumvented with a proper set up vpn

ember ledge Mar 4, 2022, 11:21 AM

#

in terms of finding an IP of a user hiding behind VPN or proxy, not possible unless the actual proxy or vpn go ahead and leak that info, or they get hacked

#

in terms of finding an IP of a server hiding behind front-end proxy/cache, that is possible

distant cairn Mar 4, 2022, 3:04 PM

#

could someone help me with networking?

ember ledge Mar 4, 2022, 3:58 PM

#

ember ledge no idea sry

What about on Ubuntu?

#

Any other fast alternatives?

#

I might have to use something other than sockets since u have to reconnect every time with proxies

tough pond Mar 4, 2022, 8:30 PM

#

distant cairn could someone help me with networking?

I could try my friend. WDYN?

white basalt Mar 4, 2022, 11:26 PM

#

#

can anyone shorten this ipv6 address

#

its just a practice

#

not real

#

if anyone do know just ping me or pm me

marble slate Mar 5, 2022, 12:13 AM

#

i get this error ```py
Traceback (most recent call last):
File "main.py", line 9, in <module>
json_object = json.loads(str(a.json()))
File "/nix/store/p21fdyxqb3yqflpim7g8s1mymgpnqiv7-python3-3.8.12/lib/python3.8/json/init.py", line 357, in loads
return _default_decoder.decode(s)
File "/nix/store/p21fdyxqb3yqflpim7g8s1mymgpnqiv7-python3-3.8.12/lib/python3.8/json/decoder.py", line 337, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "/nix/store/p21fdyxqb3yqflpim7g8s1mymgpnqiv7-python3-3.8.12/lib/python3.8/json/decoder.py", line 353, in raw_decode
obj, end = self.scan_once(s, idx)
json.decoder.JSONDecodeError: Expecting property name enclosed in double quotes: line 1 column 2 (char 1)

with this code

import os
import requests
import json

a = requests.get("https://www.roblox.com/users/profile/playergames-json?userId=261")

#print(a.json())

json_object = json.loads(str(a.json()))

print(json_object)

print(json.dumps(json_object, indent=1))


how do i fix?

thorn stratus Mar 5, 2022, 1:18 AM

#

sounds like your json is broken

#

what did printing it get you?

#

also why are you converting it to json then to a string and then back to json

stable quarry Mar 5, 2022, 6:53 AM

#

hi

toxic karma Mar 5, 2022, 7:00 AM

#

ember ledge in terms of finding an IP of a user hiding behind VPN or proxy, not possible unl...

from what i read we can figure if the person is using a proxy/vpn but as to figuring the actual IP address it isnt impossible. Suppose in the case of a cybercrime they would have some method to track back. Also this is without the vpn giving up log file on the user

stable quarry Mar 5, 2022, 9:02 AM

#

can someone help me with how to create packets using scapy

stark jolt Mar 5, 2022, 9:04 AM

#

I'm starting a Minecraft server DDoS protection service, I know how to write the API etc, and what I want to do. I just want a helping hand as the task is quite large. If anybody would like to help me, feel free to pm me 🙂

cobalt raptor Mar 5, 2022, 2:06 PM

#

stark jolt I'm starting a Minecraft server DDoS protection service, I know how to write the...

hey, i'm not really good with backend but i can help you with a beautiful web page

stark jolt Mar 5, 2022, 2:10 PM

#

Like a web panel?

#

@cobalt raptor

#

If you would be able to make a web panel that runs off the API that would be amazing.

#

If not, just a typical website would be handy 🙂

cobalt raptor Mar 5, 2022, 2:20 PM

#

both

#

the panel could be made with vue js

#

@stark jolt

stark jolt Mar 5, 2022, 2:46 PM

#

Sick, I'll pm you a link to the discord server

ember ledge Mar 5, 2022, 3:05 PM

#

ember ledge What about on Ubuntu?

sure thing

ember ledge Mar 5, 2022, 3:08 PM

#

toxic karma from what i read we can figure if the person is using a proxy/vpn but as to figu...

from what i read we can figure if the person is using a proxy/vpn - of course, quite easily, when you use either of those, your public IP becomes the one of a proxy or a VPN server and those IPs are all compiled into one list you can find on github already

#

https://github.com/ejrv/VPNs

GitHub

GitHub - ejrv/VPNs: List of datacenter & VPN IP addresses

List of datacenter & VPN IP addresses. Contribute to ejrv/VPNs development by creating an account on GitHub.

#

Suppose in the case of a cybercrime they would have some method to track back. - well as i mentioned if the VPN/proxy gets hacked or leaked from inside, then sure

#

but it's not supposed to happen

gray lagoon Mar 5, 2022, 3:53 PM

#

Hello Guys, Is there any free hosting service like pythonanywhere but has free ALWAYS ON tasks?

ember ledge Mar 5, 2022, 4:55 PM

#

ember ledge ```r.py", line 1150, in find_spec return Nimporter.import_nim_code(fullname,...

@ember ledge I get this on Ubuntu

#

The thing is

#

I don’t have apt

#

On these systems

#

I’m using

ember ledge Mar 5, 2022, 8:43 PM

#

ember ledge ```r.py", line 1150, in find_spec return Nimporter.import_nim_code(fullname,...

uh is this linux subsystem for windows

ember ledge Mar 5, 2022, 8:45 PM

#

ember ledge uh is this linux subsystem for windows

It’s my replit server I think

#

man

#

#

so the error message mentions this

#

nimble seems to be an alternative for pip

#

of some sort

ember ledge Mar 5, 2022, 8:46 PM

#

ember ledge It’s my replit server I think

uh

#

any possibility to rent a 4 dollar linux server

#

for a month

#

I do

#

Btw

#

or perhaps install linux on a USB

#

I use an aws server

#

Or I could use an Ubuntu vm

#

i mean if you already have an ec2

#

on amazon then why not use that one

#

what linux distro is it running?

#

if linux at all

#

ubuntu

#

server

ember ledge Mar 5, 2022, 8:49 PM

#

ember ledge if linux at all

my thing is that even looking through the replies to comments that people using ubuntu cant eve nget it to work

#

hey, im trying to scrape this site:
https://zillow.com/homes/Washington,-DC_rb
and it works fine with postman, but when i try to do a get request in python im getting this error:
requests.exceptions.TooManyRedirects: Exceeded 30 redirects.

#

anyone got a clue how i can fix this?

hoary blaze Mar 5, 2022, 9:36 PM

#

I have a prompt on a site that needs some text to be input in, how can i make that my python script inputs the text in the prompt

ember ledge Mar 5, 2022, 10:40 PM

#

ember ledge my thing is that even looking through the replies to comments that people using ...

ok getting same error

#

so its basically caused by third party

#

library

#

that faster than requests uses

#

uh let me find some alternatives

#

@ember ledge

#

https://stackoverflow.com/questions/62172201/whats-the-fastest-way-to-send-a-post-request-with-python

Stack Overflow

What's the fastest way to send a post request with python?

I tried the normal "requests" module but it's really slow, is there any faster way of sending a POST request?

import requests
from time import time
t = time()
requests.post("https://httpbin.org...

#

urllib3 is being suggested a lot so try that one

#

considering requests is built on top of that

ember ledge Mar 5, 2022, 10:44 PM

#

ember ledge so its basically caused by third party

Yeah

ember ledge Mar 5, 2022, 10:44 PM

#

ember ledge urllib3 is being suggested a lot so try that one

Ok bet

#

Unless there is some way to keep a connection open over a proxy

#

In socket

ember ledge Mar 5, 2022, 10:48 PM

#

ember ledge Unless there is some way to keep a connection open over a proxy

its possible but you have to read

#

the whole body on each request

#

i already posted the code above at some point

ember ledge Mar 5, 2022, 10:49 PM

#

ember ledge i already posted the code above at some point

I know

ember ledge Mar 5, 2022, 10:49 PM

#

ember ledge its possible but you have to read

I know

#

I tried that

#

Didn’t work

#

what was the code

#

again?

#

can you post it

#

Uh

#

I’m not at my pc lemme see if I posted it

#

@ember ledge it’s because it’s a proxy

#

I don’t think u can keep it open even if u read the entire request

ember ledge Mar 5, 2022, 10:50 PM

#

ember ledge ```python for i in range(20): sock = socket.socket() sock.settimeout(5) ...

.

elder mason Mar 5, 2022, 11:05 PM

#

hey Folks ,
i was tryna to use sys.argv with socket connect but it failed

s = socket.socket(socket.AF_INET,socket.SOCK_STREAM)
s.connect(sys.argv[1],port)

s.connect(sys.argv[1],port)
TypeError: socket.connect() takes exactly one argument (2 given)

ember ledge Mar 5, 2022, 11:41 PM

#

.connect() takes a tuple

#

(host, port)

#

as a single parameter

#

s.connect(("www.google.com", 80))

ember ledge Mar 6, 2022, 12:29 AM

#

@ember ledge is it possible to get data again without reconnecting while through a proxy?

ember ledge Mar 6, 2022, 4:02 AM

#

ember ledge <@456226577798135808> is it possible to get data again without reconnecting whil...

wdym data again

#

as i said

#

give me your code again

#

or just use the one

#

i posted before

ember ledge Mar 6, 2022, 4:02 AM

#

ember ledge i already posted the code above at some point

.

#

for i in range(20):
    sock = socket.socket()
    sock.settimeout(5)
    sock.connect(("209.127.191.180", 9279))

    # sock = ssl_context.wrap_socket(
    #     sock,
    #     server_side=False,
    #     do_handshake_on_connect=False,
    #     suppress_ragged_eofs=False,
    #     server_hostname="209.127.191.180")
    # sock.do_handshake()

    # print(string)
    t0 = time.time()
    sock.send(
        bytearray(string,"utf-8"))
    response=sock.recv(4096)+sock.recv(4096)
    # receive rest of body
    # print(response)
    # content_length=int(response.split(b"Content-Length: ")[1].split(b"\r\n")[0])
    # print(content_length)
    # print(body)
    # print(response)
    t1 = time.time()
    elapsed.append(t1-t0)
    time.sleep(1)
    try:
        sock.shutdown(2)
    except OSError:
        pass
    sock.close()```

#

@ember ledge

ember ledge Mar 6, 2022, 12:10 PM

#

https://www.toptal.com/developers/hastebin/ovadenacov.python

Hastebin: Send and Save Text or Code Snippets for Free | ToptalÂ®

Hastebin is a free web-based pastebin service for storing and sharing text and code snippets with anyone. Get started now.

#

@ember ledge

#

this kind of logic

sour bough Mar 6, 2022, 8:01 PM

#

#discord-bots

ember ledge Mar 6, 2022, 8:40 PM

#

@ember ledge ConnectionAbortedError: [WinError 10053] An established connection was aborted by the software in your host machine

#

this happens when trying to stay connected through the proxy

ember ledge Mar 6, 2022, 10:59 PM

#

Hey, anyone would like to join me for a matrix (messaging protocol not the movie) inspired project?

ember ledge Mar 7, 2022, 1:44 AM

#

ember ledge this happens when trying to stay connected through the proxy

@ember ledge and how to send authentication for the proxy

ember ledge Mar 7, 2022, 2:39 AM

#

Authorization header?

#

wasn't it mentioned in the docs for your proxy provider

ember ledge Mar 7, 2022, 2:40 AM

#

ember ledge <@456226577798135808> ConnectionAbortedError: [WinError 10053] An established co...

hm, can you paste the code

#

i think its being aborted on your side not on proxy's side

#

well

#

the thing is

#

i can use request sessions just fine

#

which might just be my solution for now

ember ledge Mar 7, 2022, 2:40 AM

#

ember ledge ```python for i in range(20): sock = socket.socket() sock.settimeout(5) ...

@ember ledge

#

its not the one

#

where you re-use the same connection to send multiple requests

#

oo

#

ok

#

wait

ember ledge Mar 7, 2022, 2:41 AM

#

ember ledge ```python blocksize = 1024 ** 2 # ssl_context = ssl.SSLContext(ssl.PROTOCOL_SSLv...

.

ember ledge Mar 7, 2022, 2:43 AM

#

ember ledge ```python blocksize = 1024 ** 2 # ssl_context = ssl.SSLContext(ssl.PROTOCOL_SSLv...

i dont think you are reading whole response there

#

thats the problem

#

just dont overcomplicate things

#

use

#

foooooooook

#

no

#


    response=""
    while True:
      data = sock.recv(512)
      if (len(data) < 1):
          break
      
      response = response + data.decode()

#

to read whole resp

#

thats not the right code

#

i do read the whole thing

#

lemme just c opy and paste it

#

!paste

errant bayBOT Mar 7, 2022, 2:44 AM

#

Pasting large amounts of code

If your code is too long to fit in a codeblock in discord, you can paste your code here:
https://paste.pythondiscord.com/

After pasting your code, save it by clicking the floppy disk icon in the top right, or by typing ctrl + S. After doing that, the URL should change. Copy the URL and post it here so others can see it.

ember ledge Mar 7, 2022, 2:44 AM

#

https://paste.pythondiscord.com/ahekevepot

#

@ember ledge

ember ledge Mar 7, 2022, 2:48 AM

#

ember ledge https://paste.pythondiscord.com/ahekevepot

do you have Connection: keep-alive

#

header

#

yes

#

in your HTTP request

#

i recently added it

#

and tested it

#

it says win error

#

is that a windows issue?

#

no what matters is the actual message

#

in the errror

#

i assume sock.send

#

breaks

#

on the second loop iteration

#

it coould be that your proxy does not support it

#

no

#

on the 3rd loop iteration

#

but my thing is why would it work for sessions

#

so first 2 work fine?

#

you are actually able to send 2 requests via the same TCP connection?

ember ledge Mar 7, 2022, 2:50 AM

#

ember ledge but my thing is why would it work for sessions

i dont believe sessions does multiple requests over same TCP connection

#

no

#

because

#

as the proxy here doesnt seem to support it

#

wait

#

actually

#

lemme test taht proxy on session

#

no that proxy supports it

ember ledge Mar 7, 2022, 2:52 AM

#

ember ledge https://paste.pythondiscord.com/ahekevepot

you can try connecting to www.example.com here and it will work fine

#

i tested it

#

with a proxy?

#

nah

#

instead of proxy

#

eyah

#

eyah

#

yeah

#

thats the thing

#

it doesnt work with the proxy

#

so it must be the proxy not supporting it

#

and i tested this proxy on request sessions

#

and that works fine

#

once again

#

you wouldnt be able to tell

#

if multiple requests are sent over same TCP connection

#

with sessions

#

oo ok

#

if the proxy doesnt support it

#

it would still work and not raise

#

errors

#

sessions will just send one requests

#

per TCP connection

ember ledge Mar 7, 2022, 2:54 AM

#

ember ledge if multiple requests are sent over same TCP connection

thats not the primary use for sessions

#

they are mostly for storing cookies across whole session

#

well it does reduce latency

#

it keeps the connection open

#

how do you know

#

209.127.191.180
0.37s elapsed for requests.Session() no.
200
209.127.191.180
0.08s elapsed for requests.Session() no.
200
209.127.191.180
0.08s elapsed for requests.Session() no.
200
209.127.191.180
0.07s elapsed for requests.Session() no.
200
209.127.191.180
0.08s elapsed for requests.Session() no.

Process finished with exit code -1

#

the connections are cached for later use

#

on sessions

#

same proxy

#

i think im just going to give up on scraping with socket and just use sessions

#

but i still probably should use sessions for the post request

#

@ember ledge how long can i keep aconnection open?

ember ledge Mar 7, 2022, 3:23 AM

#

ok i found the issue

#

@ember ledge

ember ledge Mar 7, 2022, 3:24 AM

#

ember ledge ```py response="" while True: data = sock.recv(512) if (len...

actually the issue is with this

#

can you just try response = sock.recv(8000)

#

inside of each loop iteration

#

and see if it works

#

first

ember ledge Mar 7, 2022, 3:58 AM

#

ember ledge first

im psure theres a max bytes i can receive

#

in one go

#

and the thing is

#

i still get the same error

#

so

#

i do that right

#

8000 bytes

#

but no matter what i cant read the entire thing

#

and i still get the same error

#

https://paste.pythondiscord.com/ludacukage.python

#

@ember ledge

#

againnnn

#

i did the reading by smaller N

#

proxies

#

again

#

try it

#

ok

#

just adapt the code

#

to proxy

#

elapsed = []
sock = socket.socket()
sock.settimeout(5)
sock.connect(("209.127.191.180", 9279))

for i in range(20):
    # sock = ssl_context.wrap_socket(
    #     sock,
    #     server_side=False,
    #     do_handshake_on_connect=False,
    #     suppress_ragged_eofs=False,
    #     server_hostname="209.127.191.180")
    # sock.do_handshake()

    # print(string)
    t0 = time.time()
    sock.send(
        bytearray(string, "utf-8"))

    response = ""
    response = sock.recv(20000)

    # receive rest of body
    # print(response)
    # content_length=int(response.split(b"Content-Length: ")[1].split(b"\r\n")[0])
    # print(content_length)
    # print(body)
    print(response)

    t1 = time.time()
    elapsed.append(t1 - t0)
    time.sleep(1)

try:
    sock.shutdown(2)
except OSError:
    pass

sock.close()

#

@ember ledge

#

right

#

what about this

#

well

#

it was 8000

#

?

#

but urs is different

#

lemme try it

#

yeah because reading 8K bytes

#

is not smart

#

usually we do it in smaller chunks

ember ledge Mar 7, 2022, 4:04 AM

#

ember ledge usually we do it in smaller chunks

yeah

#

well

#

@ember ledge basically

#

i removed the if not data part

#

so i can loop it

#

an

#

and

#

same error

ember ledge Mar 7, 2022, 4:10 AM

#

ember ledge i removed the if not data part

?

#

what

#

what are you talking about

#

did my code not work with the proxy

#

or what

#

oh

#

nah it didnt

#

so

#

elapsed = []
sock = socket.socket()
sock.settimeout(5)
sock.connect(("209.127.191.180", 9279))

for i in range(20):
    # sock = ssl_context.wrap_socket(
    #     sock,
    #     server_side=False,
    #     do_handshake_on_connect=False,
    #     suppress_ragged_eofs=False,
    #     server_hostname="209.127.191.180")
    # sock.do_handshake()

    # print(string)
    t0 = time.time()
    sock.send(bytearray(string, "utf-8"))

    response = ""
    headers_done = False
    cl_found = False

    # Getting everything before the body
    while not headers_done:
        data = sock.recv(50).decode()

        # Finding Content-Length
        if not cl_found:
            r = re.search("content-length: ([\d]+)\\r\\n", response.lower() + data.lower())
            if r:
                body_length = int(r.group(1))
                cl_found = True

        # Finding beginning of the body
        r_2 = (response + data).find("\r\n\r\n")
        if r_2 > -1:
            print("yes")
            # body_start = len(response) + r_2 + 4
            headers_done = True
            body_bytes_read = len(
                response + data) - r_2 - 4  # In case we accidentally read some bytes already from the body
            body_length = body_length - body_bytes_read

        # if not data:
        #     break

        response = response + data

    # Getting body
    data = sock.recv(body_length).decode()
    response = response + data

    print(response)

    t1 = time.time()
    elapsed.append(t1 - t0)
    time.sleep(1)

try:
    sock.shutdown(2)
except OSError:
    pass

#

copy and pasted it

#

        #     break

#

i removed this

#

dont remove this

#

FeelsWeirdMan

ember ledge Mar 7, 2022, 4:12 AM

#

ember ledge dont remove this

it wont loop otherwise

#

defeating the purpose

#

ye

#

nah

#

its supposed to break out

#

of the while true

#

loop

#

not the main loop

#

but its alright

ember ledge Mar 7, 2022, 4:12 AM

#

ember ledge but its alright

it does break the main loop

#

the sample code i gave works even if you comment

#

out

#

e

#

it doesnt with a proxy

ember ledge Mar 7, 2022, 4:12 AM

#

ember ledge it does break the main loop

it doesnt

#

wait

#

o

#

it just breaks out of while not headers_done

ember ledge Mar 7, 2022, 4:13 AM

#

ember ledge it doesnt with a proxy

hm

#

honestly

#

for i in range(20):
    # sock = ssl_context.wrap_socket(
    #     sock,
    #     server_side=False,
    #     do_handshake_on_connect=False,
    #     suppress_ragged_eofs=False,
    #     server_hostname="209.127.191.180")
    # sock.do_handshake()

    # print(string)
    t0 = time.time()
    sock.send(bytearray(string, "utf-8"))

    response = ""
    headers_done = False
    cl_found = False

    # Getting everything before the body
    while not headers_done:
        data = sock.recv(50).decode()

        # Finding Content-Length
        if not cl_found:
            r = re.search("content-length: ([\d]+)\\r\\n", response.lower() + data.lower())
            if r:
                body_length = int(r.group(1))
                cl_found = True

        # Finding beginning of the body
        r_2 = (response + data).find("\r\n\r\n")
        if r_2 > -1:
            print("yes")
            # body_start = len(response) + r_2 + 4
            headers_done = True
            body_bytes_read = len(
                response + data) - r_2 - 4  # In case we accidentally read some bytes already from the body
            body_length = body_length - body_bytes_read

        if not data:
            break

        response = response + data

    # Getting body
    data = sock.recv(body_length).decode()
    response = response + data

    print(response)

    t1 = time.time()
    elapsed.append(t1 - t0)
    time.sleep(1)

try:
    sock.shutdown(2)
except OSError:
    pass```

#

what is the error

#

you are getting now?

#

ConnectionResetError: [WinError 10054] An existing connection was forcibly closed by the remote host

#

its actually different

#

it went from 10053

#

to 10054

#

what did u change here tho?

#

uh so basically we were reading

#

more than we should have

#

which was my mistake

#

i had to parse each data in smaller chunks

#

and find content-length

#

yes

#

actually

#

that was my original code

#

uh

#

it did somethign like that

#

uh

#

well sry about that then

ember ledge Mar 7, 2022, 4:16 AM

#

ember ledge ConnectionResetError: [WinError 10054] An existing connection was forcibly close...

uh

#

well

#

honestly

#

no

#

no

#

like

#

a while ago

#

and it still didnt work

ember ledge Mar 7, 2022, 4:16 AM

#

ember ledge well sry about that then

ur good lmao

ember ledge Mar 7, 2022, 4:16 AM

#

ember ledge ConnectionResetError: [WinError 10054] An existing connection was forcibly close...

proxy closed the connection

#

hmmm maybe i have to tell hte proxy to keep the connection open?

#

idk

#

so same code works for www.example.com but doens't work when you change host to proxy

#

try a different proxy?

#

so proxy doesnt support multiple TCP requests over same connection is my guess

#

uh

#

sure

#

ok

#

questioon

#

how to do verificaiton

#

of the proxy

#

like this one has user and pass

#

so even without specifying connection header it defaults to Connection: keep-alive when using http/1.1

#

but yeah usually you do it via that header

#

not by any other means

ember ledge Mar 7, 2022, 4:17 AM

#

ember ledge like this one has user and pass

check the docs

#

of your proxy provider

#

heh

#

uh

#

its usually just an Authorization header

#

oo

#

ok

#

that you add to all the other headers of your request

ember ledge Mar 7, 2022, 4:18 AM

#

ember ledge ConnectionResetError: [WinError 10054] An existing connection was forcibly close...

as the error implies its proxy closing the connection

#

i highly doubt they have docs about that

#

so its not an issue on your end

#

no more

ember ledge Mar 7, 2022, 4:18 AM

#

ember ledge i highly doubt they have docs about that

they have

#

docs for requests

#

i saw them

#

before

#

OO

#

i meant for my proxy provider

#

right

#

thats what i am talking about

#

your proxy provider has docs

#

https://ashburnproxies.com/

#

about how to use their proxy with requests

#

i promise u they dont

#

and it mentions autherization headers

#

its a new one?

#

wasnt it

#

proxy

#

uh

#

something

#

before

#

it was webshare

#

before

#

right

#

webshare

#

has docs

#

would it be the same for both proxies?

#

?

#

it depends on the proxy

#

sometimes one proxy could require something like ProxyAuth header

#

while the other one requires AuthorizationForProxy

#

actually

#

ill ask the ashburnproxies guy

#

wait

ember ledge Mar 7, 2022, 4:21 AM

#

ember ledge https://ashburnproxies.com/

your dashboard here probably mentions some token

#

would socks proxies support sending multiple tcp over 1 connection

#

api token of sorts

ember ledge Mar 7, 2022, 4:21 AM

#

ember ledge your dashboard here probably mentions some token

dont have that lol

ember ledge Mar 7, 2022, 4:21 AM

#

ember ledge would socks proxies support sending multiple tcp over 1 connection

i believe so

ember ledge Mar 7, 2022, 4:22 AM

#

ember ledge i believe so

i could test it

#

socks proxy supports all types of application level protocols

#

HTTP, FTP, SMTP

#

while http proxy only supports, well, HTTP

ember ledge Mar 7, 2022, 4:23 AM

#

ember ledge would socks proxies support sending multiple tcp over 1 connection

once again i think even your current proxies should support it

ember ledge Mar 7, 2022, 4:23 AM

#

ember ledge ill ask the ashburnproxies guy

ask the guy if multiple requests over one TCP connection is supported

#

as well

#

bet

#

i asked

#

hes replying to me in real time

#

Proxy-Authenticate: Basic realm="Please enter username and password"

#

i found this in the response headers when not giving authentication @ember ledge

#

the guy doesnt seem to understand my question

#

im telling him im trying to authenticate through the headers

#

but

#

he doesnt know what that means?

#

idfk

ember ledge Mar 7, 2022, 4:39 AM

#

ember ledge he doesnt know what that means?

yeah lol

#

your average support representative

ember ledge Mar 7, 2022, 4:40 AM

#

ember ledge your average support representative

he isnt a support represenatitve

#

he owns the entire thing

#

LOL

#

https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Proxy-Authenticate

Proxy-Authenticate - HTTP | MDN

The HTTP Proxy-Authenticate response header defines the
authentication method that should be used to gain access to a resource behind a
proxy server. It authenticates the request to the proxy server, allowing
it to transmit the request further.

#

hahaha

#

im literally reading that now

#

thats the response header

#

https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Authorization

Authorization - HTTP | MDN

The HTTP Authorization request header can be used to provide credentials that authenticate a user agent with a server, allowing access to a protected resource.

#

thats the request

ember ledge Mar 7, 2022, 4:41 AM

#

ember ledge thats the response header

so not what im supposed to send

#

#

#

well

#

he says it should work

#

credentials is what you are missing

ember ledge Mar 7, 2022, 4:42 AM

#

ember ledge Proxy-Authenticate: Basic realm="Please enter username and password"

uh

#

i assume the value should be username,password and base64 encoded

#

dies

#

close enough

#

its :

#

not comma

#

in-between

#

but yeah that should be it

#

"Authorization: user:pass"
?

#

@ember ledge ^

#

base64 encode user:pass and add Basic

#

Authorization: Basic base_64_encoded

ember ledge Mar 7, 2022, 5:29 AM

#

"Authorization: user,pass Basic base_64_encoded"

#

@ember ledge

ember ledge Mar 7, 2022, 8:41 AM

#

ember ledge "Authorization: user,pass Basic base_64_encoded"

man

#


cred = "user:pass"

encoded = base64(cred)

final = "Authorization: Basic " + encoded

#

@ember ledge

#

pseudo code

#

just think of user:pass as your real credentials

ember ledge Mar 7, 2022, 9:17 AM

#

you silly goose

ember ledge Mar 7, 2022, 10:51 AM

#

Fr

white sage Mar 7, 2022, 5:09 PM

#

Hi, do you know a library to do traversal-nat ?

ember ledge Mar 7, 2022, 6:22 PM

#

hi

#

why can I go to some sites with the tor browser but as soon as I try to do a direct get request with python request over tor proxy on the same site (with the same headers) I get a 403 error

ember ledge Mar 7, 2022, 6:25 PM

#

ember ledge Fr

did it all work?

ember ledge Mar 7, 2022, 6:25 PM

#

ember ledge why can I go to some sites with the tor browser but as soon as I try to do a dir...

is it onion website

ember ledge Mar 7, 2022, 6:25 PM

#

ember ledge did it all work?

I ended up going to sleep and not trying it

#

FeelsWeirdMan

ember ledge Mar 7, 2022, 6:25 PM

#

ember ledge is it onion website

no

#

uh

ember ledge Mar 7, 2022, 6:26 PM

#

ember ledge why can I go to some sites with the tor browser but as soon as I try to do a dir...

how are you using tor proxy

#

with python request

#

post your code

#

!paste

errant bayBOT Mar 7, 2022, 6:27 PM

#

Pasting large amounts of code

If your code is too long to fit in a codeblock in discord, you can paste your code here:
https://paste.pythondiscord.com/

After pasting your code, save it by clicking the floppy disk icon in the top right, or by typing ctrl + S. After doing that, the URL should change. Copy the URL and post it here so others can see it.

ember ledge Mar 7, 2022, 6:29 PM

#

with this website it work but not on all site

#

https://paste.pythondiscord.com/ripiluraze

#

s.proxies = {'http': 'socks5://localhost:9050', 'https': 'socks5://localhost:9050'}

#

you had a typo

#

what ?

#

ah never mind

#

apparently socks5h is a thing

#

have you tried socks5://

#

nevertheless

#

instead of socks5h://

#

i dont see a difference

pulsar hawk Mar 7, 2022, 6:58 PM

#

What is the roadmap of learning networking?

tough pond Mar 7, 2022, 11:11 PM

#

pulsar hawk What is the roadmap of learning networking?

You could use the Cisco CCNA Roadmap. Is a nice background due to the focus of the learning is not based just in Cisco, but learn the concepts from the scratch.

ember ledge Mar 8, 2022, 4:01 AM

#

@ember ledge

#

thanks for the help dude