python-discussion

[golden mortar]

It was decided that we would try it out, but I'm definitely going to vote for getting rid of them again.

[warped lotus]

😆

[true flax]

Hi

[dry pike]

I vote that we get all servers emojis allowed again. idgaf about reactions. gimme my rust emojis back!

[inland karma]

no, missbehaviour gets you banned

as our rules clearly states

[velvet trout]

@dataclass(slots=True)
class ParseContext:
    command: Command
    namespace: Namespace
    token_stream: TokenStream
    end_of_options: bool = False
    operand_index: int = 0

Okay so this class is used when we call parse_arguments() function, the function creates and uses this object and passes to other functions who need the context to work. It is mutated by both others functions and a while loop as we parse arguments.

I was wondering if I should use ParseContext or ParsingContext as a class name?

Are -ing in class names a recommended convention?

[solemn rain]

PC

[velvet trout]

Pythonic Computer

[solemn rain]

Program Counter

[inland karma]

use imerative speac

sp ParseContext

[dry pike]

what

[velvet trout]

?

EnglishErrorForMe

[dry pike]

imperative speech?

[inland karma]

ermm.. trying to live translate 😄

[dry pike]

parsecontext sounds like a function

[inland karma]

yes, like Go West instead of Going West

[final acorn]

@inland karma Are you the server manager?

[inland karma]

what is a server manager?

i do manage servers in my basement and at work

[velvet trout]

good point, although using CapWords convention do helps defferentiating

[inland karma]

but i would say im a programmer

[final acorn]

okk

[solemn rain]

CamelCase

[timid ember]

I thought that was camelcase

[inland karma]

did i answer your question? not sure if i did

[velvet trout]

What's the difference between a programmer and a developer

[warped lotus]

oh sorry boss

[dry pike]

PascalCase

camelCase

[golden mortar]

He is a moderator and part of staff. He's not the only one, though.

[velvet trout]

No no

That camel has first word small

[dry pike]

snek_kes

[sacred cypress]

oh reactions got enabled in this channel?

[pseudo coyote]

I don’t know what it stands for

[solemn rain]

Dynamic programming

[inland karma]

a developer is a professional that makes, creates, manages software applications. the whole life cyclus. from planning, writing, maintaining, design and architecture

[velvet trout]

Why bringle?

[inland karma]

a programmer just builds the software

[grave wedge]

I researched a bit, i think the course version is a bit better as im kind of a visual learner as well: https://www.udemy.com/course/automate/?couponCode=UD100FF

[pseudo coyote]

No I don’t know what that is

[inland karma]

now, i am the fformer, i work as a devloper

but i see myself as a programmer, because that is what i find to be fun and interesting

[inland karma]

your prefered learning style does not matter at all

[solemn rain]

That is a problem solving, which i also dk much about
Its similar to breaking problem down to sub solution which are optimal and storing thr result

[inland karma]

it is more offten a hinderance to your learning

[pseudo coyote]

How does that help with backtracking?

[solemn rain]

Fibonacci is an example for DP

[inland karma]

the best way to learn programming is to read, write and do the work yourself

[solemn rain]

style

[grave wedge]

well when i learnt html and css, i tried multiple courses and books , i did courses and practiced coding it worked

[quartz fulcrum]

def commas(name_of_list):
    customized_string = []
    adding = 'and'
    last_word = name_of_list[-1]
    first_part = name_of_list[0:-2]
    for i in first_part:
        customized_string.append(i)
    customized_string.append(adding)
    customized_string.append(last_word)
    return ','.join(customized_string[0:-1])

print(commas(['ajay','lodhi']))``` what is wrong here

[grave wedge]

yes sir

[sacred cypress]

what's a bringle

[velvet trout]

So the programmer's deal is to implement given as per implementation detail/specification, and the developer does the task of making that specification?

[solemn rain]

not sure

[velvet trout]

Isn't this vegetable 🍆 called bringle in English?

[solemn rain]

sudoku problem is not solvable in polynomial time

[dry pike]

eggplant

[inland karma]

as long as you do the work yourself, you will be alright, but stay away from poor sources, like youtube and AI

[grave wedge]

its also aubergine

[grave wedge]

yes sir, got it

[dry pike]

or aubergine I think EDIT: yes as the guy above said

[inland karma]

the developer does everything,

[velvet trout]

oh

[inland karma]

a junior developer is more a programmer and a developer in training

[velvet trout]

What am i doing right now?

[inland karma]

are you doing software design and architecture?

[grave wedge]

@inland karma where have u been a developer btw if you dont mind me asking?, kind of interested

[solemn rain]

You mean company?

[grave wedge]

ues

yes*

[inland karma]

norwegian healthcare, orthopedic clinicque, accounting busniesses with integrations. Booking systems, large enterprise applications, data engineering and managing large scale platforms. .... ahh. company names..

well.. you have never heard about them

just insert company name here

[solemn rain]

Damn

[inland karma]

unless your norwwwegian

[solemn rain]

So much experience

[inland karma]

and a bunch more 😄

[grave wedge]

nah british

[karmic eagle]

Im learning python recursion

[grave wedge]

crazy, probs a 20 yr experience

[solemn rain]

Nice idk recursion too

[inland karma]

yeah, decades fly past me

[grave wedge]

respect

[karmic eagle]

How much day i should put in learning or practice it

[solemn rain]

Probably years to become expert

[karmic eagle]

In recursion

[inland karma]

closest know thing comparatively is the NHS, Champion League and BBC

[solemn rain]

yeah learn about stacks and trees that may help you a bit
And also about recurrence relations

[grave wedge]

yh yh, i understand, respect though wide variety of sectors

[inland karma]

recursion is tricky to learn, as it all happends in your head, its one off those things that you get after some effort

[velvet trout]

What's that lol. I am currently making cliargparser (i read argparse, google searched things, talked with AI about naming stuff and creating a philosophy for deterministic, zero-ambiguous parsing of cl arguments (since argparse is more of arguments focused than definitive parameters and introduces ambiguity sometimes))

So what am i?

[inland karma]

i have thought recursion in 30 minutes, and they got it, and i have failed after days of trying

[karmic eagle]

Please someone help me how should I learn python

Recursiion how much time I should put in

[dry pike]

taught?

[inland karma]

if you dont know, you are not a developer. you are a programmer! 😄

[inland karma]

ignore spelling mistakes

[velvet trout]

Oh

[karmic eagle]

I usually practice for days on similar topics like data structurea

[solemn rain]

There is no deterministic way

To estimate it

[dry pike]

it's a completely different word tho...

[karmic eagle]

Don't say until u understand it @velvet trout

[inland karma]

how much python do you know already?

[velvet trout]

?

[karmic eagle]

Till recursion

In 2 months

[inland karma]

to have teached in the past tense

[karmic eagle]

I usually practice every topic in depth

Until i master it flow

[solemn rain]

You do leetcode?

[velvet trout]

Did i say something wrong?

[karmic eagle]

No

No leet code

[solemn rain]

Ok

[dry pike]

"until recursion" is meaningless unless we know a specific course you have followed

[karmic eagle]

@velvet trout no

[pseudo coyote]

I swear to god backtracking is so frustrating

[solemn rain]

Its dfs

[inland karma]

dp is hard, do you know it?

[solemn rain]

You know dfs?

[pseudo coyote]

No

[karmic eagle]

No course only self learning topic by topic@dry pike

[inland karma]

so why are you trying to do backtracking now?

you have to learn dp first

[karmic eagle]

No one helps

[dry pike]

then "until recursion" is meaningless because the order of learning things is arbitrary

[inland karma]

backtracking is just another variant of dynamic programming

[pseudo coyote]

I’m making a sudoku solver

[inland karma]

you already have to know dp first

[karmic eagle]

Yes i learn things in order

[inland karma]

i know, but you still have to know dp

[grave wedge]

like what did you self learn using

[pseudo coyote]

How do I learn dp

[grave wedge]

it would be better to explain then

[karmic eagle]

Chatgpt youtube Indian tutorial 10hr

[inland karma]

first step is to learn recursion

do you know that?

[karmic eagle]

Practicing each topic in depth

[pseudo coyote]

I know what it is

[velvet trout]

Why 10hr is it that hard or big

[grave wedge]

could u drop the link to the course, i bet lots of people here probs know it

[inland karma]

then you should learn it

[karmic eagle]

Search on yt CWH python course

[pseudo coyote]

Recursion is basically a function calling itself right?

[grave wedge]

anyone know that one?

[karmic eagle]

Yes

[pseudo coyote]

Or is it something else

[karmic eagle]

How much time i should learn it

[inland karma]

well, you have to do some work while your doing it, thats the DP part

[solemn rain]

There are some other things to consider like base case. Or else it would keep calling for ever

[pseudo coyote]

Yeah Ik

[karmic eagle]

Yesss

[pseudo coyote]

So learn dp before backtracking basically?

[inland karma]

yeah, once you really know dp, the explaination off backtracking makes the implementation self evident

[quartz fulcrum]

@inland karma

[inland karma]

yeah

[solemn rain]

basically these all are problem solving strategies

Recursion itself follows another strategy called divide and conquer

Am not that expert at algos

[quartz fulcrum]

def commas(name_of_list):
    customized_string = []
    adding = 'and'
    last_word = name_of_list[-1]
    first_part = name_of_list[0:-2]
    for i in range(len(name_of_list) - 1):
        customized_string.append(name_of_list[i])
    customized_string.append(adding)
    customized_string.append(last_word)
    return ','.join(customized_string)

print(commas(['ajay','lodhi']))```
i used debugger to do this...but still one error is there

[solemn rain]

You start with divide and conquer then greedy like dfs then dp then backtracking

[grave wedge]

try this: def commas(name_of_list):
if len(name_of_list) == 1:
return name_of_list[0]
return ', '.join(name_of_list[:-1]) + ' and ' + name_of_list[-1]

print(commas(['ajay','lodhi']))

@quartz fulcrum

[inland karma]

what is the error?

[quartz fulcrum]

ajay,and,lodhi prints this

[solemn rain]

Name exposed

[quartz fulcrum]

i dont want two commas

[quartz fulcrum]

np

this is our hub

[inland karma]

why do you have two commas?

[grave wedge]

this line is issue wizard: return ','.join(customized_string)

[quartz fulcrum]

idk

[inland karma]

then you must figure out that

[grave wedge]

and u never used this: first_part = name_of_list[0:-2]

[inland karma]

how do you add commas?

[quartz fulcrum]

oh, it is because of join

[grave wedge]

yes

[inland karma]

yes, its because you add the commas everywhere

[quartz fulcrum]

what should i do then

[grave wedge]

copy my version and see if that works

[inland karma]

only add commas where you need them

[grave wedge]

def commas(name_of_list):
if len(name_of_list) == 1:
return name_of_list[0]
return ', '.join(name_of_list[:-1]) + ' and ' + name_of_list[-1]

print(commas(['ajay','lodhi']))

[edgy kraken]

Hey @grave wedge!

Please edit your message to use a code block

```py
print('Hello, world!')
```

This will result in the following:

print('Hello, world!')```

[quartz fulcrum]

that is easy

i shld ve thought about it

[pseudo coyote]

When is Len even useful?

[quartz fulcrum]

but didnt click

[pseudo coyote]

It seems so useless but I see it so much

[inland karma]

when you want to check the length of something

[grave wedge]

no problem

[pseudo coyote]

I know, but when does that actually come up

[pallid garden]

suppose you have a bunch of users

[inland karma]

when you want to know the length of something

when else?

[pallid garden]

and they send in some requests

you put the requests in a list

[quartz fulcrum]

show him practical side

[pallid garden]

and then you use len to find out how many requests you received

does that make sense?

[inland karma]

print(f'Your password is {len(password)} long')

[quartz fulcrum]

@pseudo coyote

[iron sable]

Guys what project should i know basics but my function knowledge is kinda cooked

[pseudo coyote]

Thanks

That’s actually the perfect example

[inland karma]

make a calculator where each operator is its own function

[iron sable]

Len is like showing how many letters in input?

[pseudo coyote]

Yh

[grave wedge]

hello world😆

[inland karma]

yes for a string, that is what length means

[iron sable]

Ok

[iron sable]

I did that

[iron sable]

And this one

[iron sable]

What about int?

[keen fox]

thats beautiful python lol

[inland karma]

an int has no length, its just a number

[iron sable]

Okay

[grave wedge]

make a bank entrance program which uses a wide variety of variables

[keen fox]

what would be the length of an int?

[iron sable]

I think its kinda complex

[iron sable]

I think its The int itself

Im not sure

[keen fox]

len(454000) what should this output

[pallid garden]

give it a try

[solemn rain]

Try

[inland karma]

then its pretty usless

[iron sable]

454000

[keen fox]

i mean what do you think

[keen fox]

i didn't ask u

[keen fox]

?? u try it if u want

[pallid garden]

oh my bad

[iron sable]

45400

454000*

[keen fox]

why would u think that?

[solemn rain]

Well you asked a question and we gave an answer

[keen fox]

didn't ask u

[iron sable]

Its an integer

[pallid garden]

did you try it?

[solemn rain]

This is general

[iron sable]

Im on my phone rn

[inland karma]

well, integers have no length, length does not make sense

[keen fox]

so you think len() would just print it?

[grave wedge]

@iron sable maybe a Grocery Store Bill Calculator

[inland karma]

because, in what base is the integer

[iron sable]

Or i dont understand the question cause my first language is russian

[velvet trout]

How cool would it be if i override the built-in data type's __len__ dunder from CPython source code and install it on all my college computers

[solemn rain]

wow you can put that in your resume

[iron sable]

Or it will print 6

[inland karma]

integers are immutable, you cant change them. you can make your own CustomInt

[keen fox]

why is that?

[iron sable]

Cause of the…idk how to explain like its starts counting how many numbers are there s

[velvet trout]

Yeah i know int type does not impls dunder len method, but what about list, tuple, dict and such types to change their dunder len to show random lengths

[pallid garden]

programming is not a guessing game

just say you dont know if you dont know the answer

[solemn rain]

randomized search algorithm enters

[pallid garden]

it's ok to not know something

[inland karma]

no i mean, the source is immutable, you cant monkey patch the object

[quartz fulcrum]

if it is written like '89000', then len work otherwise you cant put len on numbers\

[inland karma]

none of the builtins let you do that

[iron sable]

I think i know but idk how to say it in English

[inland karma]

so you would have to change the C source, build python again, and then you can

[iron sable]

K

[pallid garden]

well, once you are on your pc, try running len(5) and see what happens

[velvet trout]

What if i edit that in cpython source code and then reinstall python, like you know the code hosted on github in objects/ directory

[iron sable]

Sure

[thorn mist]

whys the server profile looks like a clown

[keen fox]

yes and for that the object must be an iterable you must need a starting point and an ending point
the number 45000 starts at 1 2 3 4.... 45000 so why would its len be 6? and its pretty obvious that its 45000 so why not just let it be?

[solemn rain]

what if i just made another function called len()

[inland karma]

just fork the repo and change the language, you are free to do so

[solemn rain]

That is overriding?

[velvet trout]

Wonder if it will even install or initialize the interpreter if i do that lol

[inland karma]

in python, behaviour is connected to functionality. len is the function, __len__ govern the behaviour

[velvet trout]

Yah, but i was thinking about doing evil things changing dunder len to give random number and install it on all college computers

🥰

[quartz fulcrum]

it happened to me twice.
i feel like i forget everything about git and github second time

[solemn rain]

Why

[inland karma]

you can change the name off a function, you cant change the behaviour if the source is immutable. you can however make your own object and implemnt your own __len__

[iron sable]

Alr

[inland karma]

use git every day, and you will learn it over time

[grave wedge]

can python be used for quantitative analysis?

or quant trading

[solemn rain]

Yeah you mean i make a class and copy the len dunder code from string class

Or inherit string and override

[velvet trout]

Because they made me learn stuff which is Offtopic to my core stream subjects + teaching 8085 assembly microprocess (i am not interested tbh, i wanna learn something like C (low) or python (high)) + made me write code on white paper instead on an IDE + their education/teaching methods sucks, its rote learning/marks based.

So taking my revenge

[quartz fulcrum]

but from where do i start..
i have a udemy course on it. it is lengthy and learning from it will be too boring

[solemn rain]

Aahh

[velvet trout]

Also included one of the regional/commonly spoken human language in IT/CS streams. I was like, what— why should i care about language over computer languages.

[runic flower]

fun fact: this won't work

[inland karma]

when you write a small amount of code, commit it, when you add another small amountl commit that

and continue to do that

[velvet trout]

😢

[wintry tendon]

Lol 😂

[quartz fulcrum]

are you from punjab?

[velvet trout]

"fun" — that ruined my fun 😔

[solemn rain]

we also did have stuff like physics chemistry mechanics engineering drawing electrical electronics civil mechanical basics all for our 1st year

But not languages

Like regional ones

[velvet trout]

What stream?

[solemn rain]

CS

[frigid coral]

heyy anyone learning backend framework??

[wintry tendon]

Ye

[velvet trout]

Wanna know? This was a newly implemented Education policy. Imagine being this dumb that you kill stream subjects by adding additional off topic and unrelated fluff.

imma continue making evil plans using my little programming language

[quartz fulcrum]

do it..if your heart is clean...you will regret it

[velvet trout]

I was heartful all the time.
I understood the hard way:
Peace was never an option.

Fight for your right

[silver plover]

US universities will have three sets of courses you need; university wide general education, college specific core clssses, and then major specific requirements. Gened often includes things like writing& languages

[solemn rain]

What is your question, do you have any?

[frigid coral]

like i am starting to learn flask so i want a roadmap to proceed

[silver plover]

Start with the flask tutorial, they have a nice one

[quartz fulcrum]

are you 7 years old?

[frigid coral]

18

[inland karma]

be kind

[quartz fulcrum]

oh fine

[inland karma]

dont look for roadmaps, they are mostly all terrible

the tutorial meantioned above is a great place to start

[frigid coral]

ok there own docs??

[solemn rain]

Flask docs seem good.
But if you want a insight of how backend dev works you can check https://roadmap.sh/backend

But don't try to learn everything they list
Choose what tech you like

[quartz fulcrum]

i just guessed hig age from the name

[velvet trout]

I see. I just want to learn CS purely, no extra Subjects i have done lots of subjects for last 12 years (Indian system). Now i just want to focus, not get distracted, especially for students like me who are bad at learning things than average students. 🥀

[silver plover]

Yah, I hear you. This is a common complaint about University curricula. I felt the same. I think Englands system is cleaner; last three years are major only, afaik

[velvet trout]

On my way to transfer to england

[solemn rain]

I think now a days especially in web dev people are first choosing tech then making projects with the tech

In old days first problem might be solved and tech were used as tools

To solve the problem

i am part of modern day dev

[silver plover]

Yes and no, picking a new stack is hard and risky. There's safety in sticking with well known patterns

My companies have been burned by decisions. My last company went through 4 web frameworks in 10 years

[pallid garden]

i feel like authentik is overkill for my use case @inland karma

[inland karma]

its an overkill for me aswell... but it wasnice to test out

[pallid garden]

heh

[inland karma]

i just wanted a mock server

[pallid garden]

right right

[solemn rain]

I am planning to buy raspberry Pi

Planning for too long now

[grave wedge]

why not arduino

[inland karma]

its sensible to have a small computer if you need one

[grave wedge]

oh right okay

[deft raptor]

he looks like the spinning cat

[solemn rain]

Can it run a web server and like 4-5 websites

[solemn rain]

Who

[deft raptor]

duck

[velvet trout]

That looks like a ball shaped

[charred tusk]

It is rather spherical

[solemn rain]

Looks like a fish

[velvet trout]

Ball fish

[solemn rain]

Puffer fish

[velvet trout]

Fisher puff

[compact horizon]

What's up guys

[unkempt sage]

does anyone know a good guide for python asyncio?

from basics to advanced

[inland karma]

https://bbc.github.io/cloudfit-public-docs/asyncio/asyncio-part-1.html

python docs andrealpython is also good

[velvet trout]

Wait BBC news channel also teach Python?

[unkempt sage]

lol

ok thanks a lot guys

[stiff saddle]

yeah lol

yes

[crisp jay]

In my country this is expensive as shit

[wintry tendon]

Btw guys why does python use snake case

[naive ermine]

pretty easy to read

[craggy trench]

underscores look like proper spaces

[wintry tendon]

🤔

[craggy trench]

python still uses PascalCase for classes and unseparated lowercase for many stdlib modules though

[wintry tendon]

Pascal and camel case look good

[silver plover]

stdlib has a mix of weirdness for historical reasons.

[naive ermine]

I don't like the look of Pascal or camel case

[craggy trench]

i like how PascalCase distinguishes classes from functions and normal instances

[naive ermine]

Agreed, its a nice distinction

[wintry tendon]

C++ also has snake case in its std lib, hard to read

[velvet trout]

Because it is python

[naive ermine]

this_reads_better_to_me vs ThisReadsBetterToMe

[wintry tendon]

Ye, that's good

[velvet trout]

🐍

[wintry tendon]

Ye maybe sometimes snake case looks good

[velvet trout]

"sometimes"?

[wintry tendon]

Snake case looks good only on long names

[shrewd pine]

I imagine there is a sliding scale when divulging AI usage is required/appropriate

[lofty sable]

Having snake case in python gives you just an opportunity, you don't lose anything.

[shrewd pine]

like, for full on vibe coding it's obvious, for "I used one copilot completion" less so

[vale plume]

ehats the occasion for the new banner? looks sick

[velvet trout]

so? I am not sure which to pick. Both sounds good

[gaunt badge]

See #changelog

[vale plume]

cheers, that’s nice

[shrewd pine]

or when there was something AI generated that was inspected and mostly all rewritten in the process

code of Theseus

[quick compass]

Hello

I am trying to write a program which takes a linkendin address and returns everything in the js page. But the bot detection keeps stopping my code. How can I bypass it?

[ocean ridge]

Isnt there a free api for linkedin?

[quick compass]

which?

[ocean ridge]

searcch it up 🙃

[quick compass]

I only came across playwright, which is paid.

[ocean ridge]

um do u know whats api

[runic flower]

I was writing a function and the IDE autocompleted a default return, made me jump, I don't use any AI in my IDE. It was a good line though so I kept it. Probably from a legacy copilot install.

[quick compass]

something that acts as a communicator between two programs?

ah. sorry mb, Its proxycurl. not playwright.

[golden mortar]

!rule 5

[edgy kraken]

Rules

5. Do not provide or request help on projects that may violate terms of service, or that may be deemed inappropriate, malicious, or illegal.

[golden mortar]

Bypassing the anti-botting measures of Linkedin would definitely violate their ToS

[quick compass]

I didn't know that. But it can be done using proxycurl api. how so?

[golden mortar]

It's not about whether it can be done or not, it's about us having a rule to not assist with projects that are inappropriate, malicious, illegal or violating the ToS of other services.

I. e., we can't give you any information about that type of project.

As a side note, it seems like Proxycurl was shut down in 2025 following a lawsuit. From Linkedin, for not following their ToS.

[runic flower]

That last bit is hysterical in context.

[golden mortar]

Isn't it?

[quick compass]

I got it as a work project. What am I supposed to do now. 😐

[runic flower]

(a) ask your boss, (b) ask linkedin how to use their api
other than that you're crossing over into asking for help with something we can't help you with,

[tribal drum]

Hi all , I have been working on extensive doj parser and research pipeline in nativelab for parsing Epstein case files and segregate them as per timing event sequence missing sequence and hypothesis generation , for this currently m choosing , multi context pdf chunk reasoning with back context support , based on continuation keywords and event consolidation , any other technique for retrieval of large files without loading all in context with maximum back context would be helpful. Also needed help on image identification models for low quality images , if any .

[warped lotus]

https://discord.com/channels/267624335836053506/1035199133436354600

[tribal drum]

Ohk

[warped lotus]

the name of a parser

made me excited

[golden mortar]

Linkedin has official API services, but the kind of data you want is probably not gonna be available for free, especially not to another business. Maybe look into what those options are and explain them to your boss.

[warped lotus]

is that socratis on ur pfp?

[golden mortar]

If they literally want you to steal data through scraping, then sorry, your employer is pretty shady and we just can't help with that.

[golden mortar]

No, it's Epicurus.

[dry pike]

Epicure? they made a statue of the CEO of the seasonings and dips company?

[quick compass]

Yeah, thanks for the info. I informed my supervisor. Will look into paid apis. Also, just a curious question, is web crawling illegal everywhere or just linkedin?

[golden mortar]

It's not typically illegal anywhere, but the vast majority of major services forbid it in their ToS

[tribal drum]

Depends on bot.txt

[quick compass]

Bot.txt?

[golden mortar]

Some types of automated crawling can be permitted, like search engine and AI indexers, but it depends on the service what they allow

[golden mortar]

https://en.wikipedia.org/wiki/Robots.txt

[tribal drum]

Yeah each site manages a bot,txt for bots to state terms of crawling and limitations or to deny crawling before scraping any site you must make sure your model complies with bot.txt

Yeah robots.txt

[golden mortar]

But even if they permit stuff like that, they virtually never permit you to just scrape a bunch of raw data off of them for free

[runic flower]

I'm using a simple one:
User-agent: *
Disallow: /*
Allow: /

But I also have more rules against bots implemented as cloudflare rules.

[quick compass]

Ah, I was told to store everything on the domain to a json file. Including the redirects, image names etc. Which thanks to you now I know is illegal.

[tribal drum]

I did some research rss feeds are mostly allowed and msn has pretty lease for bots

[pallid garden]

to a json file?

nevermind the legality

[golden mortar]

Well, not illegal. It's not a crime. But it can lead to getting banned as well as civil lawsuits.

[pallid garden]

the format is questionable

[tribal drum]

Format conversion can be done later after scraping anyway

[pallid garden]

who stores things on a website to a json

[tribal drum]

His use case might be different ,who knows ?

[pallid garden]

yea but that doesnt stop me from questioning their engineering decision

[tribal drum]

Anyways , i don't think that question makes sense for data storing , it maybe his convenience and other framework dependencies

[golden mortar]

I mean, if I'm just experimenting with a new feature, I might just dump data in whatever way

[tribal drum]

True

[golden mortar]

You don't gotta architect everything upfront

[pallid garden]

surely you would just dump the files into a directory

[golden mortar]

Yeah, maybe? Maybe a json file?

[quick compass]

Not really my place to question in what format they need as an output. Besides, why is it so wrong? I always get my outputs in json format.

[tribal drum]

if it works , it works

[pallid garden]

a website consists of multiple files, some of which would be image files, which are binary files, which is weird to store in json

[tribal drum]

Cmon , why would someone store binary in json he would just store img path in json that's it

[quick compass]

Oh. Well, I wasnt told to get the images par se. Just the names?

[tribal drum]

Also json is one of most easily supported library for python and other frameworks

[pallid garden]

json is not a library...

[golden mortar]

You could store text data and metadata in json, and paths to any binary blob files.

[pallid garden]

yea, makes more sense

eh whatever it's against TOS

[tribal drum]

import json

[compact token]

A there a python tutorial here

[hidden laurel]

is it possible to get python syntax in VScode WITHOUT access to internet?

[tribal drum]

Yeah

Download extention

[hidden laurel]

please refer to the ladder half of the message

[tribal drum]

Hmm i don't get it

[hidden laurel]

i can use a USB pendrive, but not administrator perms

[tribal drum]

Ohk , download extension file from GitHub install using file

[undone grove]

hi

[pallid garden]

json is a format, python's standard library merely provides a convenient way to parse and serialize objects to that format

[tribal drum]

Cmon,

[pallid garden]

it's not like a python creation/thing

is what im saying

[hidden laurel]

k. so git clone https://github.com/microsoft/vscode-python.git in the USB? What would need to be done after that?

[tribal drum]

Yeah ik

[tribal drum]

Download .vsix

[hidden laurel]

& where is that?

[tribal drum]

https://marketplace.visualstudio.com/_apis/public/gallery/publishers/ms-python/vsextensions/python/latest/vspackage

[velvet trout]

Is namedtuple recommended for a tuple of size 2 and you do not want to remember indexes (although easy to remember but still) ?

I do not want to create a full-fledged class for this short thing.

i think i am beginning to find good use cases of using namedtuple?

[tribal drum]

code --install-extension ms-python.python.vsix

Replace with ur filename

[undone grove]

what is the bitwise operator

[hidden laurel]

uhhh arm64?

[celest osprey]

things like shifting bits etc

[tribal drum]

How will I know that 😭

[cyan scaffold]

Operators that operate on bits from one or two integers

[golden mortar]

@tribal drum I think their point is that you can't download things if you have no internet.

[tribal drum]

Nah he's downloading from another device and transferring via usb

K guys if anywant wants help dm me, m going off now

[hidden laurel]

this

[cyan scaffold]

There must be a tutorial somewhere on installing Visual Studio Code extensions from a file

[dry yacht]

What's the difference between a namedtuple and a dataclass?

[tribal drum]

Btw vsix works on any platform until u have vscode

[runic flower]

one is a tuple with names, the other is a shortcut for making classes

[velvet trout]

You asking to know my knowledge or wants to learn the knowledge from me?

[pallid garden]

@inland karma i just realized that i mixed up bearer token and access token, which are 2 separate things

EDIT: nevermind, apparently bearer is a type of access token

[velvet trout]

It is just like normal tuple but you access it through name (attributes) instead of subscription protocol / indexing [int].

That's how i understand them

I know the essence of dataclasses but I couldn't provide a formal proper explanation.

[cyan scaffold]

I think they allow accessing via index like normally, just that they also allow accessing via attributes

[dry yacht]

I want to learn the knowledge from you, cause I just checked the docs for named tuple just now and I was not sure how they differ in practice

[velvet trout]

Oh so both, nice.

I am pretty new too, to namedtuple

[cyan scaffold]

Same I don't use them

[runic flower]

One thing to keep in mind: namedtuples, are tuples, so the values are immuatable

[dry yacht]

So are dataclasses, no?

[ocean ridge]

hey. Who is interested in game dev but game dev feels to complex / hard so you dont learn it??

[runic flower]

no, dataclasses are just shortcuts for classes. you can make them immutable but they are not by default

[ocean ridge]

Im thinking of making a game engine for my next project

[cyan scaffold]

Dataclasses are classes with useful default implementations for things

[velvet trout]

Well, i only find the need to use namedtuple is when i want less overhead of a custom class and a dataclass, where you just want a container to be able to unpack, index and access by name (namedtuple also provides a good repr like dataclasses), it will make sense if you first begin a tuple and eventually realize automatically that it could become a good namedtuple too.

That's based on my experience. No idea if there's a detailed list or recommend good practices about when to use them vs when not.

[dry yacht]

Ah okay, so a namedtuple is basically a frozen dataclasses.dataclass?
In terms of usage, obviously not under the hood

[velvet trout]

It's just a slightly advanced tuple, and i like that about it

[ocean ridge]

??

[velvet trout]

I view dataclasses as rather concise'nification of custom classes with a good repr, init & comparison methods. They're for simple states or facts. Go with a full fledged when you find yourself conflicting with dataclass or doing almost everything a dataclass already does.

[runic flower]

there's also simplenamespace.

import json
from types import SimpleNamespace

x = {'data': {'type': 'msg_logs',
  'content': [{'msg': 'We be nesting data'}, {'msg': "lot's of data"}]}}

logs = json.loads(json.dumps(x), object_hook=lambda d: SimpleNamespace(**d))
print([x.msg for x in logs.data.content])
# ['We be nesting data', "lot's of data"]

[velvet trout]

Yup! That's another niche & cool thing

[golden mortar]

That's why I got into programming in the first place

[prime sorrel]

so about this shift key...

[ocean ridge]

Personally i ahve faced a lot of issues with multiplayer in godot. I just feel like its easier to do certain parts in threejs but setting up camera, llighting all that in threejs is hell. So i was thinking maybe my own game engine for creating web / mobile / desktop games with javascript as scripting language

[golden mortar]

It sounds like you are trying to do too much at once.

If you are struggling with camera and lighting, don't do a multiplayer game.

[ocean ridge]

Its not too much. Babylonjs exists

[golden mortar]

Do simpler games to start with and work your way up.

[ocean ridge]

No.. I am kinda pretty good in godot

[golden mortar]

So you don't need to learn everything at once.

[ocean ridge]

Just that lighting and camera setup is hard in threejs compared to godot due to lack of ui for game editor

and godot's multiplayer is hell.

I mean its easier than unity / unreal but its really hard compared to syncing things easily by js

I was thinking maybe a game engine Ui on top of babylon with ui similar to godot and js as scripting lang

[golden mortar]

It feels like a common pattern for aspiring gamedevs to spend all their time developing game engines instead of games.

[ocean ridge]

I mean its going to benefit a lot of game devs...right??

[golden mortar]

If you finish it and it turns out to be popular. Which is a very long shot.

[ocean ridge]

I dont know if its going to benefit a lot of ppl tho. thats why im asking before starting the projet

[pallid garden]

ot0

[golden mortar]

Mispaste

[pallid garden]

basically dont trust microslop

[ocean ridge]

fact.

[ocean ridge]

anyways, should i try doing this project? I was thinking normal js scripting + unreal's bp system / scratch based block coding For no code game dev.

[golden mortar]

It's your choice.

I'd rather make an actual game myself, even if I have to make compromises with my tooling sometimes.

[pastel sluice]

For perspective I've been writing my own 3D FPS game in Python, and it's challenging, but I appreciate how dirty I'm getting my hands

[ocean ridge]

i mean i dont really have any projects to do 🤣

I mean if it helps a lot of ppl if done correctly why not give a try

[pastel sluice]

If you see an instance of something you want to try yourself, absolutely give it a shot, and by all means stand on the shoulders of giants

[pallid garden]

stand on the shoulders of gAInts?

[dire shale]

New hotkey discovered!

In PyCharm, place your key on a line

Hit command + up-arrow, it'll move the whole line up one!

[pastel sluice]

no

[tropic temple]

vv useful. works in most common editors

[pallid garden]

i jest, but people here would genuinely do that

[pastel sluice]

Sadly, yes

[dire shale]

Hey, AI is a useful tool

If used correctly

[pastel sluice]

It's useful when it is being applied in a highly narrow, specific, refined way, and the vast majority of how we currently use it does not fit these criteria

It's being sold as the Everything Machine, when it should be the This One Very Specific And Narrowly Defined Thing Machine

[dire shale]

I take this stance: AI can only ever reflect for own expertise back at you. For now anyway. It can't do anything you can't do, and you've got to be wary of it's pitfalls

[velvet hull]

https://discordapp.com/channels/267624335836053506/291284109232308226/1481648276863193191

[craggy trench]

the thneed

[dire shale]

But if used by an expert in a field, it can make a huge difference in both the amount of legwork you have to do, and the quality of your work

[charred tusk]

That title didn't fit in a browser tab so we had to shorten it

[pallid garden]

yea it just so happens that the things that I do are not mainstream enough yet for LLMs to be able to handle them

[dire shale]

RIP

[hallow schooner]

i use arch btw

[dire shale]

💀

[pastel sluice]

It's a light novel title, you see

[pallid garden]

it's almost as if experts in the field are doing things that LLMs cant

/jk im not an expert

[dire shale]

_>

[prime sorrel]

@inland karma thanks for attempting a solution to the programming problem after i left. alas, I did try it (read it all sadly) and it wasn't quite what I was looking for. I am up for the help channel thing alas, I must confess, I don't know how it works on discord.

[dire shale]

In a year, I've given myself a solid understanding of ML. It took a lot of work and there's still a long way to go. But I could never have gotten this far without ChatGPT to work through it with me

[pastel sluice]

I don't know about never but it's good you were able to get constructive use out of it.

[golden mortar]

#❓｜how-to-get-help

[charred tusk]

https://support.discord.com/hc/en-us/articles/6208479917079-Forum-Channels-FAQ

[dire shale]

And this project I'm working on? A programming language - an executable graph based ontology of the MtG universe, with bytecode interpreter and other bells and whistles? Again, AI has been invaluable

[charred tusk]

oh yeah you're that guy

[dire shale]

I prefer to think of the AI not as a separate thing, but as myself as a cyborg. As with any skill, I have to take care to maintain best practice. I can't just throw a problem at an LLM and expect a solution

But I'm so, so tired of people complaining about AI, it being evil, it being useless

If it isn't helping you, you're using it wrong

[charred tusk]

Its not that prescriptive

[robust ledge]

Many of us are equally tired of being told we're "using it wrong". So you use your LLMs and I won't. We can co-exist that way.

[inland karma]

no worries, its easy to get sidetracks into overlapping details! 😄

[inland karma]

true, details matter when you want to make things work

[pallid garden]

im still confused btw

[pastel sluice]

Most of the complaints about it being evil and useless and so on are not unfounded, for the same reason that complaints about coal mining being dirty and environmentally devastating are not unfounded (even though you can make an argument its energy was how we kickstarted the industrial revolution). People are not happy about these things for good reasons, and we're only going to get good things out of them by being skeptical, not hagiographic

[pallid garden]

still reading docs

[inland karma]

many confusions are connected to understanding what things really are, and HTTP is the same way. HTTP is just sharing text with each other

some text has meaning, like 400 and GET

but another thing that has meaning is Bearer

[pallid garden]

go on?

[inland karma]

so wwhen you use GET you know that this is a http verb for getting data

[dire shale]

All of that is fair. For as much use I get out of AI, it's a many faceted phenomenon. Some parts are great. Others, horrifying. If nothing else, it's civilization-changing. All (well, most) of the well reasoned critisism of AI, I agree with

[inland karma]

when you use Bearer you already know that the next piece of data is a auth token

[pallid garden]

mhmm

[inland karma]

and all the tools we use are just abstraction on top of just sharing text messages back and forth

authentication has a predetermied methods for how many times text is shared back and forth, and what those text messages must contain

[pastel sluice]

A general question: if you are writing a Python package, and you have elements in it that are not themselves Python files (text or image resources), what's a best practice way to handle those things as part of the package?

[dire shale]

But its getting really old for me listening to people - not really smart coders like on here - but people who genuinely have no idea what they're talking about complaining about the technology that I use every day. And more than that, complaining about the people who the technology

[pallid garden]

there doesnt seem to be a good resource with some standardized examples of what message gets sent to which endpoint and what the endpoint responds with

[inland karma]

and we connect words on top of this to explain more in detail, but HTTP is simple in nature. and AUTH is simple as well, it has multiple simple steps

[inland karma]

OAuth2 is a spec, and if you use that you must follow the spec

[pastel sluice]

WAIT https://docs.python.org/3/library/importlib.resources.html I think I found something relevant.

[pallid garden]

yea, the spec is like 20 pages long

which i mean... sure

but like

it's 20 pages

[inland karma]

and the part you use, maybe JWT, is only a few pages

[pallid garden]

is it?

hold on

[charred tusk]

As someone who's been a solo dev my entire life, having something that can respond immediately whenever I want has been enormously appreciated
Its given me multiple "why don't you just..." suggestions that have halved some of my scripts

But I spent Tuesday trying to get it to write a feature and it fell of the rails practically immediately, it got stuck in a loop with itself because I renamed a bunch of stuff and it couldn't decide whether to ask me to use the new names or the old names

It takes skill to "prompt engineer" and get the context explicit enough for the AI to be able to "stay focused"
And the time it takes to do that is commonly longer than the time it would have taken for me to just spit out the code myself

[inland karma]

i dont remember, been ages since i read it

the point im making, is, its simple, the abstraction on top might make it harder to use or understand

[robust ledge]

The build backend you use can assist with this as well. Hatch, for example, automagically includes all files withing the ./src tree. setup-tools has knobs to configure for what non .py files to include within the ./src tree.
Outside of the ./src tree is where I fall over with knowledge.

[runic wharf]

Hello

[dire shale]

Out of curiosity, which model do you use?

[pallid garden]

well at least i have something "interesting" to figure out

[charred tusk]

Open AI's latest when using the .com
Opus 4.6 when going through Copilot

[inland karma]

so when you are working with authentication, details matter. and when you know which part you are using, follow the rules lied out in the spec, and/or your auth library/provider docs

[dire shale]

Fuck yeah. I know it's a rich ecosystem, but like a simp I just went for the most familar name when I started using AI, and ChatGPT has never done me wrong

[inland karma]

if you have curl installed, you can do everything from the terminal

[dire shale]

Not really, anyway

[inland karma]

no fancy python auth lib needed

[pallid garden]

in this case i am the provider

which is the problem

before this ive only used oauth to authenticate

[inland karma]

and you have to follow the spec the client expect. thats the OAuth2 spec (if your using oauth2)

[pallid garden]

not implement it myself

[pastel sluice]

I'll have to experiment. I want to build some facilities into pydeploy that allow you to bundle non-Python resources with a Python package elegantly with your deployable. Right now you list directories to copy over that are placed just off the root of the deployed artifact directory, but I'm open to better solutions wherever they may be.

[pallid garden]

yea i am both the client and the auth

[inland karma]

the client side, thats authlib

[pallid garden]

this one is trivial

[inland karma]

!pypi authlib

[edgy kraken]

Authlib v1.6.9

The ultimate Python library in building OAuth and OpenID Connect servers and clients.

Released on <t:1772437441:D>.

[charred tusk]

I hate that thing

[pallid garden]

hmm usually i use the provider's own api client

[charred tusk]

Do they support SQLA 2.0 yet?

[pallid garden]

like e.g. i use the gcp api client instead of authlib to connect to google drive

[robust ledge]

They're all doing the same Oauth2 handshake. Just different baked in parameters.

[charred tusk]

Yeah half the time I just end up doing it manually

[robust ledge]

Roll your own OAuth2 handshake once. It's interesting how simple the flow actually is. Demystifies is.

[pallid garden]

roll your own oauth provider meanwhile...

[robust ledge]

If you want, just don't use it in prod.

[charred tusk]

Still not terrible

[inland karma]

yes, important to demystify things

[pallid garden]

it's just for experimentation mostly

[inland karma]

thats why i think every engeneer should raw dog their own http server

[charred tusk]

what dog? 🐕‍🦺

[robust ledge]

Which is the best reason.

[pallid garden]

rust book!

[charred tusk]

lol, after the political fiasco with the US goverment, recursive_error convinced me to try Claude Code so I just canceled my Gippity sub and subbed to Claude instead
I get Gemini included with my other Google subscriptions too
So time to try something new and spice it up a bit

[robust ledge]

It's impressive how much clarity it can bring.

[inland karma]

oh yes. same with anything that is protocol based

i made a mc server by hand. also demystified a lot

[pallid garden]

the "understanding what it really is" is actually not that problematic

it's the standardized implementation

[dire shale]

I love the pure programmer energy in this statement

[pallid garden]

that is the difficult part

[charred tusk]

@robust ledge have you done your own OAuth provider?
I did it myself after getting angry at how terrible Authlib was
It only supports Client Credentials at the moment, but thats all I need to implement myself, interactive auth will go through Microsoft's massive SSO ecosystem

[pallid garden]

this might just be for experimentation, but i still want to make it realistic from the client side

[dire shale]

And you know, that's a great suggestion. I... struggle... with servers. Building one from scratch would probably be a good exercise

[shrewd pine]

||which one?||

[pallid garden]

isnt authlib a client library

[charred tusk]

yes

[charred tusk]

its both

[robust ledge]

Provider, no. Client, yes.

[inland karma]

i did this with SAML2 a few years ago. provider and client....

[pallid garden]

wait what

[golden mortar]

The thing I'm tired of are people on both sides being irrationally for or against it. There are both real benefits and real risks/harms, and it's when people turn a blind eye to or overinflate either one that it's frustrating to me.

[charred tusk]

https://docs.authlib.org/en/stable/flask/index.html

[pallid garden]

i was reading this page https://docs.authlib.org/en/latest/oauth/2/intro.html and i lol-ed at the drawing

[robust ledge]

Nuance and whimsy are starving for use in the Internet.

[shrewd pine]

instructions unclear, ingested rustonomicon

[inland karma]

https://docs.authlib.org/en/latest/flask/2/authorization-server.html

[charred tusk]

yessss

[inland karma]

or if you use django

https://docs.authlib.org/en/latest/django/2/index.html

[charred tusk]

https://github.com/impressdesigns/manageorders-sdk/blob/main/src/manageorders_sdk/client.py#L27-L40

[edgy kraken]

src/manageorders_sdk/client.py lines 27 to 40

def _update_token(self: Self) -> None:
    """Update the OAUTH token."""
    if self.token_expires_at > datetime.now(tz=UTC):
        return

    auth_dict = {
        "username": self.username,
        "password": self.password,
    }
    response = httpx.post(f"{self.base_url}/v1/manageorders/signin", json=auth_dict)
    response.raise_for_status()
    data = response.json()
    self.token = data["id_token"]
    self.token_expires_at = datetime.now(tz=UTC) + timedelta(hours=1)```

[charred tusk]

Ignore the id_token

These dumbasses got AWS Cognito to do everything for them and still couldn't read the instructions

[inland karma]

lol

[charred tusk]

But yeah, with OAUTH, its "just" a POST /token (for machine auth)

[inland karma]

😄

and callback

well, if your doing sso that is.

[pallid garden]

omg authlib already has an example: https://github.com/authlib/example-oauth2-server

how did i not see this

[inland karma]

read all the words

[charred tusk]

try reading

jinx 🍻

[inland karma]

dont skip anything

[inland karma]

😄

[pallid garden]

i prefer reading the code...

[inland karma]

then the author failed

you gain knowledge from docs, if the docs is made correct

[pallid garden]

ok yknow the docs is pretty good

[inland karma]

learning, understanding, goals and information

[dire shale]

This. This is why I'm so raw

[pallid garden]

compared to what ive read for the past day

[dire shale]

Its such a big issue, and I'm so tired of people claiming to have a solid answer to it

[inland karma]

here, if the docs have this, its a good doc

[pallid garden]

which are OPs trying to sell their ready made

[quartz fulcrum]

i am surprised how a small code can do very productive things

[ocean ridge]

ikr

[quartz fulcrum]

whats this

[inland karma]

that is a good measurement of quality

many lines of code is not a good measurement

[inland karma]

an overview over what good documentation looks like

[pallid garden]

ok the authlib's docs is actually very good

[inland karma]

i find it well made

[pallid garden]

though it might also be that i have been gathering and synthesizing loads of resources from all over the place so now it's like a revision/summary

[inland karma]

i do like to read docs, its a great hobby

same with reading PRs in reverse order

[pallid garden]

i seldom like to read docs that are made by companies with a product to sell

[bronze dragon]

like really old already-merged ones?

or like from the most recent comment?

[inland karma]

merged ones yes. but not reversed in that way, newest to oldest

[inland karma]

yes

understanding of a codebase is something you get from reading PRs, last changed to least. it gives you an idea of what just broke, what just had to change and what the current syle is

though, probably not a hobby everyone should have

[pallid garden]

what? reading PRs?

[inland karma]

reading a codebase trough PRs... its often not important to know a random codebase

ofc, a codebase you work on, super important, but thats not a hobby, thats work

[bronze dragon]

I agree
I've done that before when I want to understand some open-source project

[inland karma]

im reading tinygrad right now

[pallid garden]

Authlib provides a ready to use AuthorizationServer which has built-in tools to handle requests and responses:
there we go

that's where the good docs end

[ocean ridge]

i smell clanker content

[pallid garden]

it's not

this is authlib's docs

i mean i guess this was what i really wanted?

but now im too deep into trying to understand how it works

behind the scenes

[inland karma]

im investing time into getting my AI model to actually work this time

because if pewdipie can make his own ai model, then for sure i should be able to! 😄

[quartz fulcrum]

hey i used to watch pewdiepie's videos

micecraft series is best

[inland karma]

though, i actually think he is a genius under all the joke-fasade

[shrewd pine]

(I saw his stuff before he got large)

[inland karma]

i have only seen his python developer journey

[pallid garden]

"i was doing it before it's cool"

(/jk)

[shrewd pine]

I remember his early amnesia days

[quartz fulcrum]

i was his viewer when logan paul became popular

[shrewd pine]

I haven't followed him for many many years, I recall he's a university dropout who turned full time video game streamer. Granted it's been a while since then

[empty quail]

Sup snek

or snekians

or what do we say

[shrewd pine]

oh god, it's been 15 years?

[empty quail]

snek family 🐍

[inland karma]

https://www.youtube.com/watch?v=aV4j5pXLP-I

[shrewd pine]

I know he made that video recently, yeah

[quick compass]

Hello

Is it possible to make my own api which scraps the web(specifically linkedin) for data?

[shrewd pine]

he spent his early years shouting at chairs and barrels

[raw bramble]

Ooh, love the new icon, what’s it for?

[grave tree]

Holi!

[pallid garden]

#changelog

[raw bramble]

Thanks I always lose it 🙏

[inland karma]

linkedin does not allow you to scrape their data. i think they have an API you can buy access to though. but have not looked into that much lately

[shrewd pine]

apparently it was like 2 weeks ago

[pallid garden]

following your question from just now, do you know what is an "API"

[inland karma]

Im sorry @grave tree if i was abrasive yesterday, it was not my intent to come of like that. Ill try to be better in the future! 😄

[grave tree]

ah yeah, I don't our branding calendar was updated for the 2026 dates for the holidays that shift

[quick compass]

Thought so. Told the same thing to my supervisor. And this was his reply

"Whats the use if we cant make our own and have to depend on external API

RATHER USing then figure out how they are doung and try to make one" 🙄

[pallid garden]

no really, do you know what an API is?

[quick compass]

Mostly only the definition? It's a program which is used for communication between two softwares?

[pallid garden]

yes, in this case, you are trying to access some data from linkedin right?

[inland karma]

linkedin and other large sites, like amazon spend a lot of resources by dynamically change their frontend code so you cant scrape it. and they also enforce login so you have to accept their terms and not only relly on robots.txt, and their terms say you are not allowed to scrape their site.

[pallid garden]

the "accessing data" is being done by a program that linkedin writes, that's why it has access to the data

[ocean ridge]

i know how to bypass it but ima gatekeep :/ (mainly cloudflare anti bot and other anti bot things)

[pallid garden]

it's like asking if other people can write a program to access your company's database @quick compass

[inland karma]

https://www.linkedin.com/developers/

[shrewd pine]

(an api is not a program, but rather how programs interact)

[dire shale]

I'd tweak this, and say it's a collection of functions inside a program purpose built to query and answer queries from another program

[shrewd pine]

(a programmable interface that applications can use)

[inland karma]

!pypi linkedin-api-client

[edgy kraken]

linkedin-api-client v0.3.0

Official Python client library for LinkedIn APIs

Released on <t:1688054772:D>.

[dire shale]

But, I'm no expert. I think it's in an important distinction, whether an API is a program which wraps around another program, or, a set of protocols inside a program

[harsh anchor]

an interface for applications that is programmable?

[inland karma]

only thing that matters is that it is an interffface you can program

[quick compass]

Atleast from the above reply you know they dont want to use paid apis.

how fcked am i?

[pallid garden]

if you want to let other people interact with your company's database in a limited way, you can write a program that does the "accessing data" part, and then let other people call this program through an API.

hence, it's a logical contradiction to suggest that you can write an API to access linkedin's data yourself

[shrewd pine]

one could say that

[inland karma]

my favorite api is pathlib

[shrewd pine]

is that an "is" or a "has a" relation?

[harsh anchor]

it's quite logical to write an API that provides methods to scrape LinkedIn

[pallid garden]

why dont you help out then

since you are saying it's possible

[bronze dragon]

[harsh anchor]

[bronze dragon]

creating an API doesn't necessarily imply direct access to the database

you could have (and indeed there are many) APIs which wrap a scraper

[quick compass]

Question, why is scraping not permitted? It's not like I am accessing restricted data, just what's on their page.

[pallid garden]

...

[harsh anchor]

they don't want you to

[grave tree]

Because you are now accessing just their data and not their advertising or the other things on their platform. They don't like that

[tribal drum]

Guys, you must see this before working
LinkedIn's Terms of Service (ToS) strictly prohibit the use of bots, crawlers, scrapers, and other automated methods to access, search, or scrape data from the platform. Automated activity that violates these terms often leads to account restrictions, "LinkedIn jail," or permanent bans. [1, 2, 3, 4]
Key Aspects of LinkedIn’s Robot/Automation Policy:

• Prohibited Activities: The User Agreement explicitly forbids using any software or scripts to scrape data, copy profiles, or send mass messages/connection requests.
• Detection Measures: LinkedIn uses sophisticated AI to detect non-human behavior, including rapid profile viewing, unnaturally consistent timing between actions, and unusual IP addresses.
• Penalties: Violating these policies can result in temporary or permanent bans. In some cases, users are required to submit government-issued ID to restore access.
• Permitted Activity: Only authorized API integrations or manual browsing are generally allowed. [1, 2, 3, 5, 6, 7]

Permitted Automated Crawling & IndexingLinkedIn has specific, separate terms for authorized crawling (e.g., for search engines):

• Express Permission Required: Automated crawling without prior written permission is strictly prohibited.
• Respect Robots.txt: Approved bots must abide by robot exclusion headers.
• Usage Constraints: Data collected must be used only for search indexing in public search engines. [8]

Risks and Enforcement

• Account Restrictions: Using browser extensions or automation software to mimic human activity often triggers warnings, such as "You've reached the weekly invitation limit" or "We've noticed unusual activity".
• Data Privacy: Automated scraping is often considered a violation of both LinkedIn's user agreement and privacy legislation.
• Lawsuits: LinkedIn has taken legal action against entities that scrape data, treating such actions as a direct threat to their business model. [1, 3, 6, 10, 11]
]

[inland karma]

it is very resource intensive, and they want to keep their data on their own platform in most cases. its all policy of use, not law. we do repect any sites terms of service

[pallid garden]

the whole point why they are asking for an API is because they are told it's against TOS to scrape the frontend

[grave tree]

Automated activity that violates these terms often leads to account restrictions, "LinkedIn jail," or permanent bans
lmfao, I love that they have a name for it "LinkedIn jail"

[pallid garden]

im not saying it's technically impossible

[inland karma]

this is what i summed up earlier, but yes. your not allowed to scrape linkedin

[pallid garden]

im saying it's impossible according to the TOS

[harsh anchor]

you said it was a logical contradiction

[pallid garden]

can i ask you, what's the point of this API?

[bronze dragon]

I mean, does the supervisor think that if you scrape you aren't dependent on them somehow? If anything you are more dependent.

[tribal drum]

Anyways half of linked in is ai generated , better you can use deepseek ai

[bronze dragon]

truth

[grave tree]

I think we can take it down a notch, y'all are just talking slightly past each other.
zehata is answering the question of "can you build an API without violating LinkedIn's ToS?". Other folks are looking at the question without the ToS restriction

[quick compass]

I guess he doesn't know what he is talking about. Ill have to talk to someone else and make him understand.

Btw hi Bharadwaj. 👋

[pallid garden]

why would you build a frontend scraper, provide an API for that scraper and then go and use this API to get the information from the page. that's just the same as building a frontend scraper without the API part

[bronze dragon]

hi 👋

[tribal drum]

And it will be blocked eventually 🤣

[pallid garden]

^ this is the logical contradiction i was trying to say, i apologise if i phrased it wrongly earlier

[quick compass]

So bottom point is, there is no "allowed" way apart from using paid api, is there guys?

[tribal drum]

Anyways if you wanna scrape linked in the illeagel way is use chromium chrome driver then use beautiful soup , and np.random to generate request intervals this way you can skip detection

[dark wharf]

yh you are right.

[pallid garden]

doesnt work anymore

[dark wharf]

but we can use not allowed way.

[tribal drum]

Ohh ,

[quick compass]

Done that. Didn't work.

[pallid garden]

chrome driver gets caught easily

[bronze dragon]

we shouldn't discuss that here anyway

[tribal drum]

Second option you have is use mediapipe and on screen ocr

[pallid garden]

for all intents and purposes, consider it impossible

[dark wharf]

hmm, you can use another way

there are very many method

[tribal drum]

But all are illeagel

[quick compass]

Eh, its okay. I dont want to do something stupid and get my ip banned.

[tribal drum]

Every advice given here by me was for " educational purposes only."

[dark wharf]

you will not get you ip banned.
you can use proxies

[pallid garden]

that doesnt work in a court of law, just so you know

[dark wharf]

but it's ok

[grave tree]

pls y'all, our rules 😩

[tribal drum]

It works , aldr faced it once

In ipo

[dark wharf]

ok

[brittle merlin]

happy holi python??

[velvet trout]

unauthorized scraping discussion in my python discussion chat?

[pallid garden]

must be a day that ends with -day

[humble jungle]

Saint Patrick day

[tribal drum]

Nah I was giving education on mediapipe and chromium usage

[peak egret]

pi day when

[oblique spindle]

[pallid garden]

the real pydis .topic list:

• is AI going to take over software engineering in a <some duration of time>, <some famous person> said so
• how do I scrape <some website with TOS disallowing scraping>
• python is so slow, <some other programming language> is better
• what youtube channel should i watch to learn programming
• ive been making <some impossible software>, i have met a small bug, can someone DM me, im willing to pay
• *sends .topic command* ".topic list is so boring, we need to update it"

[tribal drum]

Python was genuinly slow not only cuz of language but cuz of abstraction layer using a custom linux with stripped off unnecessary features and binary penalty helps a lot

[bronze dragon]

don't forget "I'm looking for partners in my business/project"

[proud nacelle]

'Any Python professionals in here that can mentor/coach me'

[steel pulsar]

please use pink logo server its better

[tribal drum]

Not free currently buddy but can help you with errors if any

[proud nacelle]

Who cares about the logo

[grave tree]

I'm just glad the mojo topic has died a swift death, that was such an annoying period of time

[steel pulsar]

me

[proud nacelle]

The what?

[jade robin]

is mojo replacing python?

🍿

[verbal citrus]

unironically an LLM based .topic command would be nice ...it should pull the recent N discussion in #python-discussion and then return a new topic ...prolly a good case for LLM

[grave tree]

when mojo was first "released"/advertised, I swear every 5th message was about it replacing python

[surreal knot]

it'll come back next february :-)

[proud nacelle]

Wth is Mojo 😂

[grave tree]

Typically when the topic command is used, it means that we need a new topic different from the previous discussion

[tribal drum]

Python is for ml , now making a calculator in it and comparing to c and java is like , rest we understand 🙂

[proud nacelle]

I've never heard of Mojo in my entire life

[verbal citrus]

yeah exactly, so the LLM could make sure the topic is always different based on the context

[jade robin]

i read a lot of blog posts/podcasts on it

[cerulean ravine]

as mentioned, the topic died a swift death.

[tribal drum]

M using multiple llm to complete the Epstein files and reconstruction of their doj

[pallid garden]

im sure that would be totally accurate

[verbal citrus]

it's a custom lang meant to "speed up" python ai dev workflows

[tribal drum]

Anyways , if you compile a python app with nuitka , speed improves ,

But yeah for large apps and library time is 📈

[proud nacelle]

I don't really get why people hype programming languages so much in the first place. Kinda comes across as LARP most of the time

[pallid garden]

but hey, at least it did what it promised: it was faster than python...
...at dying

[harsh anchor]

it is used, though

[verbal citrus]

by who? mojo?

[warped lotus]

mojo is not OS?

I have never seen a programming language which is not open source

wait maybe apple script

[proud nacelle]

C++?

[warped lotus]

it is not open sourced???????

wait what hold on

[grave tree]

What part of C++ is not open source?

[pallid garden]

apparently zilog isnt

[warped lotus]

now what is that

[pallid garden]

mainframe programming language

or am i mistaken

i was indeed mistaken

it's some microprocessor

[cyan scaffold]

Didn't Microsoft close C# and then people forked it?

[sonic cargo]

do someone know how can i verify me ?

[warped lotus]

wym by that

[verbal citrus]

you need read #voice-verification properly

[bronze dragon]

do you mean voice verification?

[sonic cargo]

okey i just need time in chat

thaks

[proud nacelle]

Certain (popular) compilers can be not open source like MSVC

[silver plover]

Isn't large portions of Java / JDK still closed source?

[grave tree]

But things like clang are open source right? The spec certainly isn't closed source

[bronze dragon]

any language could have proprietary implementations, in theory

[proud nacelle]

Clang is open source ye

[bronze dragon]

a more salient point would be that the official ISO C++ standard is behind a paywall
* working drafts are public, but the actual final standard costs a couple hundred USD to access

[grave tree]

Someone can make a closed source Python implementation, I don't know if that would make Python closed source though

[pallid garden]

oh what

[proud nacelle]

Yeah, but you can't really say a language is closed or open anyway I guess. It's mostly about the compiler

[grave tree]

I think you can in terms of spec though, right?

[proud nacelle]

What do you mean?

[pallid garden]

^ see

[grave tree]

A language, at its core, is a specification. It is describing behavior or how to use the programming language and how the programming language responds and is structured. It's why sometimes we have to draw a clear distinction between a programming language as defined by the spec vs implementation behavior

You'll often have a reference or default implementation, but that's still based on the spec

[peak egret]

my language only has one compiler, there is no spec, would you consider this closed-source?

[grave tree]

If you wanted someone else to use your language, you wouldn't provide them any sort of docs or instructions for how it behaves?

[peak egret]

i do have docs but they are FAR from being a spec

[proud nacelle]

I don't think I really understand what you're saying. Are you talking about documentation for a language or the syntax itself?

[upper dirge]

how am I meant to escape the bytestring so the exec will run?

exec('z=[[v," "][2<sum(z[x-61+y:][:1]<"$"<v<"/"for y in b'\x08\t\n<>pqr')```

[tribal drum]

Coding roulette 🔥

[pallid garden]

well no, since you put it on github...

[upper dirge]

SyntaxError: unexpected character after line continuation character

[kind thicket]

In the strictest of sense, can I take your compiler, modify it and redistribute it according to the four freedom?

That said, there is far more to the sustainability of an open source project:

• Does it depend on a single committer?
• Are changes welcome and made inclusive and without friction?

Some might talk about open community for instance

[tribal drum]

Create a global variable , use backslash there escape byte string

[kind thicket]

It's also why the Apache Software Foundation has the motto Community over code

[upper dirge]

it's not possible without making it global?

[tribal drum]

It is depends on python version then

[peak egret]

the project pretty much relies on me and i dont accept anyone elses opinions

unless you fork it ofcourse

[tribal drum]

This issue mostly was solved in python 3.12

[upper dirge]

I'm using 3.14

[pliant fractal]

anyone here know discord.py?

[kind thicket]

yeah, that would not make it non-OSS.
These questions come after, once it becomes popular

[tribal drum]

Hmm i don't know much then but global variable may do it or even a local variable

[grave tree]

Look at C++ or Python, the language is large and its expected behavior is documented in the docs, right? The docs describe the syntax and the behavior

[upper dirge]

id like to not have to make it a variable

[proud nacelle]

Ah, well I see that as part of a programming language.

If you have a 'programming language' without providing documentation it wouldn't be any different from writing random English words :)

Kinda like saying I fixed an impossible math question without showing the proof or a verified result

[cyan scaffold]

With a backslash before the quote in b' and its closing quote?

[kind thicket]

you could argue the source code is the spec

[grave tree]

I think you're still not fully understanding me. Mojo has basic usage instructions, but it does not describe the language's expected behavior, right? I'd compare it to Python's documentation, especially of its token parsing, etc

[upper dirge]

i changed the quotes and now I'm getting this error

  File "<stdin>", line 3, in <module>
  File "<string>", line 1
    z=[[v," "][2<sum(z[x-61+y:][:1]<"$"<v<"/"for y in b
                                                      ^
SyntaxError: unterminated string literal (detected at line 1)
exit status 1```

[grave tree]

the issue is the ' after the b is closing the exec string, no?

[cyan scaffold]

Which is why you escape it

[velvet trout]

Shouldn't you use r strings in such cases?

[bronze dragon]

you need to double-backslash there

[upper dirge]

ah yes

[cyan scaffold]

'my b\'byte string\''

[upper dirge]

it works now

exec('z=[[v," "][2<sum(z[x-61+y:][:1]<"$"<v<"/"for y in b"\\x08\\t\\n<>pqr")]for x,v in enumerate(z)];'*650)```

[tribal drum]

Still it won't run

U r comparing list with a string

[velvet trout]

Very pythonic code btw

[exotic solstice]

soo dontasktoask . com is blocked in this server. Why so?

[velvet trout]

[grave tree]

Because it's often dropped in a rude manner and creates a non-friendly atmosphere

[bronze dragon]

it's friendlier to tell them directly to not ask to ask, instead of dropping a link

[pallid garden]

it's code golfing

[stray field]

Instead, just say hello.

[pallid garden]

different kind of priority

[proud nacelle]

I know 0.0 about Mojo so no clue if it describes the expected behaviour or not

[bronze dragon]

just say "please ask your question directly" or something of that form

[exotic solstice]

I see, okay didn't knew it was rude

[upper dirge]

yeah I think I'll just go back to my last method cause this is longer

[tribal drum]

Ohk

[velvet trout]

I see

[pallid garden]

im reading this documentation on oauth: https://connect2id.com/learn/oauth-2#example-auth-code-flow-step-1
it says:

state: Optional opaque value set by the client to maintain state between request and callback.
what does opaque mean here? that this value is not relevant to the auth process?

[tribal drum]

I forgot , is oauth by Google?

[quartz fulcrum]

from where do i learn git and github?

[bronze dragon]

opaque means you're not supposed to rely on or modify the actual contents of this value. you get it from somewhere and you pass it on as-is.
(substitute you = whoever it is opaque for)

[proud nacelle]

Quite a lot of guides on the internet / Youtube

[pallid garden]

oauth is a standard that is used by multiple companies including google

[pallid garden]

so, it's not related to the process yea

it's just a way for the client to store some info to get back on the callback

[exotic solstice]

https://git-scm.com/learn

[tribal drum]

git init
git add .
git diff
git status
git commit -m "abcd"

[muted rampart]

https://en.wikipedia.org/wiki/OAuth
Seems a group of folks got together from different companies, made a spec, and many companies have since adopted it

[ocean ridge]

@pallid garden Personally I use firebase for authentication. its dead simple. Just suggesting

[bronze dragon]

https://beej.us/guide/bggit/html/split/

[tribal drum]

Thanks

[tribal drum]

Thanks

[pallid garden]

this is an academic exploration

[ocean ridge]

Alright

[pallid garden]

if the cluster has an internal oauth implementation, the public facing API should provide the relevant routes as well, right?

it doesnt actually do anything, the data just gets passed through

to the client, the public facing API is the oauth interface, and the fact that the oauth service is running on another node is just an implementation detail, right?

[velvet trout]

def keyword actually goes in the same order as in alphabets.

A, B, C, [D, E, F], G, ...

Another reason why python is easy, if you know alphabets ^

[crisp jay]

https://git-scm.com/doc/ext

[crisp jay]

Wdym

[cerulean ravine]

collections.abc then def. Now we just need something in the stdlib called ghi

[velvet trout]

A B C [D E F] G ... X Y Z

python's function defining keyword def has letters in the same order as they appear in English alphabets A to Z.

Wait i read that wrong lol

[random bison]

is pyautogui worth using

[velvet trout]

And then jkl, mno, pqr, stu, vwx, yz!
one letter less

[velvet trout]

Everything in python is worth using
Because of such a beautiful language it is.

[spice hill]

A beautiful conclusion. Delving even further, G-H-I just kludgily live, making not one Pythonista's quintessential repertoire

[velvet trout]

raise EnglishError("gotta use translator")

[stray field]

git diff seems a bit useless here. You already staged everything, so the diff will be empty.

[velvet trout]

Oh, but finishing this easter egg would be cool ngl

[gleaming marten]

git diff --cached works

[stray field]

So will git diff --staged

[spice hill]

maybe the spam and ham can be fried in ghi