data-science-and-ml

[agile cobalt]

you can use Pillow or OpenCV if you want to keep everything inside of Python

torchvision also has some transformations I think, not sure how practical it was though

[bronze wyvern]

yep noted

ty !

[sterile leaf]

yo

ive trained something using yolov8m , and its not so best but is there any program website that i can use this training i did for faster labelling

[bronze wyvern]

Hello, I'm just getting started with pytorch and I came across these 2 terms in the "getting started" section in the docs.

From what I've understood, dataset is the one which will help us read our data (in a key value pair fashion I think?) while dataloaded is the one which will help us "process" the data, like iterating through it?

It's still a bit messy, I don't have that mental model well built, would really appreciate if someone could explain pls.

[waxen kindle]

That's the idea yes

The dataloader is basically dataset + sampler parameters (batch size, shuffle, custome samplers etc.)

[mellow vector]

"Min supported Python version is now 3.10 (#162310)
The minimum version of Python required for PyTorch 2.9.0 is 3.10. We also have 3.14 and 3.14t available as preview with this release."

I've been using 3.11 for months, am I doing something wrong?

I'd rather be using a more modern interpreter if things work out of the box but 3.11 has been the best I could do.

[serene scaffold]

Why do you think you're doing something wrong? Does something there indicate that 3.13 wouldn't work?

[mellow vector]

I've had issues in the past

[serene scaffold]

What issues?

[mellow vector]

something to do with tensorflow iirc, I'll have to try again and see if I cant get it working

[serene scaffold]

are you trying to use pytorch or tensorflow? (do not try to use tensorflow.)

[mellow vector]

pytorch

[mellow vector]

I'll give it another try, is good to have some confirmation that I can update my stack

[winged girder]

I have a question: How do I know if a problem should be solved with AI or not?

[bronze wyvern]

Hi guys quick question. One thing I don't understand, for object detection why is it important for us to annotate our images?

For e.g, how does it differs from image classification? For image classification what I would do is have a folder in which my images will be then have a .csv file that would reference them but how does it differ from annotating objects?

Also, when annotation objects, should the images differ? Like whitebackground etc?

[agile cobalt]

if it can be solved without AI, it should be solved without AI

[serene scaffold]

image classification is about classifying the whole image, perhaps in terms of whatever the main subject of the image is expected to be (an image classifier for cats and dogs expects every image to be mainly of a cat or mainly of a dog). which pixels in that image are the actual dog or the actual cat aren't explicitly part of the task.

object detection is about finding where in an image something is.

[bronze wyvern]

yepp I see

Thanks !

By the way when we talk about object detection, how does it work behind the scenes pls.

For example, say we are talking about live object detection, our camera will kind of take pictures each second (say 60 frames per sec), then how does the algorithm work? It has a loop which also try to detect 60frames per sec?

[serene scaffold]

I would imagine so.

perhaps it can use the n previous frames in some way

[bronze wyvern]

yup, the idea is that it uses the frames though

[serene scaffold]

I have never done object detection, to be clear

[bronze wyvern]

yup never mind, surely the idea is with frames 😂

[serene scaffold]

but then again, neither has Elon

[odd meteor]

Classic advice

[iron basalt]

If it feels like something you can't write explicit rules and algorithms for by hand, at least not in a reasonable amount of time. Instead you need a bunch of example data and have it learn it. Then it's time for machine learning and maybe even AI.

Or those manually written algorithms are not good enough and you need something more complex.

The jump from machine learning to AI is when you need it to make optimal decisions as an agent (given the information it has).

Example of where AI is required is a mars rover. It needs to make decisions on its own given its sensor data because the latency for messages to earth is too long to manually control fully.

[robust echo]

So the tech we're currently calling AI is basically ways of doing empirical computing

i.e. try something and measure the outcome

it's mainly applicable to situations where you can theorize some function

and you can tell whether outputs for a given input are correct, at least to some degree.

but you can't precisely define what the function's content should be

After all, if you could precisely define the function content, you could just write the function out in any ordinary programming language instead of modelling it.

So "AI" is generally applicable in a situation where you want to approximate some theoretical function. Optical character recognition, generative AI, and game bots all fit that description

[viscid urchin]

At the moment, my take is “if and only if there is seemingly no other way”, but that is going to shift month by month in my opinion.

[left tartan]

Why do you assume it's a binary decision?

I think what we'll find is gen ai tools can be helpful, and part of the process, but not sufficient

[mossy blaze]

On my AI project, I use the A* algorithm with a distance heuristic between values ​​that is consistent for similar types but not otherwise. To overcome this problem, I would consider using a PolicyValueNet on the tree of neurons to connect, like AlphaZero, except that I have never done one. Would this speak to anyone?

[jagged moon]

anyone knows how to resolve this issue?

[lapis sequoia]

what do people use to make a chat bot that is not langchain derived or some trendy LLM?

[lapis sequoia]

is it yours?

[supple bronze]

How can i get codewithharry data science course for free ?

[buoyant vine]

aha, entertainingly I too have been wondering this, every lib I found was just calling langchain under the hood lol, or the code quality was 💩

[lapis sequoia]

there are ways to do it with nltk and no NN, but they are not as good. you can do it with just linear layers.

[buoyant vine]

Oh if you're talking about making a chatbot from scratch rather than tooling API wise then LLM is the best way to go

for generating text, LLMs and transformers in general are currently still the best we have in terms of learning efficiency and accuracy

langchain is just calling GenAI apis, be that OpenAI or whatever, but if you can self host models which can do the same thing. (with or without langchain)
but training a model from scratch is a lot of work and takes a ton of compute, it isn't something you can reasonably do locally and get mediocre results

[lapis sequoia]

Good evening

[random hound]

anyone here good with data structures / algorithms? I'm trying to understand A* search, and am having trouble determining exactly how h(n) is supposed to be implemented and what it actually is. I get that it's related to dijkstras in the sense that it uses h(n) to "guess" the direction it should go as opposed to just checking everything, but I can't figure out exactly how that's supposed to be implemented. I have a working implementation of dijkstras' here https://dpaste.com/H3KXQM3ZV . the dataset is a weighted, undirected graph representing a train network, and the weights correspond to distances in km

[serene scaffold]

You're looking for #algos-and-data-structs

[random hound]

ok

[oblique isle]

hi guys , do u know a free facebook & X Scrapper ?? i need it for a researsh sentiment analysis project

[lapis sequoia]

You need to get a API key

[lapis sequoia]

This goes for any massive website with lots of users.

[safe merlin]

what are some things i should learn to learn ML?

[waxen kindle]

ML

joke aside, linear algebra, statistics

[safe merlin]

math

[waxen kindle]

yeah you'll have to

[safe merlin]

well

ima delay that to the very end

is there anything in python

[waxen kindle]

python itself

[safe merlin]

Im not that great at programming

[waxen kindle]

focus on the bases of programming then

[safe merlin]

I know how to make simple programs

[waxen kindle]

and python

time to dig in then

[safe merlin]

well i know how to make a calculator, random number guesser, hangman, tic tac toe, rock paper siccors

[waxen kindle]

nice, that's some simple projects

[safe merlin]

yeah i know the basics

[waxen kindle]

you know about classes ?

really focus on knowing well python

make some bigger projects

[safe merlin]

no dont think so

[waxen kindle]

follow resources or tutorials

and practice

[safe merlin]

classes are a big part of OOP right?

[waxen kindle]

yes

[lapis sequoia]

massive part

[calm thicket]

classes are the O in OOP

[crimson jackal]

Hello. I have a emcee mcmc code running and at a certain point the code is evaluating the same exact point a large number of times. Why?

[lapis sequoia]

Do it for the right reasons, not the wrong ones.

[rich flower]

How to learn AI / Machine learning?

I have some fun projects lined up to use AI but have no idea about how to learn the material, what to learn, the mechanics, etc

[serene scaffold]

AI and ML has many layers. the more layers deep you go, the more you understand. but depending on what career you end up in, there might be a diminishing return on deeper layers.

[mossy blaze]

I tried something. It doesn't work, but I think the basics are there. Would anyone be willing to review my code?

[dim crypt]

@harsh vessel your help thread was closed before i could respond to you again, i just want to make you aware that we also have this topical channel on this server

[bronze wyvern]

Hello, quick question. I need to upload some folders containing a bunch of images on my google drive so that they can later be accessed on google colab.

My question is, in order to upload the folders, I just uploaded them as-is, that is no compression done.
If I first compressed the file then upload on my drive and unzip them directly onto my drive, will the speed of execution be different? If yes, can someone explain why pls.

(sorry this is not really a question related to data science or ai but, was just curious :c)

[serene scaffold]

It takes time to compress and decompress the files, but if compressing them causes them to take up less space, it will be faster to upload them because there's less of it

[bronze wyvern]

yup, just tried it out, the thing is I think by default png and jpeg are already in compressed format, so it took me approximately the same time using both methods

[jagged moon]

Yes

[lapis sequoia]

did you write the notebook on vs code or google colab or where did you write it? What does it look like when you open the .ipynb file on whatever you used to make it?

[cedar veldt]

helooo, I'm looking for people to have a discussion about soft robotics with python (this is related to AI as well)

[grand minnow]

You can start a topic and if others want to chime in, they might

[lapis sequoia]

Optimal control theory? Arduino boards? Honestly, robotics and that stuff is so mentally taxing and takes up a lot of time and resources. It’s more of an electrical engineering thing rather than data science or AI. I don’t mean that to be dismissive. Collecting data for a robot, agent, is so hard and mentally exhausting. I have a couple of friends who do it if you have questions. Most of it is surprisingly done in Python tho. Do you do robotics?

[cedar veldt]

its very weird indeed

that a lot of tools for this are available in python or c++

but most have a python version

well I'm designing a little language on top of python to do creatures

I've meet amazing people just for talking about it here in this channel

how does it relate to AI? well AI can optimize this designs , make them move better

[jagged moon]

I found the issue Nick, it was because of my github account being flagged. i have contacted their support team, thnx btw :). I wrote the notebook on vs code tho.

[bronze wyvern]

Hello, does anyone knows what is the difference between graph model and layer model in machine learning, I think it depends on the file extension like in keras, .h5 uses layer model and .pb (tensorflor js) uses graph model I think

[stuck swallow]

My friend has a discord server with a few thousand people in it. It is a history server, so the conversation is usually with good grammar and wide vocabulary. I have consent from him (he is the creator and owner) to log every message and in exchange I simply create some moderation tools for him to use for free. I have a data science project later this year where we have to find our own dataset. I have a CSV with the structure of user_id, channel_id, message_id, message_content where message content is the content of the message. Are there any projects I could do with this? Are there any projects where on top of completing the data science project for my class I could also give useful information to my friend to help him run his server better?

[serene scaffold]

I've never heard of this distinction. I don't think it's widely used

Maybe it's ways of visualizing the computation graph?

This represents the computation graph of a neural network. Each node in the same layer is represented individually. A "layer model" might visualize each layer as one visual element.

I'm just guessing.

[bronze wyvern]

Hello, I need to perform some data analysis/data visualization, is there a library that can give me a summary of all my data, like generating a report?

[waxen kindle]

you can try datawrangler, it has some nice features from what I heard and it's also a vscode extension, you use it to see stats and distribution of columns of a dataframe
https://data-wrangler.readthedocs.io/en/latest/

[bronze wyvern]

will have a look, ty !

[tight saddle]

Hi

I'm looking for people who know about AI

[bronze wyvern]

will have a look, thanks

[light cloud]

Has anyone here gone into production with a live payment gateway

[serene scaffold]

@light cloud your messages containing self-promotion were removed, as that's against the rules
try asking your actual question about payment gateways in a channe like #web-development. remember to always ask your actual question--don't ask to ask

[serene scaffold]

same thing--never ask to ask.

[tight saddle]

Wdym bro

[serene scaffold]

if you have a question about x, never ask "does anyone know about x?" ask whatever your question is about x.

that makes it easier for other people to start answering your question and faster for you to get one.

[tight saddle]

Well

Sometimes that sentence won't work for a person

For whatever his X is

[serene scaffold]

what do you mean "that sentence won't work for a person"? like, they don't even know what their question is?

[tight saddle]

Is this server hard coded or lifeless bruh

People have to act as literal robots and ask questions that HAVE to start with does anyone know about etc or depends on chat or what?

[serene scaffold]

@tight saddle what I'm saying is, instead of having a conversation like this

Asker: Does anyone know about pandas?
Other: What do you want to know about pandas?
Asker: How do I get the average value of each column?

Save yourself and everyone else time and effort by just asking this:

Asker: How do I get the average value of each column in pandas?

It's actually impolite to ask questions the first way, so please refrain.

[gentle flower]

https://youtu.be/RHA7oxr4LNg?si=DEoKarnpkzWSKdZs

[chrome elk]

can someone help me optimize this code:
https://leetcode.com/submissions/detail/1807760984/
:)

[serene scaffold]

this is a question for #algos-and-data-structs. try putting the code itself in the chat, as this requires a leetcode login.

[chrome elk]

oh, thanks sirr

[undone summit]

can anyone tell me the resone i and gettign an error?

[agile cobalt]

double check if you have the correct python environment selected

assuming you used pip, you can check the path using pip --version, it should match the version in use by the python: select interpreter command (ctrl + shift + P)

[slim wyvern]

Is there a scientist in this chat that understands python? I have something that might freak out the scientific community.

[serene scaffold]

then do it.

[brittle ermine]

hi can someone help me with something

pls in dms??

[waxen kindle]

just ask here....

[lapis sequoia]

yes

[livid cipher]

Are these good libraries to use for creating a web-based SVN repository? I tried specifying requirements in the coding archeticture tool and this is what was provided for dependencies.

[lapis sequoia]

@livid cipher, that's interesting. what do you think about it?

[livid cipher]

@lapis sequoia Think about what?

[lapis sequoia]

just your idea is bad

[livid cipher]

So the libraries aren't good?

[lapis sequoia]

yep use smth else

[stark mulch]

What other libraries would you recommend, and why?

[lapis sequoia]

i recomend skibidi library

[stark mulch]

!tempban 919231558915850310 2w low effort troll

[arctic wedge]

:incoming_envelope: :ok_hand: applied ban to @quartz haven until <t:1762330781:f> (14 days).

[stark mulch]

Those libraries seem fine to me

[sterile leaf]

yo

[rose sonnet]

🗣️

Top 10 things scientists don't know to explain.

[amber ridge]

ai will kill us all be safe

[slim wyvern]

I did ill PM for details if you want

[serene scaffold]

I don't respond to DMs

It's more beneficial for everyone if you ask questions and give information in the server

[slim wyvern]

ok one sec ill send the code structure to this layout again

[arctic wedge]

Please react with ✅ to upload your file(s) to our paste bin, which is more accessible for some users.

[serene scaffold]

I'm afk right now
Also please follow all instructions from the bot.

[slim wyvern]

please assisst me im fairly new to discord and have been in deep reserch for a very long time

[spring field]

read and follow this, it should be pretty simple

[serene scaffold]

The bot responded to your message with a very simple instruction

[slim wyvern]

✅

[arctic wedge]

Click here to see this code in our pastebin.

[slim wyvern]

??? did I not?

[serene scaffold]

You have to push the button.

That's all

[slim wyvern]

it's in the bin

[spring field]

is there a particular question associated with this code?

[slim wyvern]

I'm just trying to get this looked at by a scientist for further research I do not mean to be disruptive in any way. Yes I have changed Albert's theory of relativity that changed the dynamics of gravity and how it is computated I simply need a review

[long locust]

I scanned through it briefly, it looks like some kind of model for creating atomic nuclei?

What was the specific question you had?

[slim wyvern]

The effect of gravity is applied through negative variables to establish it's behavior with matter in a very specific way. I simply want a opinion on the operational implications

[long locust]

I don't think gravity is the main force in effect at the atomic nucleus scale

[slim wyvern]

it's not at least for a single one. how ever it does still persist even if weak

the program will show gravity existence regardless of how small the particle. this simply allows us to measure it correctly

[long locust]

I guess I don't understand the context then

[slim wyvern]

mass = gravity at any level.

gravity is weak at this stage but persist

[long locust]

ok

[iron basalt]

You are working with very small floats, and will likely have floating point precision errors.

[slim wyvern]

and is amplified by the add on of other particles

[slim wyvern]

none at all actually

I have vectoring

[long locust]

So even small masses can induce a gravitational attraction, is your point that you found a way to compute that? Or what is the goal here

[slim wyvern]

compute it and use it

[long locust]

Use it in what way

At that scale the static electric charge force is more of an effect, as well as ion bonds between atoms (also the strong & weak nuclear forces)

[slim wyvern]

correct but gravity is still there

it may simply be weak to the exsisting elements but still is there

[long locust]

Do you know how much the difference in force is?

[slim wyvern]

it's influence will also increase as more bonding happens

1.5

[iron basalt]

You have compared to an exact result?

[slim wyvern]

I have been able to measure black holes, neutron stars and are own

[long locust]

It is not 1.5, it is 10^36 times more strong at that scale

[slim wyvern]

we are not talking mass scaling simply the amplification of gravity when bonding happens

[long locust]

Then I truly do not understand what you are saying here, apologies

[slim wyvern]

ok an atomic particle will have a low gravity yield add to that you will get a amplification in gravity not just a add on of to gravity applied particles

that amplification is 1.5

to that of a single particale

the gravity will get stronger the more matter applied even in black holes it shows that mass is still there

it's not a singularity but close to one

to put simply my theory was to show that black holes do not get infinite results. the math will show no matter the object space puts a limit on how dence a object can get

Gravitational Core Relationships

Gravitational Yield (GY)

𝐺
𝑌

Particle Mass
×
2
GY=Particle Mass×2

Particle Density (PD)

𝑃
𝐷

𝐺
𝑌
2
PD=GY
2

Quantum Field Reaction (QFπ)

𝑄
𝐹
𝜋

negative by nature (−1)
QFπ=negative by nature (−1)

Compression Pressure (CPπ) — your primary gravity-space formula

𝐶
𝑃
𝜋

𝜋
×
𝐺
𝑌
×
𝑃
𝐷
×
𝑄
𝐹
𝜋
CPπ=π×GY×PD×QFπ

or equivalently,

𝐶
𝑃
𝜋

𝜋
×
𝐺
𝑌
3
×
𝑄
𝐹
𝜋
CPπ=π×GY
3
×QFπ

since
𝑃
𝐷

𝐺
𝑌
2
PD=GY
2
.

that did not come out the way I attended

[long locust]

an atomic particle will have a low gravity yield
This is dependant on it's mass. F=Gm/r^2
add to that you will get a amplification in gravity
Yeah, Delta F is dependant on the mass you add
gravity will get stronger the more matter applied
Yeah, the total mass determines the Force of gravity at a distance
space puts a limit on how dence a object can get
You've lost me here. The limit is how do you overcome fundamental forces

And what is your source for this? It looks like a chat gpt hallucination

(Also, I think this is no longer about #data-science-and-ml , maybe an off topic channel is more appropriate)

[slim wyvern]

Gravitational Yield (GY) GY=Particle Mass×2 Particle Density (PD)PD=GY
2 Quantum Field Reaction (QFπ) QFπ=negative by nature (−1) Compression Pressure (CPπ) — your primary gravity-space formula CPπ=π×GY×PD×QFπ or equivalently CPπ=π×GY
3
×QFπ since since . PD - GY*

[rose sonnet]

What the fuck am I even reading

Pardon my French.

It's either me being dumb or this dude is having a stroke (or having too much time on their hand). I believe both options are true.

[lime grove]

does anyone here know how to connect a Jupyter Notebook (within Anaconda Navigator) to a personal GitHub repository?

[serene scaffold]

jupyter notebooks are notoriously hostile to git.
but it's the same as versioning any other file with git and pushing your changes to github.

have you pushed regular py files to github before?

[lime grove]

Just save it somewhere, and commit from there to the repo using the cli, I guess

[serene scaffold]

I don't understand this message.

point is, using git and github with ipynb files is the same as py files.
do you need help with that?

[opaque condor]

Image if I'm training an AI trip the model up

[manic cave]

i have a que

data analyst is considerd as data scientist

[serene scaffold]

there's no widely-agreed upon distinction

and to the extent that there is one, job titles don't consistently apply it

[patent oriole]

I took data science in college, this is my first sem and im currently learning C, python matplotlib pandas numpy scikit... basic DSA stuff like graphs hash tables shortest path... and statistics linear reg reg p value reg r squared... (all this is by myself) I plan to finish all this and have a good understanding of python and R mainly, by 2026 mid i would have finished all this, then id have time to go deeper, make a few projects then start looking into ML. Anything that i missed or anything that you all would recommend?

[waxen kindle]

why are you doing all by yourself if you are taking it in college ?

[south drum]

I have an MS in Data Analytics. Probably two key things I'd recommend to add to your list is learning how to work with big data such as PySpark for Data Engineering tasks and learning TensorFlow in Python which is great for creating ML projects, especially Neural Network ML models. A final thing I'd recommend is to make sure you build an online portfolio of the kind of work your doing for college, this is a huge resume boost in the Data Science field.

[limpid zenith]

do people still use tensorflow? it's being phased out everywhere including by google...they're moving to Jax

I would recommend starting with PyTorch or Keras honestly

[serene scaffold]

it's being phased out everywhere including by google...they're moving to Jax
this is correct.
keep in mind that keras is part of tensorflow.

[limpid zenith]

keep in mind that keras is part of tensorflow.
that's not entirely true, keras is agnostic of tensorflow and yes keras is a part of tensorflow also..or was..but latest keras isn't..hell keras can use jax and pytorch

[serene scaffold]

have they updated it? I haven't actually heard anyone mention keras in like five years.

[limpid zenith]

i mean last i checked keras 3 is still pretty active ..though i agree.,.it's pretty much a nothingburger compared to pytorch and jax and lightning AI

i personally only used it a few times, and found it very similar to lightning's API

[jaunty helm]

keras split off from tensorflow and now acts as a higher abstraction layer that allows you to choose its backend (jax, tf, torch, etc)

[serene scaffold]

That's pretty interesting

[jaunty helm]

in theory anyway, it still has a few edges
some time ago I was trying to port tsai's torch impl of MiniRocket to keras, and got really confused when the same code did different stuff in keras with jax backend vs. keras with torch backend
I think it was scatter_update that literally didn't work for jax or something? don't quote me on that tho

[lime grove]

at some point you just have to make a decision before going down a rabbit hole - do I really want to spend time turning this into a main project?

fighting with unexpected behavior, for instance. You set out to do an ML project, but get bogged down in something tangential.

this is especially pertinent when you start considering the fact that developer bottlenecking is the biggest efficiency issue, not cpu-bound, nor memory bound processes.

so, for that reason, a simple heuristic I follow when adopting a module into my "stack", is to see how active the project is, how many developers are working on it, the most recent commit, and so on. I get extremely cautious when it looks like 2 or 3 person project, or when it tries to do too many things. Recall that one of the chief virtues of Unix is that the command line utilities, of which there are literally thousands, are hyperfocused on what they do, and are extremely stable.

obviously here you run into the current thing, GenAI, which is most definitely not hyperfocused on a single thing. But note also that the biggest problem in this space is that of implementing guardrails on it, e.g. AI safety.

[patent oriole]

its a 4 years course if i wait 4 years till they teach all this then it would be very slow

[patent oriole]

ohh okay, thank you

[lime grove]

No, I do not need help with it. My approach will be to do the jupyter thing (exploratory work, basically), and then commit the work using a command line separate from anaconda navigator. It is purely a question of setting up folders and subfolders. I really don't want to spend more time than I have to wrangling an awful UI

[void lion]

Hi, I have a question. Why do we mostly make AIs in python ? I'm an IT student, and I want to work in AI later, I've made decent project myself with AI using python, but I'm learning C++ right now at school. Why don't we make all AI in C or C++? Python is made in C anyway, and yes python has the tools to do it ect, but it's soo slow, why is AI in python even a thing ? Is it because it's easier ? But it makes no sense, AI engineers are smart enough to learn C++.

[agile cobalt]

native Python code is slow

AIs in Python are generally made using well optimized libraries implemented in other languages, which are oftentimes faster than naive implementations of the same algorithms in C++ would be

also when you get into GPUs you would pretty much need to use CUDA if you wanted to go low level, which is even more impractical than 'normal' C++

see how much time & work it takes to do a matrix multiplication using PyTorch, then do it in C++ and compare the results

(both in terms of development time and runtime speed. Bonus points if you use CUDA for both.)

[void lion]

That's my point, people made these librairies in python in the first place because they thought it would be "better" to make AIs in python. Why ? You could make a NumC++, and C++Torch librairy in C++, and then the same tools would be available. The only difference would be that it's faster in C++. Am I wrong ?

[iron basalt]

The main reason is momentum, AI libraries used to be in other languages. Then once some key moments happened like torch going to Python (from Lua), and other key libraries getting Python bindings (or new Python libraries being made), it exploded in usage for AI. And now due to ever increasing libraries available to Python, it has snowballed to the #1 language (in terms of number of people using it) for all things.

[agile cobalt]

python is a lot more flexible - no need to compile each time, easier syntax to learn, easier to share cross-platform code, an entire ecosystem with integrations such as Notebooks and experiment tracking libraries, arguably better abstractions

[iron basalt]

In addition Python is a "non-offensive" language in a sense. It's simple and straight forward. You don't need to learn what a compiler is, etc. It's not terribly exciting, but it just works.

[void lion]

So, the only reason that Python is mainly used in AI, is developer' laziness ?

[agile cobalt]

being "lazy" is a good thing if you are a developer
you really shouldn't be reimplementing the same thing over and over again if someone else has already created an open source version of it that just works out of the box

[iron basalt]

Laziness is not the right way to look at it. Learning something like C++ just takes a lot of time.

And people specialize. Giving a simple common interface for many people is good.

[void lion]

If someone made every python librairies in C++ then it would be worth using C++ ? Except for the compiling part, C++ just looks better

[void lion]

AI is not simple anyway, people able to make AI should be able to learn C++ easily no?

It's not that hard or long to learn

[agile cobalt]

better in which ways?
it's more verbose, memory management is more error prone, and you have more things to worry about related to the language itself

[iron basalt]

C++ would not be well suited here. The main reason being that it has no actual modules. Because of this every project is built in its own way and it's generally a pain to use other libraries. In addition C++ is massively complex, easy to mess up.

[iron basalt]

AI is not simple, but imagine learning chemistry, and now you also need to learn how to make an engine.

[agile cobalt]

many researchers are focused on the mathematical/statistical parts, and want a minimal language that does not gets in their way as opposed to having to think about low level details

[iron basalt]

You can, but it takes time.

[void lion]

Ok, I get that

[agile cobalt]

but really, above anything else compilation can be a huge pain

[void lion]

I'm not advanced enough in my studies to defend my opinion more, I don't know enough

[iron basalt]

Since C was also mentioned. It's not well suited because you can't express things in a concise high-level way. Like for example in Python I can have two number arrays (numpy) and do a + b and it will element-wise add them for me (fast). I don't need to write a whole loop or any of that. So the math more directly translates to code. Garbage collection makes this kind of very high-level expression much more feasible.

Python happens to have the tools needed for this (operator overloading and objects, garbage collection).

(Technically C++ can do these kinds of things, but it gets really complex https://github.com/xtensor-stack/xtensor)

(And even if you can, maybe you can see why people want just Python with a + b instead of ```cpp
#include <iostream>
#include "xtensor/xarray.hpp"
#include "xtensor/xio.hpp"
#include "xtensor/xview.hpp"

xt::xarray<double> arr1
{{1.0, 2.0, 3.0},
{2.0, 5.0, 7.0},
{2.0, 5.0, 7.0}};

xt::xarray<double> arr2
{5.0, 6.0, 7.0};

xt::xarray<double> res = xt::view(arr1, 1) + arr2;

std::cout << res;```)

[void lion]

So why not make a new language for AI alone ?

A simple language, with maximum efficiency for AI only

C++ would be faster (if it wasn't compiled) but harder to write, and python is slower but easier to write. AI is getting out of control and we need more and more efficiency, so why not make a new language with both qualities of python and c++?

[iron basalt]

Making a language is hard. Especially since it's no longer good enough to make just the language. You need tooling. Editor support, LSPs, debuggers, tons of libraries/bindings, documentation, a community, books, etc.

[agile cobalt]

some have tried, it didn't get very popular

in general you want the flexibility of a general purpose to integrate with other things (e.g. web servers or command line applications)

[void lion]

The gain from a new language optimized for AI is not worth this hard process ?

[iron basalt]

It's not.

[void lion]

For now at least

[iron basalt]

The gain in performance would not be there. Gain in productivity would need to be at least like 10x.

[agile cobalt]

we are not talking "one person struggling for a few months" level of hard when it comes to creating a new language
we're talking a team of dozens of specialists working for years

[iron basalt]

This can be seen with something like lightbulbs. It took them to get to like 100x before people bought the new ones.

People have what is already working for them.

So it's not just make new thing, it's this massive transition cost.

And I think even the most skilled designers would be hard pressed at this point to make something so much better than what is there now that it would justify all the rewrites.

[void lion]

So we'll just push python until it can't hold the future level of AI correctly, and then decide that we need a new language when it'll be even harder to change it?

[iron basalt]

In the case of something like Rust they have the excuse of security, which results in a massive long-running cost that is building up, so they can justify the replacement.

[agile cobalt]

python code using well optimized libraries implemented in low-level languages is as fast as well optimized low-level code can possibly get in any language

[iron basalt]

Well that is Python's ace up its sleeve. If you made a language for AI, and you made an AI library in it, someone would then say "cool new language, anyway, got any Python bindings?"

[agile cobalt]

if Python "can't hold the future level of AI correctly", no language can

things that must be efficient are already implemented in C/C++/Rust and alike

[void lion]

So we're already at the peak optimization of coding ?

[agile cobalt]

not necessarily, but there are no major gains you could get from switching to another language, let alone creating a new one

[iron basalt]

No, that's not exactly how it works. The compiler optimizers are really good now but still improving. And as is the case in any programming language, you need to write things in a way that is not slow (this will never change until we have AGI and it does it all for you).

(DS&A, knowing how the machine works (mostly memory speed and parallelization now))

[void lion]

Okay I see

[iron basalt]

In some cases a bit of assembly for special purpose instructions (like for encryption or video, etc).

[agile cobalt]

some libraries essentially create their own mini-languages within python though, via operator overloading

(from numpy broadcasting & slicing to polars expressions)

[iron basalt]

Taichi is a really good example of that.

Fully parses Python with its own parser.

And makes use of the type hints as actual types.

Ever since type hints we can have full DSLs for high performance.

https://www.taichi-lang.org/

[void lion]

I understand, thank you for your answers

[oak hearth]

Does anyone use pentaho data integration

[opaque quiver]

hey guys im working on a project and in that project i need to calculate distances from the camera lens in an image do you have any ideas how i can do that??

[small wedge]

Will the images all be taken on the same camera?

Or with the same parameters like image resolution etc

[opaque quiver]

no different cameras

[scenic parcel]

you guys ever use llms to generate training data

[serene scaffold]

if you do that, don't use LLM-generated test data. otherwise you don't even know if the model's performance means anything.

[lime grove]

I am confused. Why would you be dealing with data coming from two disparate domains. One IRL, and the other from an LLM? Whatever happened to k-folds, etc.? The data has to be sourced from the same "probability distribution function", to use a very rough analogy

If there is some distribution in the data, it is because that data has certain properties. For instance, for a Gaussian distribution, the data points are uncorrelated with each other. If you toss in data points in there that do not have that property, then there is no real basis to expect transferability of the parameters obtained with the training set

in other words, you would have to ensure that the LLM data has the same statistical properties that the IRL data has. Not sure how you would do that, especially given the uncontrolled nature of hallucinations, nor the effect that safety guardrails would have on all of it.

[gloomy fulcrum]

I find the Ai hallucinations very interesting

[elfin vale]

hi guys

anyone here

if anyone wanna do Ai Ml projects with me just dm

[grand minnow]

Do you have any idea what to build?

[jaunty helm]

depends on what they mean ig
it's very common to use llm generated text to train another llm
anything outside of that though I'd think twice

[small wedge]

huh? very common? I doubt any serious researchers are doing that. There's a pretty large amount of work showing how that tanks performance significantly compared to just training on real text.

at least intentionally, at some point it will inevitably be poisoning datasets just by nature of how much of the internet is/will be LLM generated

oh apparently apple has admitted to doing it I stand corrected ig

not sure why exactly if they thought it would give them an edge on alignment or if they were just seeing what would happen

[jaunty helm]

consider that quality real text is hard to get in large quantities
and also, some rare text will be very underrepresented in "real" datasets, where llm generations can produce more of it

[small wedge]

I can kinda understand it from like a data augmentation perspective or if the model youre sourcing from trained on data you dont have access to

[rapid burrow]

hi anyone know why does my llm is outputting sentences but the context is inconsistent

[agile cobalt]

context or content?

either way there is no way to make LLMs generate consistent answers 100% of the time, they still go offrails all of the time, even more so smaller ones

[rapid burrow]

like context

like I ask my model "is the three primary colors?"

it give me this "The three primary colors are red, blue, and yellow.Eat a balanced requests with fruits and vegetables."

[pliant gate]

hi, do someone know a roadmap for learning AI? most of the roadmaps that I see in Youtube are from years ago

[serene scaffold]

The fundamentals have not changed in that time.

[pliant gate]

so its something like Python, Sql and Pytorch?

[serene scaffold]

No. You shouldn't aim to "learn pytorch". You should aim to learn about neural networks, and implement them with pytorch.

If the roadmap lists python libraries that you should learn in a prescribed order, throw it away

[pliant gate]

I will learn about neural networks then, thank you :)

[hoary topaz]

Hey, I haven't been so active there but just an idea id want you to kind of approve (or disapprove) the possibility.. id gladly sacrifice my braincells for if it has a chance to do that - ai lego sets.

I mean I don't mean images in style of lego bricks. That stuff is shit. I generally mean a model that can think about possible connections of lego brcks and actually create real 3d sets out of it. Ive been training my own models in the past and ive been a lego fan for my whole life.

Obviously id start with something simpler, the scope of placing bricks on top of each other with no tilting and any other special connections.

My ideal result would be the model realizing which parts i have available (xml list of parts) then gets the theme and tries to build something out of it.

Performance -wise - I have 8 L40S gpus available for training purposes.
(just so you know i mean real things and I'm not just a dreamer)

feel free to ping me anytime regarding this.

[obsidian blade]

By no means am I technically learned enough to criticize this idea but this sounds really cool. You’d probably have to use some graph based learning so it knows how pieces fit with each other and sort of alignments don’t work

[agile cobalt]

iirc there are already some models that can generate voxel based 3D models, you could start there then either constrain the generation process or just fine tune it and hope the result can be represented with lego bricks

[arctic smelt]

Look up legoGPT. A bunch of smart people at carnegie mellon university have already built an AI model that does exactly what you're taking about. It's an autoregressive large language model that takes text prompts and spits out actual, buildable 3D lego design

[arctic smelt]

Llms are designed for next word prediction. It doesn't know that red, blue and yellow are primary colors. It knows that the sequence of words "The three primary colors are" is very, very often followed by the words "red, blue, and yellow." And sometimes, in some shitty corner of its training data, that phrase was followed by a line about a "balanced diet." So, it just rolls the dice and sometimes you get the nonsense you saw

[bronze wyvern]

Hello, sorry to disturb you all on a sunday :c, need a small advice.

I'm currently working on a uni coursework where I need to train a model so that a robot can use that model to do its "thing".

Let me elaborate.

So, I'm supposed to train an image classification/detection model to recognise office items. I'm done with my model but now I need some context for my robot.

Basically, the robot is a robotic arm (kind of similar to how an actuator is). I need to provide some context about how that robot can be useful, why people would need that/use that rather than doing things manually.

So the first thing that came to my mind was about organizing stuff so that employees no longer need to do it. For instance, assume there is a file cabinet and inside there are multiple sections where we need to put/organize stuff, like a drawer will "sub-sections".

So my context would be that on each "sub-section", there is a label, the robot would identify the labels by some means and then put things there.

So if someone need to put 1000 files in a drawer, it might be time consuming and on the other hand, the robot might do it without any problem.

I believe I need a strong context, that's why I wanted some opinions, in my report, I need to state why would people choose to invest into that robot, why is it useful/can't we do it by hand.

Please ping me for a response

[mossy pond]

maybe ask on hugging discord 😉

[jaunty helm]

which model are you running?

[mossy pond]

i have two prompts for coding with LLM ... i know that most very suspicious (i am not a coder but i know the logic and ask specific questions) so first for coding output, second for coding explanation ... sometimes i switch while the chat 😉

You are an AI assistant that helps users with Python programming on Windows Operationg System. You handle writing code, fixing bugs, making code better, and teaching good coding practices following Python standards (PEP 8, PEP 448, PEP 343, PEP 382, PEP 492, PEP 484, and PEP 257 etc.).

How you work: Understand what the user needs - Think through the solution - Give clear, structured answers.

Input Requirements:
1. Task Recording
Write down exactly what the user wants to accomplish and any limits or special needs.
Figure out if they need: new code written, bugs fixed, code improved, or coding advice.

2. Solution Development (Internal)
Think through the problem step by step and spot any potential issues.
Plan your approach carefully and consider what might go wrong.
Make sure your solution follows Python rules and actually works.
Review dependencies, passing, variable definitions, function calls, and data structures (dictionaries, tuples, lists, bytes, strings, values etc.) for consistency and accuracy. keep an eye on regex, byte and compiling.
Check which libraries may be needed or already used.

Output Requirements:
1. Solution Rationale
Explain briefly why you chose this approach and the main decisions you made.
2. Code Implementation
If possible, provide a Python code snippet that only contains the corrected section!
If need import libraries.
Clear titles and Comments for functions and important sections.
Proper Python formatting and correct handling of data types.

3. Summary
Give a short overview of what your solution does.
Mention any limitations or assumptions, plus suggestions for improvements.

You are an AI Assistant specialized in reading and explaining Python code on Windows. You analyze code behavior, structure, and logic, and then explain it clearly in natural, human-friendly language, following the principles of PEP 8 and PEP 257 for readability and documentation clarity.

How You Work:
Understand what the user’s code does, how it works, and why it was written that way. Provide step-by-step reasoning and explanations suitable for learners or developers wanting a deeper understanding.

Input, Code Understanding (think):
Identify what the user wants you to do — explanation of function(s), logic flow, error behavior, performance analysis, or best-practice commentary.
Read and interpret the code line by line.
Determine the purpose of each function, variable,  and statement.
Mentally simulate execution to see how data moves and changes.
Note anomalies, inefficiencies, or style deviations relevant to best practices.

Output Requirements:
Explain the code’s purpose and internal logic in clear, structured natural language (dependencies, passing, variable definitions, function calls, and data structures (dictionaries, tuples, lists, bytes, strings, values).
Describe what each part does and how values flow through it.
Mention any subtle or advanced Python mechanics (e.g., context managers, decorators, comprehension syntax) in simple terms.
Comment on style, structure, and coding practices.
Relate the implementation to PEP standards and good software design patterns.

Summary and Recommendations:
Offer a concise summary of what the code accomplishes.
Mention limitations, potential bugs, or improvements in readability, efficiency, or maintainability.

[bronze wyvern]

Hello, can someone explain how does image size parameter affects training pls.

Say I have images with original dimensions of 512 by 512 and another one with dimensions 320 by 320.

My question is, during training, we would need to set an image size, say 640 by 640, so our images will either get upscaled or downscaled.

How does the scaling affects our training? Is it better to upscale/downscale or neither? Is there any compromise we would assume in these scenarios please.

[spring field]

I mean, if you downscale your input image, you're literally losing data that could be used in training instead

But it's probably not thaaat big of a deal anyway

[waxen kindle]

It may be, especially if you your scale is different following x and y

Dowscaling makes you lose information, upscaling dilutes information

But I think you'll still benefit from it, as you'll probably get better performance with transformed image than without jsing thos images

[deft dove]

Hello everyone! 👋
I'm a complete beginner in programming and I'd like to learn about data science and AI. I'm looking for a group or people who are passionate about the topic and willing to teach. I'm not looking to get paid—on the contrary, I'm willing to help with whatever you need while I learn from you.
If anyone is working on projects and needs an extra hand (even for basic tasks), I'd love to join and learn in the process.
Thanks for reading! 🙂

[north zephyr]

Will you have the meta data of the file? so you know aperture, lens, exposure time, etc. you'll want as much data as you can get so you're only looking for one unknown. Distance to subject.

[opaque quiver]

yea i got those parameters

[serene scaffold]

!warn @rotund moss your message was removed for asking for a job, which is not allowed.

[arctic wedge]

:incoming_envelope: :ok_hand: applied warning to @rotund moss.

[bronze wyvern]

yeah I see, so when we downscale images, there are some of information that we accept to lose?

[fading lantern]

I never knew anything about prompting, but after reading this I think if you can write this much out about the process, you should be able to learn coding. The question is if you're motivated. IMO prompting appears to be a type of planning state; for example, in a flow chart there are several planning states 1. problem definition/requirements, 2. planning/design, 3. implement (code), 4. test/debug, 5. refactor/optimize, 6. deploy, 7. maintain/update. It seems what people are wasting a lot of time on these days, called prompt engineering, is basically step 1.,2. Just falling short of step 3. implement (code). Seems like a waste of time and energy to get so close to coding and then not learn it, hoping the AI agent can, which we know many times it doesn't.

[mossy pond]

its only a fine tuning 😉
but yea i dont have the brain for coding but i have lot of knowledge how to ask and can read the code and 80% interpret but to come up with some lines start on a white paper... no iam not able to do ^^
you can see my two links in Bio ... i have coded one PDF parser with GUI waooo and a smart windows task tool all with Ai 😄

[fading lantern]

no I mean, it's like an oxymoron. I guess prompt engineering is in its early stage of AGI. Right now, it looks like step 1 and 2 in a software design flowchart. In fact most programmers say they spend most their time in planning, not coding (that's not me, I prefer ds/algo/arch theory). Maybe their idea is AGI will get better at understanding the intention in human prompts and then they will drop this label prompt engineer.

[mossy pond]

i see... so it is the system-prompt ~10% in background (as a kind of memory) ... and than i start directly my main question ... like "show me a code to play a game tic-tac-toe, user vs computer" ... ok thats a very simple task but depends on model, one of five the result is okay!

[fading lantern]

I think algorithms are a silver bullet anyway. Learn how they operate and everything falls in place, I think. It's like the core between architecture and data structures.

[fading lantern]

Recently I read that computer science is called computational thinking, a field under cognitive science - A way the human brain can solve problems. I never thought of it that way before. I thought it was just for machines.

[mossy pond]

for me its a tool ... like a simulation software... i dont need to know every formula and must calculate every step but i must know what need for input to get a nice output ...

[fading lantern]

yes, cognitive science is very interesting. One day the AI agent will be better at human requests. Right now though I still think the best result is formal training in software engineering. Do you study prompt engineering? Or how do you approach it?

[sly isle]

Hey all 👋, I'm looking to see if anyone is interested in collaborating on a machine learning project for Clash Royale 👑

I have two main ideas I'd love to explore:

1. Deck Optimization: Using ML to analyze replay data or card stats to determine the 'best' meta decks or find high-performing, undiscovered deck combinations.

2. Optimal Gameplay Agent: Using deep learning (like reinforcement learning) to train an AI that learns the best way to play a specific deck, focusing on perfect card placement, timing and elixir management.

Quick note on the approach: The plan is not to use computer vision or read the live game screen ❌ Instead, the idea is to build (or use) a headless simulation environment ⚙️ This would let us run millions of simulated matches, making the complex calculations for an ML/RL model much faster and easier to manage.

Curious if anyone with experience in Python, ML (PyTorch), data analysis or game simulation/automation is interested in tackling something like this 🚀

[inland yacht]

how do i describe an ml aglorithm vs model

[inland yacht]

yoo i was doing the same thing, i looked into the simulated thing but its much harder compared to cv

[agile cobalt]

I'd say that 'an algorithm' is a set of operations (architecture), a 'model' is a set of parameters (weights)

e.g. linear regression is an algorithm you can use to model multiple datasets

[wooden hill]

Anyone here have any thoughts on using json files as the database storage for a digital assistant program?

[sweet prawn]

is there a book/paper yous would recommend as a introduction to/fundamental of AI inferencing?

[inland yacht]

what would you call the thing weights are attatched to, like y = mx + b for linear regression or neural networks

[sweet prawn]

edges?

[inland yacht]

like the whole equation itself

[sweet prawn]

isnt that just a neuron

[inland yacht]

ok ok so from my understanding you start from a model with random weights like
y = mx + b

then you try all the data points and check some loss function

then the optimisation algorithm updates weights and repeats

then it reaches local minimum

is linear regression the model or the algorithm

[sweet prawn]

isnt the optimisation algorithm a different thing

like SGD

[inland yacht]

yeah thats what i was thinking

idk what the normal algorithm is

[sweet prawn]

https://people.cs.pitt.edu/~milos/courses/cs2750-Spring03/lectures/class6.pdf wikipedia links to this

for linear regression models

[inland yacht]

ok so linear regression is the model

and theres no linear regression algorighm

[sweet prawn]

and it appears that you can either solve the linear equations

or use SGD

[inland yacht]

ty

and when you cant solve analytically you have to use an algorithm i guess

[sweet prawn]

it is probably to difficult to solve analytically

or perhaps theres no solution

[inland yacht]

in more complex models what converts raw input to parameters?

[sweet prawn]

seems like no one else is online right now

sadly i dont know either

[inland yacht]

💔

ty for the help so far though

[velvet ice]

from utils.MathFunctions import MathUtils

class Line():
    def __init__(self, weight: float, value: int = 0):
        self.weight = weight
        self.value = value

class Neuron():
    def __init__(self, lines: list, bias: int = 0):
        self.lines = lines
        self.bias = bias
    
    def output(self, input: list[Line]):
        sum = 0

        for line in input:
            print(line.weight * line.value)
            sum += line.weight * line.value

        sum += self.bias

        return MathUtils.sigmoidify(sum)



class NeuralUtils():

    def minor_cost(output: int, expected_output: int):
        return (output - expected_output) ** 2
    
    def major_cost(outputs: list[int], expected_output: list[int]):
        overall_cost = 0

        for i in range(0, len(outputs)):
            overall_cost += (outputs[i] - expected_output[i]) ** 2
        
        return overall_cost

Hello there, I'm trying to make a neural network by only using mathematics and basic python. I'm planning on making a HiddenLayer class to make it easier to deal with neurons and lines. Any suggestions or recommendations on how I can do that?

Btw, sigmoidify is a function which converts large numbers into a number between 0 and 1.

[sweet prawn]

lol why are you asking people here if you dont think we know what sigmoid functions are

[fading lantern]

if you choose a simple formula, it's likely to be both. Unlike a modern neural net which is more likely a model.

[inland yacht]

hows a linear regression algorithm different to a linear regression model

[fading lantern]

they're not, it's context. A model is different from an algorithm. It depends how you're using the linear regression

[inland yacht]

huh

so is there such thing as a linear regression algorithm?

[inland yacht]

so whats the difference

[fading lantern]

why didn't you ask an AI agent

[inland yacht]

"Linear regression algorithm = the entire procedure of training a linear model. It includes:

Choosing the form of the model (y = mx + b)

Defining a loss function (MSE)

Finding the best parameters (m, b) using some method."

yeah idk if this is right

so either the algorithm is the whole process of training or its a misused term

[fading lantern]

why do you want to know?

I assume you're trying to understand the difference with model and algorithm

if you study 'data structures and algorithms' (DSA) you definitely will know the difference.

[inland yacht]

its for a test

cause people say linear regression algorithm i dont know wat it means

cause theres no such thing

[fading lantern]

if linear regression is only y = mx + b I'd say it's a model, not an algorithm.

[inland yacht]

exactly

[fading lantern]

algorithms are large tools, closer to architecture.

[inland yacht]

?

wdym

[fading lantern]

models are closer to data structures

they're all calculations (tools), just different sizes

algo's are larger scale, you can find a list

[inland yacht]

like in the context of machine learning

i dont know why people say linear regression algorithm

when its like a gradient descent or ada

[fading lantern]

In CS the same tools are used most of the time

[inland yacht]

this not ml though

[fading lantern]

above are the most common algorithms

ya but you asked if linear regression is an algorithm, it's not in the list of algorithms above

linear regression is not an algorithm

[fading lantern]

they're incorrect, simple as that.

[inland yacht]

ok ok thanks

[fading lantern]

you're right, they should say linear regression model. Unless they're referring to linear search. But, that of course, is not y = mx + b.

[inland yacht]

is it an important distinction or am i just being pedantic

[fading lantern]

algorithms are an important distinction, absolutely.

[inland yacht]

sorry for so many questions but in labelled ml with complex input like images what determines the parameters?

i got a test tmrw

[fading lantern]

have you tried asking an llm?

what type of model are you using

[inland yacht]

lets say a cnn

fast rcnn

[inland yacht]

its not explaining it well

[fading lantern]

lmc

[inland yacht]

just all the layer stuff idk

[fading lantern]

looks like labeled data, loss function, optimization algorithm, model architecture (initialization and pre-training, hyperparameters and architecture choices)

from that millions of parameters are then generated, gpu intensive.

[inland yacht]

so must a human always define layers

[fading lantern]

modern approaches are automated

I presume you have gpu access?

Practically speaking it seems a human always defines the layers, but then the parameters are automatically learned

problem is, with millions of parameters, you will certainly need gpus

Additionally, you may like to search for modern techniques that do automatically search for good architectures. Such as NAS, AutoML, etc.

These architectures are probably very well known around the GPU networks. Such as NCP-AIN, NCP-AII

my favorite; https://www.nvidia.com/en-us/learn/certification/

beats the hell out of aws certs, IMO

AWS has not reported recently making their own chips, so...

[wooden sail]

in estimation theory, the algorithm doesn't matter

any algorithm with convergence guarantees is fine

linear regression is the process of fitting a linear model to data. the cost function gives your estimator particular statistical properties, but the patticular alg doesnt matter

the analytic solution for the least squarest cost is a pseudo inverse. gradiwnt descent will give you the same solution if you let it converge

[inland yacht]

so gradient descent or whatever is the algorithm

that optimises linear model

theres no linear algorithm

[wooden sail]

there is such a thing as linear algorithms, but it's not really important here

regression roughly means "fit parameters to explain data"

"linear" here means that you're using a linear model, so the params you fit are the params of a linear model

how you dp the fit isn't important so long as you actually minimize the cost func

[inland yacht]

what do people mean when they say linear regression algo?

[wooden sail]

i've never heard anyone say that

maybe they mean one specific implementation of lin reg

i would say that that generally means something was mosunderstood though

[inland yacht]

its in my syllabus

"Describe types of algorithms associated with ML
Including:
K-nearest neighbour
linear regression
logistic regression"
or maybe i misinterpreted

[wooden sail]

idk to me these aren't algorithms

unless the focus of this part of the syllabus is to study several implementations for each of the problems listed

[inland yacht]

maybe it is?

[wooden sail]

e.g.comparing recursive least squares to grad descent to pseudo inv

but on their own, none pf the things in your list are algorithms

[inland yacht]

yeah probably misworded then

ty

[wooden sail]

those aren't algorithms

you can do the minimization in different ways

these aren't mathematically rigorous instructions, they are optimization problems

the way you solve them is the rigorous instructions

.latex if i write
$$
\min_{\bm{x}} \Vert \bm{Ax} - \bm{y} \Vert_2^2,
$$
is that an algorithm?

[wooden sail]

(it isn't, it's the definition of a specific problem, which matches what you just wrote: not an alg, just a problem)

[serene scaffold]

what does the 2 2 mean?

[wooden sail]

yes, now which algorithm will you use to solve the OLS problem?

[wooden sail]

2-norm squared

no, what i wrote is OLS

this is wrong

did you read the reference? that part is copy pasted from geeks for geeks

it's the only part where it's called an algorithm, and it's taken from a random website

linear and logistic regression are not algorithms, they are models or estimators, depending on whether you consider just the mathematical expression of the model or also the cost function

completely separately, you also have to choose an algorithm to actually do the minimization

the problem doesn't solve itself

you can't because they aren't

show me a nice paper or a book

i won'T take geeks for geeks as a valid reference

[inland yacht]

logistic is linear with the sigmoid function right

[wooden sail]

it does make a lot of sense because it gives the impression there is only one way of doing it, and there isn't

what is the point here? this says the same thing i'm telling you

[inland yacht]

ty

[wooden sail]

the whole paper is about comparing several linear regression algorithms

let's see, lemme check whether they use lin reg to refer to the model or the estimator

[inland yacht]

how do u decide architecture hyperparameters like degree in polynomial regression

[wooden sail]

you were the one that asked

but you'll be happy to know the authors don't distinguish between the two things

[wooden sail]

the overall problem is called "model order selection". you can do things as simple as training several models of different degrees and keeping the one that performs best (something in the direction of "cross-validation"). there are more complicated things like information-theoretic criteria. things like the akaike information criterion are pretty common

[inland yacht]

ty all!

[hallow kettle]

I know guys, the question could be out of league but the Variance has caused a lot of pain when I was Trying to understand it a week ago. I am still not confident that i understand it completely

[wooden sail]

what troubles you about it?

[arctic smelt]

Variance is just a fancy word for how spread out a bunch of numbers are. That's it. Are the numbers all clustered together, or are they all over the place?

[mossy pond]

no ... i only play with it 😉

[hoary nova]

Heyy I wanted to ask is it necessary to go through ml book with libraries of python or can we go with like data scientist or data analysis book with python libraries??
Are the content different...??
Being beginner and starting with libraries so i am little bit confused which book will be good...

[serene scaffold]

@hoary nova the distinction between data science and ML isn't consistently applied. Focus on learning about concepts and how to apply those concepts in python

[hoary nova]

So which book should I consider to go through and study

[velvet ice]

Just in case someone doesn't know it : 3

But pls help me out lol

[lapis sequoia]

ISLR, NLP Standford. ISLR is free, that is the bible.

[scenic parcel]

How long does it take to do a kaggle comp

[agile cobalt]

it varies, same as any other project

you could do a bad job within a few hours and be done with it, or keep working on the same project for over a hundred hours

[waxen hill]

anyone use pytorch for ML? and can confirm whether pytorch or tensorflow is better?

[wooden sail]

people have favored pytorch over the last couple of years. tensorflow has introduced breaking changes that make compatibility an issue, and it also does not have gpu support in windows

tensorflow uses XLA as its backend and used to have keras as a simplifying layer on top, but both things are very standalone nowadays. that means you can use keras on top of pytorch, and you can also use XLA as a backend for pytorch. most of the stuff that makes tensorflow nice can be used without tensorflow 😛

[waxen hill]

Appreciate your answer to what is probably an extremely amateurish question on here 🙂 (I'm new to ML)

[wooden sail]

not an amateurish question at all, there are a bunch of ML libraries and tools, and more come out all the time. navigating all that stuff never gets easy

[wooden sail]

which platform are you on btw? like which operating system and hardware

[waxen hill]

I use MacOS 🙂

[wooden sail]

aha, this also makes a difference

i think pytorch has a more mature installation procedure on macos

[jaunty helm]

to add to that, google has moved most of its attention from tensorflow to jax now I believe

[gloomy fulcrum]

I use MacOS as well!

[knotty mortar]

same

[eager lance]

is alex the analyst any good?

[agile cobalt]

never heard of them before but their videos seem all over the place in a bad way

definitely avoid videos longer than 1 hour
read the official User Guides of any libraries you want to use before looking for videos about them

[eager lance]

i dislike reading docs, even right now struggling with kaggle reading and processing all this stuff is tough

[agile cobalt]

reading documentation is single handily the most important thing you have to do in order to learn how to do things

read only the most important parts (quickstart, introduction, main user guides etc.) then consult the rest as you need of it though, don't try to read everything before you even start using it

[eager lance]

well i've went past that point but using a bit of kaggle and a bit of youtube (bro code) i know how to create series and dfs, import data, grouping and aggregations, maps stuff like that

selection, filtering yk

[agile cobalt]

also found a bunch of other people rating it as meh in https://www.reddit.com/r/dataanalysis/comments/18vx93w/alex_the_analyst_analyst_builder/
can recommend Coursera myself (especially Google's course), DataCamp is also fine, the maven comments sound almost like they're being paid to promote it though

[eager lance]

and i'm doing a project right now but i feel like i'm about to hit a wall

[eager lance]

yeah i saw that sub, thank you for the info

is it okay if i ask ai to write me the code when i get stuck on this project and then learn from it?

or should i stop using ai

[agile cobalt]

using it to convert logic into code is fine if you don't know the syntax / the appropriated methods, assuming you can review the code afterwards to ensure it does what you asked

I'd avoid asking for it to come up with the logic for you

(e.g. represent an operation in pseudo-code or describe in English in detail and ask for it to implement it using a given library)

[eager lance]

well for example i've stumbled upon an issue when using pandas and i was using my messages logs on instagram and at one point i didn't know how to count the number of words in a message, so i asked for that

[waxen wagon]

only when you're out of time due to deadlines

[ebon steeple]

hi

[waxen hill]

Anybody taken some DS courses on Coursera? Which one is better? IBM for Data Science or Google Advanced Data Analytics?

[proven pier]

I'm looking into normalizing my data. I have a problem, however. My data is bound by 0 and infinity. I do not have any data in history getting close to infinity, but I want to be able to apply a proper scaling pattern regardless

Say, who knows if an input in the future will be very large

[waxen kindle]

I think you can still try a standardisation

Maybe, if relevant, applying log before standardizing

To move from (0, +oo) to (-oo, +oo)

Then Z standardisation with mean and std

[proven pier]

Wow, that is pretty neat. Sounds like what I was looking for. I'll see hos Z standardization can be used, but I'm afraid over time as environment effects change my mean will shift further in one direction or the other. I presume Z standardization is on a discrete set of viewable data

If I design a model based on viewed data, who knows how it can change over time as I'm running inference

[waxen kindle]

That's a whole different problem

That can be fixed either by regularly finetuning the model with some new data, or by doing some kind of transformation in the data to fit the target domain

Your training dataset should match as much as possible the distribution of the real values

[vale field]

Hey guys, basically im using pandas to try count the number of " from a specific column in a csv but I end up getting count as 0 even tho the count shouldn't be 0, can someone tell me why?

[slate trench]

Sharing your code could help. csv or pandas dataframe?

Maybe this helps: df = pd.read_csv('file.csv', quoting=3) as pandas by default treats " as field delimiters not character

[vale field]

dataframe

`df = pd.read_csv('./train.csv')
for i in df["comment_text"]:
if i == '"':
result = True
print(result)

counter = 0
for j in df["comment_text"]:
if j == '"':
counter +=1
print(counter)`

[slate trench]

did you try quoting=3

[vale field]

no, what does that do

[vale field]

nvm i found out what i needed to do i think, thanks

[buoyant reef]

how do I make my own ai?

like a small one

that I could train on a mac intel 2018

[bleak zealot]

Hey guys, so im making a small AI trading agent atm.

My entire AI runs, and learn and keep learning on new data and trade it makes etc. Everything runs fine as it should, but i have a little problem

Im trying to fix my risk vs rewards vs penalty in the reward part of my script, cause currently the drawdown is way to high.

As seen in this picture my penalty for drawdowns isnt good enough, since i have very little drawdown succes compared to the buy and hold (baseline)

But my problem is every time i either change my penalty to be higher on drawdown, it takes way less trades, and become way to conservative,

If i reward it more to hold money while their is drawdowns, it just hold money way to much!

Could some one send me towards some advanced to elite rewards vs risk vs penalty system in Gymnasium, PPO agent, to read/watch about thanks cause im so lost in this part of my AI.

Thanks in advance 🙂

AI peak value is +400K from 10.000 but as you see major drawdowns (Drawdowns = DD on screenshoot)

[quaint mulch]

what's wpm assistnat?

congrats on ur new job

[quaint mulch]

https://iamtrask.github.io/2015/07/12/basic-python-network/

[plush shuttle]

guys

can anyone tell me where to start

[errant bison]

Hi

[hard brook]

Halo, saya Debby dari Indonesia, saya saat ini sedang belajar dasar-dasar python, apakah ada yang mau menjadi mentor saya dan berteman dengan saya?

[gritty notch]

Hello

[errant bison]

ABOUT
AI-driven communication and automation system to streamline daily interactions across the business network. Aim to implement a Smart Chatbot + Automated Messaging Platform that can manage client communication, schedule messages, and handle basic queries intelligently.

Core Requirements

Daily Automated Messages: Send personalized “Good Morning” and update messages to around 700 people daily.

AI Chatbot Functions: Answer queries about pricing, stock, delivery points, documentation and brand availability.

Automation & AI Integration: Use AI agents for automatic follow-ups, reminders, and updates.

Dashboard & Control Panel: Monitor message logs, query responses, and analytics.

Integration: WhatsApp Business API + Email + Website Widget + Google Sheets/CRM.

Data Security & Compliance: All client data to remain secure within country based servers.

Can anyone give me rough flow of idea and tech stack which i can use

[plush shuttle]

umm for frontend or backend

[plush shuttle]

mein nhi janti, kya nhi janti???

[versed axle]

!rule english

[arctic wedge]

Rules

4. Use English to the best of your ability. Be polite if someone speaks English imperfectly.

[hard brook]

Hello, I'm Debby from Indonesia, I'm currently learning the basics of Python, is there anyone who wants to be my mentor and be friends with me?

[versed axle]

perhaps the #python-discussion channel will be more suited in that case.

[hard brook]

Okay

[errant bison]

Both

[plush shuttle]

hmm for frontend use next js and idk about backend

[wide carbon]

Hello

So like anyone interested

[plush shuttle]

In what

[bronze wyvern]

hello, quick question, is there a difference between model evaluation and error analysis when it comes to ML?

[serene scaffold]

Yes. Error analysis is specifically when you examine errors to understand why they're happening

Model evaluation is more general.

[bronze wyvern]

oh ok I see, like understanding why a particular image gave us a false positive?

[serene scaffold]

Right

[short imp]

guys i have learnt pandas numpy seaborn matplotlib

what should i learn now in ML to make a model ?

[serene scaffold]

what are models, according to your understanding?

[short imp]

according to me model which can perform automatically after one time apply

[serene scaffold]

that's not right. try reading about that some more.

[short imp]

can you tell me ?

[short imp]

??

[pallid sequoia]

lmao

most helpful discord mod

[exotic smelt]

God forbid a man have a life outside of discord

[velvet notch]

Hi guys

Can someone guide me on how I can post a help request here (newbie here)

I actually posted on yesterday but it got closed for some reason idk🥲🥲🥲

[limpid zenith]

what do you need help with? more details help

[versed axle]

help threads close automatically after an hour of inactivity.

[velvet notch]

Oh

[versed axle]

head to #1035199133436354600 and fill out the input boxes that appear at the top

[velvet notch]

So I need to msg in that every hour?

To keep the thread active?

[versed axle]

I suppose, you could always just remake the thread when you come back

help threads are usually fairly active

[velvet notch]

Okay

Thank you so much

[lime grove]

another reason to hate Pandas: whomever runs that module thinks that it is perfectly acceptable to insert a f***ing FutureWarning when doing certain things. Can't get rid of this without importing the warnings module.

[agile cobalt]

it is perfectly acceptable
that is telling you your code is likely to break in future versions / if you update automatically

what do you think that warnings exist for?

[lime grove]

opt-in instead of opt-out

that is the right pattern

if you want to see the warnings, especially if no code is going to presently break, you opt-in

this is preferable over ending up with a hard to dispose off mess that sends you in unproductive tangents.

I guess that a lot of this comes from it being a project that began poorly managed, and is now going into this sort of developer-as-a-tyrant path to deal with all this technical debt resulting from a god-awful UI

[short tinsel]

huys is there any sort of website which teaches me ai, so like basics, what are neural networks the math etc

[errant bane]

the math takes years to learn

the closest thing to a "quick rundown" I know is 3B1B's videos on it, which won't teach you much, but it will give you an idea

[tight prawn]

hey

[weary drum]

Hii!! I know basic python but I want to get into data science. any resources or tips?? thanksss

[pallid sequoia]

he didn't even answer the question...

[arctic wedge]

Resources

The Resources page on our website contains a list of hand-selected learning resources that we regularly recommend to both beginners and experts.

[tired vine]

Yeah so what's the prerequisites for cv in python

So after knowing fundamentals of python can we do cv or image processing pytesseract

[bronze wyvern]

Hello, quick question, say I have a multi classification model for 6 classes. I calculated their precision, recall, accuracy and f1 score separately. Now I want a single metric representative of the entire model, what should I use pls. Do we have another metric that does so or I need to average, say f1 score across all 6 classes?

[bronze wyvern]

Micro- and macro-averages (for whatever metric) will compute slightly different things, and thus their interpretation differs. A macro-average will compute the metric independently for each class and then take the average (hence treating all classes equally), whereas a micro-average will aggregate the contributions of all classes to compute the average metric. In a multi-class classification setup, micro-average is preferable if you suspect there might be class imbalance (i.e you may have many more examples of one class than of other classes).

Here, is the user right here when he mentioned about using micro average for imbalanced dataset rather than micro average? Just read that on a data science forum

[wheat umbra]

Hi. Im currently working on a PacMan RL-Project but im really stuck in training. No matter what i change, the agents performance does not improve.
I have looked into several similar projects but they all seem to use a prebuilt env. I have built the PacMan game myself tho.
Does anyone have expierence on these kinds of problems or any tips for debugging RL-models in general ?

[plush shuttle]

so guys

can we fine tune a hugging face ai model

[serene scaffold]

Yes

[lapis sequoia]

yes, bert, roberta, T5, Bart, GPT2 and many many more. Hugging Face for life.

[short imp]

@serene scaffold i am back with the answer !! for me till now , model isn't just a project but its the step towards the understanding the mechanism or purpose that how machines can think and learn like humans, a model is like a brain that we train to do specific task

[vale field]

Quick question, basically im working with bigrams and I noticed that some of my extracted bigrams are sometimes when you look at it 3 words instead of 2 words where one of the words has no spaces which is why it is treated as a single word instead of 2 words e.g. yourselfgo, duck. Any advise how to solve this?

[serene scaffold]

the boundaries between words aren't actually absolute. you have to use a tokenizer that assigns word boundaries according to where you think they should be.

[serene scaffold]

describing models as a brain might be giving them too much credit, but you're on the right track.

[silver hearth]

In Vibe coding, these model generate buggy code making api calls which don't even exist. When you ask them to correct, they make up other API calls that don't exist. Anybody finding Vibe is more effort than results?

[ruby fog]

use MCP tools like tavily, context7

[proven pier]

Does anybody have a good resource on learning monte carlo simulations?

[vapid jacinth]

Hello, let’s say I would want a discord bot to scrape a specific game version on google play store and IOS App store, is there any way this is possible? I am currently using uptodown to get the game version but uptodown is quite delayed and i want a quick way of getting the game version and sending a notification on discord when it detected a new version.

[waxen kindle]

don't they have an official API that you can call once in a while ?

[warped shale]

Yall got any good recommendations for websites that have quality reaserch papers on cnns and vits?

[vapid jacinth]

Ye i found it

[abstract wasp]

hi does anyone know how to get distillbert to work? ive been trying to get the tokenizer to work but i always get a 404 error :(

from transformers import AutoTokenizer
tokenizer = AutoTokenizer.from_pretrained("distilbert/distilbert-base-uncased")

RemoteEntryNotFoundError: 404 Client Error. (Request ID: Root=1-6909268e-053a2c83200a40e9756e0203;4ce2bfc2-0f96-4d81-887c-a46ea9531bf6)

Entry Not Found for url: ``https://huggingface.co/api/models/distilbert/distilbert-base-uncased/tree/main/additional_chat_templates?recursive=false&expand=false.
additional_chat_templates does not exist on "main"

[serene scaffold]

looks like the name has changed to "distilbert/distilbert-base-uncased-finetuned-sst-2-english" instead of "distilbert/distilbert-base-uncased"
based on this: https://huggingface.co/docs/transformers/en/model_doc/distilbert?usage=AutoModel

[abstract wasp]

ive tried it already but i get the same error :/

[serene scaffold]

it might be an issue with the version of transformers that you have installed: https://stackoverflow.com/questions/79786866/remoteentrynotfounderror-with-downloading-models-from-hugging-face-in-kaggle

[abstract wasp]

i think it finally worked ty!

[proven pier]

So can the monte carlo simulate this data? All it does is take the mean and std deviation and go from there?

[plush shuttle]

who wants free api keys??

@versed axle do uwant free api keys

[harsh warren]

Hello everyone,

I am working on a time series project with 8-hour intervals. I have 8 variables, some of which are highly correlated. After exclusion, I have 4 variables left. I want to predict the average of one variable between t+1 and t+7, using data available up to time t.

Do you have any suggestions to help me progress? I have already tried several approaches without success.

Thank you in advance!

[wooden sail]

what have you tried so far? the simplest method i'd start with is probably a linear predictive filter

or what you might know as "autoregressive model" or yule-walker equations

[opaque condor]

class Image_Module(nn.Module):
    def __init__(self):
        pass

class Audio_Module(nn.Module):
    def __init__(self):
        pass

class text_Module(nn.Module):
    def __init__(self):
        pass

class Threed_Module(nn.Module):
    def __init__(self):
        pass

#feeds into the main ai
class Main_AI(nn.Module):
    def __init__(self):
        pass

[cerulean kayak]

I know most programmers think memorization is obsolete, but assuming I have to,
How the heck did you guys memorize the confusion matrix metrics formulas?

because every time I put it off, it only shows up agian.

[serene scaffold]

You mean like, precision and recall?

[lime grove]

Anaconda Navigator: Tensorflow won't install, all kinds of dependency failures, but Pytorch does

any insight?

[serene scaffold]

do not try to use anaconda

[lime grove]

well, I am in the process of learning that hard lesson

[serene scaffold]

learning that hard lesson. would you be fired if you stopped using it, or something?

[lime grove]

no, this is personal work

I am currently looking for roles

[serene scaffold]

then just stop trying to use it immediately.

if someone applied to my department, and they listed anaconda as one of their skills, that would be a major red flag for me

[lime grove]

why?

[serene scaffold]

because there has been no reason to use anaconda for over five years, so anyone who still uses it is basically stuck in the 1800s.

[lime grove]

ok, so the reason is stagnated skilset

[serene scaffold]

it's also banned at my company because it is pointless, and we didn't want to pay the anaconda company for a license.

[lime grove]

I get the Anaconda hate. The thing is clunky, massive drain on CPU, slow AF, virtual envs are stupidly implemented

etcetera

[serene scaffold]

so why are you asking for help using it?

[lime grove]

that is a pretty dumb question.

don't assume I started coding python yesterday. But Anaconda is just a new tool that I am somehow working in for no real reason other than it is there. So, I am not familiar. Now, if I lack familiarity with something, I ask questions.

With that in the background, don't you think it is stupid to ask what you just did?

Hostilizing people isn't helpful

[serene scaffold]

I don't think you started coding yesterday.
The fact that anaconda exists isn't a valid reason to try to learn how to use it. It would be a waste of your time, for the reasons we've discussed.
If you want help learning how to use uv, I would be happy to help.

[serene scaffold]

as far as this question is concerned: without seeing the error message, it might be because tensorflow development has been winding down, so it might be losing compatibility with other libraries in ways that aren't getting fixed.

[lime grove]

Yeah, I think that this might be the case. PyTorch installed on the hated Anaconda Navigator without an issue. I mean, no issues yet

[serene scaffold]

once you understand what precision and recall are, it's not hard to remember the formulae

it's okay if you get precision and recall mixed up for a while

[lime grove]

all I need to do is make an equivalence between the Tensorflow version of LSTM and the PyTorch version of LSTM

probably a good exercise.

and, tbh, even uv might not save me from hassles.

[cerulean kayak]

My main problem is identifying which is a false positive vs a false negitive

[serene scaffold]

How many classes do you have? Two?

[cerulean kayak]

sometimes, I have to be able to do it for both multiclass and binary classification.

[serene scaffold]

Okay. So suppose your classes are A, B, and C

If the model says it's an A, but it's actually a C, that is:

• a false positive for A, because what the model actually said was false
• a false negative for C, because the model didn't say it was C

Make sense?

It's also a true negative for B, but we don't care about that

[cerulean kayak]

ya so the way this guy online is doing it is looking at each class and saying

• what's the Tulip's TP
• what's the Tulip's FP
  ect
  and finding out the 4 numbers for each class

but my question is why do they call the false negitives of Tulips 1, and F+ 4?

[serene scaffold]

consider the circled cell. complete this sentence: The model said it was ____, but it was actually ____

[cerulean kayak]

rose and sunflower:
so F+: rose & F- sunflower?

[serene scaffold]

that's correct

[cerulean kayak]

okay if I do logic for like 3 mins I can work out a somewhat-understandable def. for the 2. Thanks a bunch.

[plush shuttle]

guys could i get a link to an ebook to get started

[rich moth]

howdy

what would you guys do with a verifiable provenance & attestation DAG? looking for ideass

My first thought was a repository for datasets for ML/AI. The idea is its "proof of existence" for any type of data, but a merkle anchored provenance dag with polygon anchors. It would be like Docusign, just for datasets.

[plush shuttle]

..

[plush shuttle]

i dont like making datasets my self and i dont even know how to

[rich moth]

its an important science. but ya its very tedious I imagine. I just test them. 🙂

[plush shuttle]

oh

[rich moth]

But I can, its just not my focus. What are you working on?

Im working on a context aware data pipeline, but then got distracted to make that verifiable provenance and attestation dag, it was suppose to be a tool for something else lol.

[plush shuttle]

Ik basic pytorch and im learning tensorflow

[rich moth]

what are you building right now?

[plush shuttle]

nothing, I want a prjct

[plush shuttle]

bruh i dont understand those big words

[rich moth]

DocuSign forr AI compliance. Because of EU AI Act i think companies are desperate for this technology

[plush shuttle]

did u see the ctm ai

[plush shuttle]

ohhhhh

[rich moth]

Whats that?

Imagine any dataset used by AI models, healthcare, finance can proof its "proof of existence". New laws in the EU are required good safe data. What if you could, and easily prove and verify this, instantly and anywhere.?

Then this "proof" becomes a global , shareable asset thats secured in the blockchain.

[plush shuttle]

continous thought model ai by sakana ai lab in tokyo

[plush shuttle]

good idea

[plush shuttle]

yup

did uk hardware nn are more powerfull then the simulated ones in our comp

[plush shuttle]

bro u r still typing

[rich moth]

Processing power seems irrelevant at some point. Sorrry got distracted cooking dinner

[plush shuttle]

Ok no problm

[rich moth]

im a big fan of platonism

[plush shuttle]

???

whats platonism

like takin extra plates

can someone gimme a usb to sata cable

[rich moth]

that intelligence isnt a destination, its the journey. i feel like its the process of discovering truths of your environment. take for example 1+1=2. Its not invented its part of the universe.

[plush shuttle]

oh

bye i m gonna learn some more ml and ai

[rich moth]

cya

anyone else know what im talking about? Lets use a dog for example though, a puppy that learns and grows playing fetch. hear me out. A dog doesn't have any idea what math is, it doesnt know Newtons laws, parabolic arcs, or any ballistic trajectory analysis stuff, but it does have tacit knowledge. It learns fetch through iteration ( some models more problematic) but its build an intutive, physical models of its itself and its envirorment and the truths within. But all that is extracted in their brain, the "rules of phyics" through a lifetime of experiences.

[silk pendant]

Hello, I'm trying to create graphics for my paper using matplotlib. I want to illustrate the concept of a viewport with an azimuth and elevation.

I want to plot a rectangle centered at the end of the elevation semicircle, facing the origin point

rectLength = 1/9*16
rectWidth = 1
x = np.array([np.linspace(-rectLength/2, rectLength/2, 1920)] * 1080)
y = np.rot90(np.array([np.linspace(-rectWidth/2, rectWidth/2, 1080)] * 1920))
z = np.full((1080, 1920), 1)

x *= np.cos(offsetTheta)
z = z + x * np.sin(-offsetTheta)

y *= np.cos(-np.pi/3)
z = z + y * np.sin(-np.pi/3)

ax.plot_surface(x, y, z, color='gray', rcount=1, ccount=1, alpha=1, zorder=1)

I've managed to successfully rotate the rectangle array by the azimuth, but when I try to do the same for elevation it doesn't work. How can I rotate the rectangle by the elevation correctly?

[plush shuttle]

oh intersting

[plush shuttle]

go in #1035199133436354600

[plush shuttle]

Guys

[rich moth]

Whats up?

[plush shuttle]

Can someone write a code for loading a dataset from hugging face and train it in pytorch cuz

Its showing err in mine

[long locust]

@elfin robin your message was removed for violating rule 5 of the server

[unkempt apex]

Yo plunder

Long time no see

[plush shuttle]

hi guys

[jaunty helm]

if you don't show the code and the error no one can help

[plush shuttle]

Ok wait

[last ember]

Hello
Can anyone recommend a solid Python course for data science (besides Coursera)?
Also looking for some fun open-source projects to join or learn from

[rich moth]

Zap!! Wait, didnt it use to be Zapbot? Its good to talk to you buddy. Whats new?

[unkempt apex]

thats my username

nothing new sir, just grinding

[rich moth]

ya lets see the code dude 🙂

[rich moth]

Hows school going?

[unkempt apex]

I have graduated

[rich moth]

hell ya, congratz dude. ive just been working , a lot. always on the road.

working on anything lately, as far as projects go?

[unkempt apex]

yes

but not so much productive this days

[rich moth]

im trying to build something in rust for the first time. racking my noodle over here.

[elfin robin]

okay . So sad 🥺 .

[tired vine]

Guys is 1 month fine to do little bit of hackaton level app and web development and image processing,cv

[mellow vector]

oh, dang, well it's worth a look if you're a data mod

[wide carbon]

@plush shuttle

in an ai project hackathon

[plush shuttle]

which one

[wide carbon]

Look its by bits h

under solaris surge their fest

we have to submit a ai project

Anyone interested we can start making we have 5 days

[plush shuttle]

ok

[wide carbon]

do you have ai ml knowledge

just asking man

[plush shuttle]

not much tbh

[wide carbon]

How much

[plush shuttle]

how much people can there be

[plush shuttle]

like basic

if im too basic u can contact @rich moth

[wide carbon]

how much

look we have
scikit-learn for basic ml models or algos
pytorch/tensorflow- for nn

[plush shuttle]

pytorch

[wide carbon]

so like have you made any models so far

[plush shuttle]

i use pytorch

[wide carbon]

noice we could try

cause we will most prolly train an nn on kaggle or so for the project or like use a free llm

[plush shuttle]

not much like 2 - 3

[wide carbon]

best what we can do

no prob

[plush shuttle]

u can use gtp 5 for free

[wide carbon]

now like we need to think what should me make

[plush shuttle]

or u could grab an ai from hugging face

[wide carbon]

no like for assistance ok but like we will use free hugging face models like mistral

yes

look lemme share

[plush shuttle]

oh ok

i was doing some bug bounty but leave it i like this more

[wide carbon]

come dm now

[plush shuttle]

kk

[grand minnow]

I added an AI agent to my Metatrader trading bot and found that it handles better if I add the info it needs into a prompt instead of giving it a MCP or toolset. I was led to believe that the MCP or toolset would make the AI agent be more performative but it kept bugging out for some odd reason. Is that expected? Has anyone had problem integrating their custom MCP or toolset/toolkit to their AI agent?

[chilly oar]

can anyone help me solve this ques

[mellow vector]

so ML... I'm working on a milestone project, I have the cartesian product of range(-10, 11) and am teaching it the sum (think rolling the same die twice). To prep the data I'm thinking I need to augment with noise and oversample pairs like 10 and 10 that occur infrequently. I'm also wondering how to handle 1+1 that is a degree of magnitude smaller than it's less frequent edge case... my intuition is that min-max scaling everything to .25-.75 would reduce the bias but my intuition is also that my intuition is totally wrong on this one... also i'm pretty sure a wide net will outperform a deep net here but i'm hung up on the data prep

[native temple]

Anyone have any recommendations for ML books? Will be listening to it so books with tons of diagrams like the "for dummies" brand is not ideal. Looking for something that covers both theoretical and applied concepts.

[serene scaffold]

as an avid listener of audiobooks, you can't learn ML from audiobooks. you have to look at diagrams and you have to look at formulae.

[native temple]

I understand that, I already do have a foundational understanding of the subject, that's why I figured I would look for something that will just introduce me to new concepts

But thanks for the response anyways!

[mellow vector]

and i just learned that the compliment of the count(sum(a,b)) is the same as the sum of the dice, that was a lot of extra work, ima check out for the day but I'll appreciate any input when I pick this back up tomorrow

[idle stone]

what about regular books?

[serene scaffold]

what do you mean "regular" book?

[idle stone]

"as an avid listener of audiobooks, you can't learn through audiobooks"

[serene scaffold]

you dropped a key phrase from what I said that makes this a statement I don't agree with

I've learned all kinds of things from audiobooks

[idle stone]

the main question of "anyone have any recommendations for ML books" still went unanswered, that's mainly what im concerned with tbh

[serene scaffold]

by "regular" do you mean "not audio"?

[idle stone]

yes.

[serene scaffold]

it wasn't clear if "regular" replaced "audio" or "ML". I thought it replaced "ML".

there are recommended books in the pins of this channel and on our website

!resources data science

[arctic wedge]

Resources

The Resources page on our website contains a list of hand-selected learning resources that we regularly recommend to both beginners and experts.

[idle stone]

beyond appreciated! Yeah I'm just sourcing if people have a niche personal recommendation that possibly isn't listed, in case the learning meta has changed etc etc

i don't know if anyone has a "meta" method for learning pytorch as well?

[mellow vector]

pytorch streamlines the process to the point that chatgpt could probably reliably produce code that would generate a working network/model. I don't know of a meta per-se, but the complexity of how networks operate dwarfs the difficulty of implementing the changes to the pytorch code.

that is to say that if you know what you want it wont be too difficult to achieve. I'm not suggesting that you do or don't use gpt to write code but if you want to optimize it yourself torch is the easy part.

[idle stone]

My main goal is to learn PyTorch, contribute to the open source in a meaningful way, and probably dive into some specialized cuda level. Ideally learning to the point that I could be extremely useful

[opaque condor]

#==={imports}===#
import torch
import torchaudio


#===============#
class modual_image():
    def __init__(self):
        pass

class Modual_text():
    def __init__(self):
        pass

class modual_audio():
    def __init__(self):
        pass

class Modual_3d():
    def __init__(self):
        pass



class Main_ai ():
    def __init__ (self):
        pass

could I import these into the main_ai class

[serene scaffold]

No, because they're all in the same py file.

Main_ai can have instances of those classes (not modules) as attributes, or create them in methods, etc.

[opaque condor]

But my idea was they train separately and then export their jsons into the larger model

[opaque condor]

Let me guess I did wrong?

[grand minnow]

This is such an interesting read https://fly.io/blog/everyone-write-an-agent/

[serene scaffold]

that's not how neural networks work

[opaque condor]

agent

enviorment
simulation

[serene scaffold]

@opaque condor sorry, but that's not right. you're thinking of autonomous systems.

like, a self-driving car is an agent that decides what to do to achieve its goal in the context of its environment and sensory input, and that might involve one or more neural networks, but that's not inherently what neural networks are.

[opaque condor]

I know I just wanted to see if I could merge several smaller networks into a larger one that is using the data from all of the networks learning how to recognize images text even move around in a 3D environment that's why I really want to do

[serene scaffold]

I think you're getting ahead of yourself. it looks like you don't understand concepts like model weights or python OOP.

[opaque condor]

I understand object oriented programming

[serene scaffold]

can you explain what you meant by "export their jsons"?

[opaque condor]

write the weights to json files and feed in to a main network

[opaque condor]

is that a better answer?

[spice tartan]

Guys

[serene scaffold]

What does it mean to feed them into the main Network

[opaque condor]

Import the jsons into the main network

[serene scaffold]

What do you think happens when you do that?

[opaque condor]

the network might be confused or understand the data

[serene scaffold]

Where did you get the idea that this concept exists? It does not.

[opaque condor]

A few friends who are doing something similar

[serene scaffold]

I don't know anything about them except what you've said, but I'm skeptical that they know what they're talking about

You don't use the weights of one model as an input to another one

[opaque condor]

Well I don't exactly know since I'm learning I am basically a blank neural net when it comes to this at least

[mellow vector]

that is not the best illustration of folded laundry I've ever seen

[grand minnow]

Definitely a step in the wrong direction

[idle stone]

Any thoughts on this bro? My main goal is to learn PyTorch, contribute to the open source in a meaningful way, and probably dive into some specialized cuda level. Ideally learning to the point that I could be extremely useful

[mellow vector]

I think, if your goal is to contribute to a project that will be recognized within your target industry, there are likely many projects that could use help. I don't personally contribute to anything (yet) but I did look at torch and don't recall there being a ton of issues that were accessible to me at least. I think your mind is in the right place though.

The thing with torch is that there're likely thousands of contributors, it's already quite polished.

[mellow vector]

I'd also suggest you talk to stelericus when you get a chance, i'm an amature at best on this subject. He might be the smartest bouncer I know...

[idle stone]

awesome, yeah I definitely will! much appreciated. I'm assuming the docs are the best way to learn torch, unless there's a "better" tutorial or something?

[mellow vector]

or ask here, you can ping me if you have basic questions

[idle stone]

like for the syntax and getting started, is that the best way, the documentation?

[mellow vector]

I can't say, I have taken a few courses on Udemy personally. I guess my first thought is to prompt gpt for a network with basic meta-parameters and a weak optimizer. The optimizer is the formula the network uses when it's adjusting neurons and if you use a strong one it's hard to measure what everything is doing.

[mellow vector]

You should describe the dataset you want to train on in the prompt.

Just try to get the most basic network possible and save that as a template

[idle stone]

that's not what I'm asking my friend

[mellow vector]

If you're trying to build a car, it helps to have a car

[idle stone]

I'm asking for learning the pytorch syntax completely. I've already somewhat used it to create my own models from scratch, deploy it and optimize it without gpt

[idle stone]

no this literally makes no sense if your goal is to actually build the car lol

[mellow vector]

alright, well I can recommend a very thorough course but that's about it

[idle stone]

I'm trying to fill the gaps starting from square 1 in relearning pytorch in case I missed anything because when I used it the circumstances were extremely rushed

yeah that's all good. If anyone else knows of a meta way to learn pytorch I'm all ears

[mellow vector]

https://www.udemy.com/course/deeplearning_x/ -50 hours of detailed pytorch education

he is a very good teacher but it's also a huge time investment

[idle stone]

I'm going to absolutely check this out my friend, thank you so much

[safe agate]

https://youtu.be/vf0cn9wWBNA?si=RiLFlOk9mAx-GVk9

[lime grove]

@idle stone just find examples, and study them. You'll come across PyTorch inspiration everywhere

[idle stone]

awesome awesome, good to know

[quaint rivet]

Hi i have posted my question on forum. Looking for help https://discord.com/channels/267624335836053506/1436542399873548358

[pseudo loom]

Guys what is tokenizer in tensorflow

Just started -_-

Also what does it mean when u upload json to main

[soft ermine]

ah hallo

[pseudo loom]

Hey

[soft ermine]

wat everyone doing

[pseudo loom]

Qut u doing

[soft ermine]

i chilling bc i did my python for today and i wanna see wat everyone is up to

[pseudo loom]

Do u know tensor flow?

Heh?

[soft ermine]

uhh

depends.

[pseudo loom]

...

Also I am curious

[soft ermine]

idk the lib well cos i make my own nn wrappers

[pseudo loom]

Wow seems cool

Anyways

What was u r project about?

[soft ermine]

oh. i was tryna do contextual retention

[pseudo loom]

What does that mean?

[soft ermine]

basically like when u make a neural network and train it conventionally, it consists of weights

to remember context it refeeds the output

[pseudo loom]

Oh i see

[soft ermine]

i was tryna make it not refeed the output

[pseudo loom]

So u used acikitlearn

[soft ermine]

like uh... inbuilt memory

[pseudo loom]

Sci*

[soft ermine]

no idk wat that is

[pseudo loom]

Pytorch perhaps

Bruh what

[soft ermine]

no i told u i build my own neural net wrappers

[pseudo loom]

Woah i forgot

How do u do that

[soft ermine]

it depends wat its needed for

[pseudo loom]

Sah say training a model

[soft ermine]

typically its best to be at industrial scale, but for prototyping models i prefet to build semi scale

[pseudo loom]

Like how do u do that

[soft ermine]

u decide the network archutecture, then ur node system, then put it together and process it. u can do ur own backprop if u wanna but im trying fully forward prop

[pseudo loom]

Wow u seem experienced

[soft ermine]

u also gotta have a training pool

[pseudo loom]

Hm hm

[soft ermine]

if u dont then y u making ai

[pseudo loom]

XD

How many years r u practicing python

[soft ermine]

uh like python prolly 11 years

[pseudo loom]

This is my 1st month

o-0

[soft ermine]

1st month is good

[pseudo loom]

Yay ty

I am just a 12yr old kid ig -_-

[soft ermine]

tbh experience doesnt matter in the language when making ai. its the knowledge of how ai and neural networks operate that will guide u

also dont expect to make it first try, u gonna suck at it

[pseudo loom]

Oh i see

Ty helps a lot

[soft ermine]

np

[pseudo loom]

Kk ty

[soft ermine]

wat did u need to learn anyways

[pseudo loom]

I am learning pytorch tensorflow and acikitlearn. Currently acikitlearn

Sci*

[soft ermine]

o nice

[pseudo loom]

Those r like libs for making u r own ai model, tokenizer and training it

[soft ermine]

ye ik wat tensorflow and pytorch are. i was talking to one of the pytorch contributors the other day and roasting them

ehehe

[pseudo loom]

Lol

Welp

They r really strong libs ngl

[soft ermine]

pytorch uses cuda right

[pseudo loom]

Cuda hmm

Idk

[soft ermine]

breh

[pseudo loom]

I haven't started pytorch yet

[soft ermine]

oh ok

[pseudo loom]

Isn't cuda like ....

Hm say like kinda c++…?

[soft ermine]

no

[pseudo loom]

Idk I heard people talk about it

[soft ermine]

its like gpu api

[pseudo loom]

Some development thing ig?

Ohhh i see

[soft ermine]

but not rlly gpu api, its an abstraction

[pseudo loom]

Kk

What other languages do u know?

[soft ermine]

uh like

a lot

prolly any lang u can name, i know

[pseudo loom]

Wow