#data-science-and-ml | Python | Page 327

snow gorge Jul 16, 2021, 2:23 PM

#

i was just

desert oar Jul 16, 2021, 2:23 PM

#

how negative are we talking? like -0.0000000352 or -10?

snow gorge Jul 16, 2021, 2:23 PM

#

misreading my stuff

#

this here

desert oar Jul 16, 2021, 2:24 PM

#

we are all delusional sometimes

snow gorge Jul 16, 2021, 2:24 PM

#

all the time

#

me

ocean osprey Jul 16, 2021, 2:34 PM

#

oh great bro...the loop is working and a new dict is created for every 50k values but the pearsonr is not executed in remaining sets

desert oar Jul 16, 2021, 2:37 PM

#

oh yeah, do range(0, n_rows + step, step):

ocean osprey Jul 16, 2021, 3:16 PM

#

desert oar oh yeah, do `range(0, n_rows + step, step):`

still same result bro. the personr is not reflected

desert oar Jul 16, 2021, 3:18 PM

#

what do you mean

#

can you clarify

#

also this should work with just range(0, n_rows, step)...

ocean osprey Jul 16, 2021, 3:19 PM

#

the values are the same. the pearsonar is not initiated for the second and other sets

desert oar Jul 16, 2021, 3:19 PM

#

show your code

#

oh, i know why

#

you copied my code verbatim without reading or understanding it 🙂

#

i made a mistake in my code

#

do you see the problem here?

for start in range(0, n_rows, step):
    end = start + step
    results[(start, end)] = pearsonr(x_np, y_np)

#

it's a good exercise to figure out the problem and try to fix it

#

i'm not sure i want to encourage people asking strangers for help and expecting that it will be 100% right the first time

#

it's like driving, if you rely too much on the gps you never learn how to actually get anywhere

ocean osprey Jul 16, 2021, 3:22 PM

#

its true. i will try bro...thanks for the help 😅

wild birch Jul 16, 2021, 3:36 PM

#

Goodmorning. I was wondering if any of you had any helpful advice or a source to help me learn how to make a dataset. Im trying to make an ai with tensorflow and numpy. I got it to work(very very basic) with numbers. Only thing is im trying to get it to save its weights and mainly work with words.

#

But words didnt work with an array, and i read that a dataset would be the best way to go, but i know nothing of how to do so.

hollow vault Jul 16, 2021, 4:18 PM

#

wild birch Goodmorning. I was wondering if any of you had any helpful advice or a source to...

hi, i am not sure will this help but you can scan this link https://towardsdatascience.com/how-to-use-dataset-in-tensorflow-c758ef9e4428

Medium

How to use Dataset in TensorFlow

The built-in Input Pipeline. Never use ‘feed-dict’ anymore

grave frost Jul 16, 2021, 4:24 PM

#

wild birch Goodmorning. I was wondering if any of you had any helpful advice or a source to...

that's NLP - look up basics like one hot encoding and n-grams then move up the chain to RNNs/LSTMs

somber prism Jul 16, 2021, 4:35 PM

#

guys i have a question , if i have labels like this in a column 'N', 'VAR', 'W', 'SSW', 'SSE', 'NNW', 'NE', 'E', 'West', 'S', 'Variable', 'WSW', 'SW', 'ESE', 'South', 'ENE', 'Calm', 'NNE', 'CALM', 'NW', 'East', 'North', 'WNW', 'SE' how i do i simplify them and also explain me whats the reason for going for that method.

wild birch Jul 16, 2021, 4:43 PM

#

grave frost that's NLP - look up basics like one hot encoding and n-grams then move up the c...

whats nlp and such?

desert oar Jul 16, 2021, 4:46 PM

#

somber prism guys i have a question , if i have labels like this in a column ```'N', 'VAR', '...

is this a homework assignment?

somber prism Jul 16, 2021, 4:47 PM

#

also i just want the reason to do that so it can be helpful in future

#

nope

#

i am just practicing with some dataset

#

this is the dataset - https://www.kaggle.com/sobhanmoosavi/us-accidents

US Accidents (3 million records -- updated)

A Countrywide Traffic Accident Dataset (2016 - 2020)

#

@desert oar

grave frost Jul 16, 2021, 4:52 PM

#

wild birch whats nlp and such?

natural language processing

cedar sun Jul 16, 2021, 7:59 PM

#

do u know about any pretrained model that extracts the main object from an image? Like returning the mask or something

reef bone Jul 16, 2021, 8:02 PM

#

not sure about the 'main' object

#

but the YOLO models do object detection and can easily be trained on custom data

#

YOLOv5 is the latest believe, its open source

#

however it draws a bounding box and will not decide which is the 'main' object if it recognizes multiple in an image, so this may be irrelevant

cedar sun Jul 16, 2021, 8:43 PM

#

reef bone however it draws a bounding box and will not decide which is the 'main' object i...

if there is only 1 object it will extract only one, right?

#

so that would be the main xd

#

but thanks

reef bone Jul 16, 2021, 8:43 PM

#

as long as its gets recognized, yes

#

you get a bounding box

cedar sun Jul 16, 2021, 8:55 PM

#

mmm i dont want a boundign box

#

i want the mask of the object

reef bone Jul 16, 2021, 9:15 PM

#

sorry then yolo is not appropriate

#

googling 'object detection mask' yields some promising results though

cedar sun Jul 16, 2021, 9:22 PM

#

but ive seen yolo masks the object too

#

:/

#

https://gyazo.com/87045f758f89143c5c6ad23abf91841f

Gyazo

#

are these models?

velvet thorn Jul 16, 2021, 11:01 PM

#

@cedar sun the term for what you want

#

is image segmentation

cedar sun Jul 16, 2021, 11:01 PM

#

isnt it SOD?

#

salient object detection?

velvet thorn Jul 16, 2021, 11:02 PM

#

predicting pixel level masks instead of bounding boxes

#

hold up

#

supervised or unsupervised?

cedar sun Jul 16, 2021, 11:02 PM

#

like, image segmentation gives me all the objects it can detect on an image

#

supervised

velvet thorn Jul 16, 2021, 11:02 PM

#

okay wait

#

if you’re talking about the aspect of

#

pixel level mask

#

vs bounding box

#

then no

#

but if you’re talking about the aspect of “main” object vs “secondary” objects

#

then yes

cedar sun Jul 16, 2021, 11:03 PM

#

is this img segmentation? https://gyazo.com/26d2852563172f9b72cfeb616f3d5007

Gyazo

velvet thorn Jul 16, 2021, 11:03 PM

#

yes

#

instance segmentation, specifically

cedar sun Jul 16, 2021, 11:05 PM

#

yeah, well, i only want like the main object

cedar sun Jul 16, 2021, 11:05 PM

#

cedar sun https://gyazo.com/87045f758f89143c5c6ad23abf91841f

like here

velvet thorn Jul 16, 2021, 11:06 PM

#

?? then why were you considering YOLO

cedar sun Jul 16, 2021, 11:12 PM

#

someone suggested lel

cedar sun Jul 16, 2021, 11:12 PM

#

reef bone but the YOLO models do object detection and can easily be trained on custom data

here

velvet thorn Jul 16, 2021, 11:13 PM

#

if you want to identify an arbitrary “main object”

#

then that is saliency detection, yes

velvet thorn Jul 16, 2021, 11:14 PM

#

cedar sun if there is only 1 object it will extract only one, right?

but you said this

#

which is kind of confusing

cedar sun Jul 16, 2021, 11:22 PM

#

huh

#

well i meant like... idk, it is called SOD

velvet thorn Jul 16, 2021, 11:38 PM

#

cedar sun well i meant like... idk, it is called SOD

if you want to identify an ARBITRARY main object, then yes

cedar sun Jul 17, 2021, 12:00 AM

#

thanks, and do u know any pretrained model loadable with keras?

visual violet Jul 17, 2021, 1:50 AM

#

hello people

#

can somebody pleae tell me a topic in bioinformatics that has something to do with Econ

rugged crypt Jul 17, 2021, 3:11 AM

#

I am going to buy a new laptop and was asking for suggestions for a processor. Almost all of them suggested me to get one with more cores. However, my use case is mostly single threaded. I will be using sklearn, Jupyter and VS code. In that case, should I be aiming for more cores or more clock speed?

serene scaffold Jul 17, 2021, 4:12 AM

#

@rugged crypt it sounds like you've answered your own question, though you might consider that some sklearn models can use more than one core.

white venture Jul 17, 2021, 4:20 AM

#

CUDA 11.4 should work fine with TensorFlow 2.5.0, right?

blazing bridge Jul 17, 2021, 5:44 AM

#

rugged crypt I am going to buy a new laptop and was asking for suggestions for a processor. A...

imo both are pretty important but if I were to choose one then it would be more cores. The more cores the better generally. Im not an expert, so someone else can confirm this or tell me that I'm wrong.

soft sorrel Jul 17, 2021, 6:50 AM

#

ValueError: Input 0 of layer lstm is incompatible with the layer: expected ndim=3, found ndim=4. Full shape received: (None, 32, 24, 7)

#

hello, until now i cant resolve this.

brisk lintel Jul 17, 2021, 7:28 AM

#

Can someone explain what does valueerror: lengths must match to compare means?

serene scaffold Jul 17, 2021, 8:03 AM

#

@brisk lintel you'll need to share the whole error message and probably the relevant code

brisk lintel Jul 17, 2021, 8:05 AM

#

import altair as alt
import streamlit as st
import pandas as pd, numpy as np


df2= pd.read_excel('owidvac.xlsx')
subset_data=df2
country_name_input = st.multiselect(
'Country name',
df2.groupby('location').count().reset_index()['location'].tolist())

st.subheader('Comparision of infection growth')
total_cases_graph = alt.Chart(subset_data["location"]==country_name_input).mark_line().encode(
    x=alt.X('date', type='nominal', title='Date'),
    y=alt.Y('new_cases',  title='New Cases'),
).properties(
    width=1500,
    height=600
).configure_axis(
    labelFontSize=17,
    titleFontSize=20
)

st.altair_chart(total_cases_graph)

#

ValueError: ('Lengths must match to compare', (34004,), (0,))
Traceback:
File "C:\Users\m\anaconda3\lib\site-packages\streamlit\script_runner.py", line 349, in _run_script
exec(code, module.dict)
File "C:\Users\m\Downloads\covid-choro-map-master\testing\test.py", line 15, in <module>
total_cases_graph = alt.Chart(subset_data["location"]==country_name_input).mark_line().encode(
File "C:\Users\m\anaconda3\lib\site-packages\pandas\core\ops\common.py", line 65, in new_method
return method(self, other)
File "C:\Users\m\anaconda3\lib\site-packages\pandas\core\arraylike.py", line 29, in eq
return self._cmp_method(other, operator.eq)
File "C:\Users\m\anaconda3\lib\site-packages\pandas\core\series.py", line 4978, in _cmp_method
res_values = ops.comparison_op(lvalues, rvalues, op)
File "C:\Users\m\anaconda3\lib\site-packages\pandas\core\ops\array_ops.py", line 223, in comparison_op
raise ValueError(

timber tartan Jul 17, 2021, 8:38 AM

#

hello guys i have a weird problem regarding some data, i have a batch of satellite measurements for sea level rise from 1992 to 2020 and when i plot a line graph there is a big chuck of data missing between 2010 to 2013. What i have noticed from that period of time is that the satellites Jason-1 and 2 are heavily alternating between each other in the data set.
here's the graph

#

#

the code i used
`import matplotlib.pyplot as plt
import pandas as pd
import numpy as np

csv_file = "slr_sla_pac_free_all_66.csv"
read_file = pd.read_csv(csv_file ,comment='#')

display(read_file)

plt.title('Sea Level rise')

plt.plot(read_file.year,read_file['TOPEX/Poseidon'])
plt.plot(read_file.year,read_file['Jason-1'])
plt.plot(read_file.year,read_file['Jason-2'])
plt.plot(read_file.year,read_file['Jason-3'])`

#

also you can find the data file here https://www.star.nesdis.noaa.gov/socd/lsa/SeaLevelRise/LSA_SLR_timeseries_regional.php

NOAA / NESDIS / STAR website

Center for Satellite Applications and Research - NOAA / NESDIS / STAR

STAR translates raw satellite data into critical information needed to inform the public and safeguard the environment across the country and around the world.

#

the 4th csv file from the Pacific Ocean

#

short heart Jul 17, 2021, 9:13 AM

#

Would it matter that much for training process if, say out of 6000 values id have a few duplicates

fair nimbus Jul 17, 2021, 11:03 AM

#

timber tartan also you can find the data file here https://www.star.nesdis.noaa.gov/socd/lsa/S...

I don't know why though it seems fair to avg them and plot that instead.

read_file['avg'] = read_file[['TOPEX/Poseidon', 'Jason-1', 'Jason-2', 'Jason-3']].mean(axis=1)
plt = read_file.plot(x='year', y='avg') #, color="DarkGreen")
fig = plt.get_figure()
fig.savefig('slr_sla_pac_free_all_66.png')

I was looking into seeing if what you had was a color / styling issue though I couldn't get it to work like that.

https://matplotlib.org/stable/users/dflt_style_changes.html

timber tartan Jul 17, 2021, 11:16 AM

#

i think il just email the profesor and tell him about the issue

midnight stag Jul 17, 2021, 11:31 AM

#

What is this error how to fix it

subtle grove Jul 17, 2021, 11:49 AM

#

Yooo I tried AI and ML and FAILED lolololol

soft sorrel Jul 17, 2021, 1:01 PM

#

i just failed my tensorflow certification exam. could someone enlighten me on how to solve the question below?

#

I been thinking about it whole day

#

https://paste.pythondiscord.com/xupahekizi.py

lapis sequoia Jul 17, 2021, 1:31 PM

#

I'm developing an AI based model which will help in preventing food wastage. I need collect data for that. Please fill this form accurately on individual basis.
https://docs.google.com/forms/d/e/1FAIpQLSfqcF1icI7SJYbz1IJy74wWUzM-WVVYh4MZxNIUP6OQ0h6wXg/viewform?usp=sf_link
Share it with others as much as possible.
Please React with a 👍 if you fill it!
It will be a great help.Thanks!

Google Docs

HUNGER CRISIS-2021

In hotels and restaurants and eateries, we witness a lot of food being wasted on a daily basis which includes the food we leave on the plates and also the extra food prepared. This food is of no use as it cannot be given away to the needy. It is moral responsibility to realise that we need to reduce the wastage of food. In order to create an AI ...

lapis sequoia Jul 17, 2021, 2:14 PM

#

I need to get data from xml which is in website

grave breach Jul 17, 2021, 2:45 PM

#

lapis sequoia I need to get data from xml which is in website

First use requests library and then xml library

#

Take a look at this:

#

https://stackoverflow.com/questions/18308529/python-requests-package-handling-xml-response

Stack Overflow

Python Requests package: Handling xml response

I like very much the requests package and its comfortable way to handle JSON responses.

Unfortunately, I did not understand if I can also process XML responses. Has anybody experience how to hand...

grave breach Jul 17, 2021, 2:49 PM

#

cedar sun thanks, and do u know any pretrained model loadable with keras?

Depends on the labels you want

cedar sun Jul 17, 2021, 3:04 PM

#

i think we already had this convs

#

u dont need labbels

grave breach Jul 17, 2021, 3:04 PM

#

You were talking with @velvet thorn

#

You talked about arbitary things

cedar sun Jul 17, 2021, 3:05 PM

#

i know but u already said the same thing about labels, and u dont need labels

grave breach Jul 17, 2021, 3:06 PM

#

For instance segmentation you need

cedar sun Jul 17, 2021, 3:06 PM

#

and if u read the conv, u will see i dont need img segmentation

grave breach Jul 17, 2021, 3:06 PM

#

velvet thorn if you want to identify an ARBITRARY main object, then yes

.

cyan drift Jul 17, 2021, 3:31 PM

#

I wanna start datascience and ai

grave breach Jul 17, 2021, 3:35 PM

#

@cyan drift "How much" math do you know?

cyan drift Jul 17, 2021, 3:43 PM

#

grave breach <@!700362473781330000> "How much" math do you know?

Class 10th

grave breach Jul 17, 2021, 3:49 PM

#

cyan drift Class 10th

Europe, USA, Russia?

#

By the way

#

I suggest you first to start studying calculus

#

Then study statistics so that you can pass reading the Hands-On Machine learning book

#

And if you want to get more experienced about neural networks, I suggest you the Deep Learning from scratch book

#

Then, if you want to know a bit more about deep learning in general, I suggest you to read this (free)

#

https://www.deeplearningbook.org/

#

It even contains some math basics for deep learning

#

@cyan drift

#

This is if you want to master the concepts, if you just need to learning AI but you don't have time, or you just need it for a practical usage

#

Check out https://www.fast.ai/

Home

Making neural nets uncool again

#

Even this contains math basics

cedar sun Jul 17, 2021, 4:21 PM

#

https://gyazo.com/f36063cc658e6c7314849dc7e15d60c2

Gyazo

#

what is this web

visual violet Jul 17, 2021, 4:23 PM

#

is this the right place to ask about machine learning?

cedar sun Jul 17, 2021, 4:29 PM

#

can someone explain why this happens on first epoch?

#

551/551 [==============================] - 420s 690ms/step - loss: 3.1702 - accuracy: 0.2863 - val_loss: 1.2984 - val_accuracy: 0.6414

ripe forge Jul 17, 2021, 4:31 PM

#

its just printing information about the first epoch.. what about it?

ripe forge Jul 17, 2021, 4:31 PM

#

visual violet is this the right place to ask about machine learning?

yes

visual violet Jul 17, 2021, 4:31 PM

#

how do i start

#

where do i learn

ripe forge Jul 17, 2021, 4:31 PM

#

cedar sun https://gyazo.com/f36063cc658e6c7314849dc7e15d60c2

that's the playground. https://playground.tensorflow.org

#

check pins for some links on where to learn

visual violet Jul 17, 2021, 4:35 PM

#

LOL

#

the pinned is all about machine learning

#

0 about stats

#

0 about data science

cedar sun Jul 17, 2021, 4:35 PM

#

ripe forge its just printing information about the first epoch.. what about it?

acc and val acc

cedar sun Jul 17, 2021, 4:36 PM

#

ripe forge that's the playground. <https://playground.tensorflow.org>

ty

ripe forge Jul 17, 2021, 4:36 PM

#

cedar sun acc and val acc

at the end of each epoch, you get the information about both the train set and the test set. that's accuracy and accuracy on test set

cedar sun Jul 17, 2021, 4:36 PM

#

...

#

why i have 0.3 acc and 0.65 val acc

ripe forge Jul 17, 2021, 4:37 PM

#

well, why not?

cedar sun Jul 17, 2021, 4:37 PM

#

cuz val acc should be lower than acc

#

not twice it

ripe forge Jul 17, 2021, 4:37 PM

#

you can have a val acc higher than train accuracy

#

depends entirely on the points in the validation data. for example, if your train test splits are not that great, then the validation data can be overrepresenting a certain type of data points

#

those could be learnt by the model during the first pass

cedar sun Jul 17, 2021, 4:38 PM

#

the split is a shuffle

ripe forge Jul 17, 2021, 4:38 PM

#

sure, but that doesn't mean that it can't happen.

#

if you want to get a better sense of what's doing on, don't try to analyse on just one epoch. what happens as you train this for more epochs, what are the trends like, etc

cedar sun Jul 17, 2021, 4:41 PM

#

okey

#

https://gyazo.com/a4fce12050c31178386a8f991502cbda

Gyazo

cyan drift Jul 17, 2021, 4:45 PM

#

grave breach Europe, USA, Russia?

India

cyan drift Jul 17, 2021, 4:45 PM

#

grave breach Then study statistics so that you can pass reading the Hands-On Machine learning...

Ik statistics :D

#

Oh i see thank you headcrab: D

cedar sun Jul 17, 2021, 4:47 PM

#

also... how to determine the correct batch size_

#

?

ripe forge Jul 17, 2021, 4:50 PM

#

trial and error really.

#

out of curiousity, what's your number of data points, and what's your split ratio

cedar sun Jul 17, 2021, 4:50 PM

#

0.2

#

and...

#

Found 4322 images belonging to 151 classes```

cedar sun Jul 17, 2021, 5:07 PM

#

is the tuner an option?

grave breach Jul 17, 2021, 5:20 PM

#

cyan drift Ik statistics :D

Awesome

#

By the way, what I linked you was just sort of a "big" path to follow

#

It's just to link you all resources

#

I don't think you need to read any single one of the book

late shell Jul 17, 2021, 5:33 PM

#

I was watching this video on youtube by andrew ng, basics of neural network. The equation he wrote for calculating z1 is z1 = W1 * X + b1, but given the neural network (the one that's at the bottom), W1 will be a vector of shape (1,9) and X.shape would be (3,m) where m is the number of training samples, right? I don't get how these 2 matrices, W1 and X are supposed to be dotted? Their shapes won't allow matrix multiplication.

tidal bough Jul 17, 2021, 5:49 PM

#

late shell I was watching this video on youtube by andrew ng, basics of neural network. The...

W1 should be a matrix of weights - so of shape (layer_1_size,input_size), so that it can transform (input_size,m) into (layer_1_size,m)

#

actually the formula is z = w^T x + b by the picture, so W's shape is (input_size,layer_1_size) then.

late shell Jul 17, 2021, 5:57 PM

#

oh, so w^[1].shape must be (3,3), Is this correct?

indigo stone Jul 17, 2021, 5:58 PM

#

I think so

late shell Jul 17, 2021, 6:31 PM

#

cool, thanks a ton @tidal bough & @indigo stone

quasi sparrow Jul 17, 2021, 6:39 PM

#

Good morning guys!

#

I want to plot different datasets with a drop down menu. I am using pandas and plyplot for graphics and data

#

Can anyone point me in the right direction?

keen prism Jul 17, 2021, 7:35 PM

#

is it bad this is slow?

#

it's like. 18 seconds per MB

quasi sparrow Jul 17, 2021, 9:19 PM

#

Thanks! 🙂

visual violet Jul 17, 2021, 10:04 PM

#

"Machine Learning is mathematics first, and programming second. "

#

oh shi

#

how can a not very good looking website like this https://home.work.caltech.edu/telecourse.html contains a lot of useful info

quasi sparrow Jul 17, 2021, 10:15 PM

#

Ohh good stuff

#

Thank you for sharing

cedar sun Jul 17, 2021, 10:53 PM

#

guys

#

are this augmentations good? or too aggresive?

#

augs = [A.Compose([
            A.Transpose(),
            A.VerticalFlip(),
            A.HorizontalFlip(),
            A.RandomBrightness(limit=0.2),
            A.RandomContrast(limit=0.2, p=0.75),
            A.OneOf([
                A.Blur(),
                A.MedianBlur(),
                A.GaussianBlur(blur_limit=5),
                A.GaussNoise(var_limit=(5.0, 10.0)),
            ]),

            A.OneOf([
                A.OpticalDistortion(distort_limit=1.0),
                A.GridDistortion(num_steps=5, distort_limit=1.0),
                A.ElasticTransform(alpha=3),
            ]),

            A.ISONoise(color_shift=(0.05, 0.1), intensity=(0.1, 0.4)),
            # A.HueSaturationValue(hue_shift_limit=10, sat_shift_limit=20, val_shift_limit=10),
            A.ShiftScaleRotate(shift_limit=0.1, scale_limit=0.1, rotate_limit=20, p=0.85)
    ])]```

#

trying to classify pokemons

quasi sparrow Jul 17, 2021, 11:06 PM

#

you can use a pretrained network

#

I don't think there are enough images of pokemons on the internet to train a classifier from scratch

#

Use transfer learning

#

https://www.kaggle.com/kwisatzhaderach/neural-networks-with-pokemon

Neural Networks with Pokemon!

Explore and run machine learning code with Kaggle Notebooks | Using data from multiple data sources

#

I don't think pokemos have consistent features to train a model.
Some of them look like lizards and or some others look like turtles, feature extraction will be hard.

#

But i might wrong! I dont know much about DL yet

cedar sun Jul 17, 2021, 11:12 PM

#

20k img for almost 1000 classes

#

i think that model sucks

#

yeah, it sucks, i already downloaded it

#

i think what i dont know is a good architecture

#

im using eff net

#

but i think all prebuilt models are for realistc imgs

quasi sparrow Jul 17, 2021, 11:15 PM

#

Hahaha I have not done image classification in a while

#

Any specific reason why you want to do a pokemon classifier?

cedar sun Jul 17, 2021, 11:16 PM

#

not rlly

#

but i achieved 0.89 for gen 1 pokemons. I just... need to clean the rest generatiosn q.q

hard canopy Jul 17, 2021, 11:23 PM

#

Have you tried using CLIP ? I' m unsure if it could classify more rare pokemons though

cedar sun Jul 17, 2021, 11:28 PM

#

clip?

hard canopy Jul 17, 2021, 11:29 PM

#

https://openai.com/blog/clip/

OpenAI

CLIP: Connecting Text and Images

We’re introducing a neural network called CLIP which efficiently learns visual concepts from natural language supervision.

#

It can classify lot of stuff out of the box

cedar sun Jul 17, 2021, 11:30 PM

#

how can i use this?

hard canopy Jul 17, 2021, 11:31 PM

#

https://github.com/openai/CLIP

GitHub

openai/CLIP

Contrastive Language-Image Pretraining. Contribute to openai/CLIP development by creating an account on GitHub.

#

import torch
import clip
from PIL import Image

device = "cuda" if torch.cuda.is_available() else "cpu"
model, preprocess = clip.load("ViT-B/32", device=device)

image = preprocess(Image.open("CLIP.png")).unsqueeze(0).to(device)
text = clip.tokenize(["a diagram", "a dog", "a cat"]).to(device)

with torch.no_grad():
    image_features = model.encode_image(image)
    text_features = model.encode_text(text)
    
    logits_per_image, logits_per_text = model(image, text)
    probs = logits_per_image.softmax(dim=-1).cpu().numpy()

print("Label probs:", probs)  # prints: [[0.9927937  0.00421068 0.00299572]]

cedar sun Jul 17, 2021, 11:32 PM

#

no no i mean

#

how would this be usefull for my case?

hard canopy Jul 17, 2021, 11:32 PM

#

well I thought you were trying to classify pokemons images ?

cedar sun Jul 17, 2021, 11:33 PM

#

yeah i am

hard canopy Jul 17, 2021, 11:33 PM

#

You can use a pretrained CLIP for this purpose

#

Or at least you can try

cedar sun Jul 17, 2021, 11:33 PM

#

but, the img im trying to predict

#

wont have any text

hard canopy Jul 17, 2021, 11:33 PM

#

pretty sure it knows get 1 pokemon

#

yeah so ?

cedar sun Jul 17, 2021, 11:34 PM

#

CLIP (Contrastive Language-Image Pre-Training) is a neural network trained on a variety of (image, text) pairs.

hard canopy Jul 17, 2021, 11:34 PM

#

it would be a pretty bad classifier if it needed the class written in the image

cedar sun Jul 17, 2021, 11:34 PM

#

what would be the text then?

#

the name of the pokemon? thats what im trying to predict

hard canopy Jul 17, 2021, 11:34 PM

#

"A picture of a Pikachu" or juste the name yeah

cedar sun Jul 17, 2021, 11:34 PM

#

that makes no sense lel

#

im trying to predict the name of the pokemon

hard canopy Jul 17, 2021, 11:35 PM

#

yeah

#

then you make an array with the name of all pokemon

#

and ask clip which one it is

cedar sun Jul 17, 2021, 11:35 PM

#

ah for training u mean?

hard canopy Jul 17, 2021, 11:35 PM

#

There is no training

cedar sun Jul 17, 2021, 11:35 PM

#

:)

#

idk how it works then

hard canopy Jul 17, 2021, 11:36 PM

#

import clip
from PIL import Image

device = "cuda" if torch.cuda.is_available() else "cpu"
model, preprocess = clip.load("ViT-B/32", device=device)

image = preprocess(Image.open("a_pokemon.png")).unsqueeze(0).to(device)
text = clip.tokenize(["a Pikachu", "a Squirtle", "a mew"]).to(device)

with torch.no_grad():
    image_features = model.encode_image(image)
    text_features = model.encode_text(text)
    
    logits_per_image, logits_per_text = model(image, text)
    probs = logits_per_image.softmax(dim=-1).cpu().numpy()

print("Label probs:", probs)  # prints: [[0.9927937  0.00421068 0.00299572]]

#

like this, it will give you a probability for each pokemon

#

I put just 3 differents pokemon for the example though

cedar sun Jul 17, 2021, 11:37 PM

#

and how does it know what pokemon is a_pokemon.png?

hard canopy Jul 17, 2021, 11:38 PM

#

I am not sure about the network architecture, or how it works internally

#

I can just say that it does work very well for many tasks

cedar sun Jul 17, 2021, 11:39 PM

#

u arent understanding

#

on ur snipper, why would it print those numbers?

#

how does the model, or whatever, knows what img is

#

if it hasnt been trained

hard canopy Jul 17, 2021, 11:40 PM

#

it has already been trained

cedar sun Jul 17, 2021, 11:40 PM

#

xD

hard canopy Jul 17, 2021, 11:40 PM

#

You are not expected to train it yourself

cedar sun Jul 17, 2021, 11:40 PM

#

so it has been trained with dogs and cats

#

so how will it recognize a pikachu

hard canopy Jul 17, 2021, 11:41 PM

#

There are likely pikachus in the training dataset

cedar sun Jul 17, 2021, 11:41 PM

#

okey. Which model are u loading?

#

cuz i doubt it will have seen all the pokemons

hard canopy Jul 17, 2021, 11:42 PM

#

ViT-B/32

cedar sun Jul 17, 2021, 11:42 PM

#

can i see somehow the classes?

#

or something?

hard canopy Jul 17, 2021, 11:42 PM

#

You can probably find the vocabulary of the text encoder yeah

cedar sun Jul 17, 2021, 11:43 PM

#

it has been trained with imagenet

hard canopy Jul 17, 2021, 11:43 PM

#

I think it' s like ~72K words

cedar sun Jul 17, 2021, 11:43 PM

#

and imagenet doesnt contain pokemons

#

or well, i see scores based on imagenet

hard canopy Jul 17, 2021, 11:44 PM

#

No it has not been trained on imagenet

#

http://projects.dfki.uni-kl.de/yfcc100m/

#

I think it has been trained on this

#

and there are pikachus inside

forest briar Jul 17, 2021, 11:44 PM

#

is there a better way of doing this to reduce cpu-gpu overhead

#

def predict(self, board):
        board = torch.FloatTensor(board.astype(np.float64))
        if args.cuda:
            board = board.contiguous().cuda()
        with torch.no_grad():
            board = board.view(1, self.board_x, self.board_y)

            self.nnet.eval()
            pi, v = self.nnet(board)

            return torch.exp(pi).data.cpu().numpy()[0], v.data.cpu().numpy()[0]

#

if i'm calling this on parallelized mcts agents

#

for every unknown state

cedar sun Jul 17, 2021, 11:45 PM

#

hard canopy http://projects.dfki.uni-kl.de/yfcc100m/

okey, but other pokemons arent

forest briar Jul 17, 2021, 11:45 PM

#

only other thing i read is that if i'm instantiating the tensor, i can just create it on cuda directly like torch.FloatTensor(board, device=torch.device('cuda'))

cedar sun Jul 17, 2021, 11:46 PM

#

yeah, only the most common pokemons

#

not all

lapis sequoia Jul 18, 2021, 2:16 AM

#

matplotlib python file that I did in atom. im new to coding in text editors. so i dont know why the matplotlib window isnt showing up when i run this cmd. fixes...

vagrant grove Jul 18, 2021, 2:22 AM

#

Did you forget to put

#

plt.show()

desert oar Jul 18, 2021, 2:31 AM

#

!paste @lapis sequoia show your code, see below 👇

arctic wedgeBOT Jul 18, 2021, 2:31 AM

#

Pasting large amounts of code

If your code is too long to fit in a codeblock in discord, you can paste your code here:
https://paste.pydis.com/

After pasting your code, save it by clicking the floppy disk icon in the top right, or by typing ctrl + S. After doing that, the URL should change. Copy the URL and post it here so others can see it.

lapis sequoia Jul 18, 2021, 2:32 AM

#

https://paste.pythondiscord.com/avegumuqen.py

desert oar Jul 18, 2021, 2:34 AM

#

@lapis sequoia put

plt.show()

at the end of the file. otherwise python doesn't know that you want to actually show the plot. jupyter notebook/lab helpfully shows you the plot anyway, under certain circumstances. but that is only in jupyter notebook or jupyterlab.

#

also, get in the habit of showing your code, data, etc. it's easier to help someone if the helper can see exactly what you are doing

lapis sequoia Jul 18, 2021, 2:36 AM

#

Okay noted. gonna try this now

#

It worked thanks

cyan drift Jul 18, 2021, 4:27 AM

#

grave breach By the way, what I linked you was just sort of a "big" path to follow

Oh ok

late shell Jul 18, 2021, 6:16 AM

#

Hello, I just studied about logistic regression through the probability approach i.e maximizing the Log Likelihood function using MLE. Now I'm studying the same thing, logistic regression but through a Neural Network. Andre Ng explains it really well but I don't get a few things. The traditional way i.e using MLE to calculate logistic regression seemed intuitive i.e the sigmoid function applied to z = w1x1 + w2x2 + b, (if only 2 features present) and I could understand what w1,w2, and b implied, and how they're related to the log odds. But its really hard to wrap my head around how a neural network learns this relationship because traditionally, the logistic regression model has 3 parameters (considering 2 features), right? but even a simple NN will have around 9 weights and 4 biases or so. How am I to interpret these 9 weights and 4 biases? This is really confusing, I can understand small pieces of the puzzle but I'm not able to put together the whole picture and see how each piece fits in the puzzle.

grave breach Jul 18, 2021, 11:00 AM

#

hard canopy "A picture of a Pikachu" or juste the name yeah

CLIP wouldn't work with that kind of things, and training a clip for this task is nonsense

#

CLIP works by correlating the meaning of a text to an image

#

Forcing the CNN to extract the meaning instead of a simple prediction

#

A normal CNN would work better

desert oar Jul 18, 2021, 11:56 AM

#

late shell Hello, I just studied about logistic regression through the probability approach...

Why would you need so many parameters? That neural network with no hidden layer literally has the same equation as traditional logistic regression

late shell Jul 18, 2021, 12:26 PM

#

#

9 weights, 4 biases......

desert oar Jul 18, 2021, 12:33 PM

#

That model isn't equivalent to logistic regression

#

It has an extra layer

#

You can use it in lieu of logistic regression, but it's not the same thing. It's certainly not an exponential family GLM anymore

#

But the general question of "how to interpret these weights" is a good one

#

I think the general answer is somewhat unsatisfying, they don't have a nice interpretation as log odds ratios or anything like that

#

You can think of the outputs from the hidden layer as a representation of the data that is learned by the model, and will be optimized for the classification problem

stuck mortar Jul 18, 2021, 1:44 PM

#

guys wat is the best lap for coding
mac or win
which one will ya suggest

lilac geyser Jul 18, 2021, 2:00 PM

#

Can this model be considered as a good model?
I'm asking this because. This is my first model that I have built with help of sklearn.

I have transformed the X, y data

serene scaffold Jul 18, 2021, 2:01 PM

#

lilac geyser Can this model be considered as a good model? I'm asking this because. This is m...

what is the model predicting for?

lilac geyser Jul 18, 2021, 2:01 PM

#

It is a regression based problem

#

I have took a kaggle dataset of Car Price
And applied RandomForestRegressor

serene scaffold Jul 18, 2021, 2:02 PM

#

link to the data?

lilac geyser Jul 18, 2021, 2:02 PM

#

This is my first time of building the ML model

lilac geyser Jul 18, 2021, 2:03 PM

#

serene scaffold link to the data?

https://www.kaggle.com/nehalbirla/vehicle-dataset-from-cardekho

Vehicle dataset

Used Cars data form websites

#

I have took
CAR DETAILS FROM CAR DEKHO.csv
As dataset

#

@serene scaffold
If you figure out something.
Please @ me

hoary wigeon Jul 18, 2021, 2:23 PM

#

i need help, i want to convert sentence into word matrix.

#

: (

serene scaffold Jul 18, 2021, 2:24 PM

#

hoary wigeon i need help, i want to convert sentence into word matrix.

can you elaborate?

hoary wigeon Jul 18, 2021, 2:24 PM

#

#

i have this

#

i want to separate each words in columns with 1s and 0s

serene scaffold Jul 18, 2021, 2:25 PM

#

what do the 1s and 0s mean?

hoary wigeon Jul 18, 2021, 2:26 PM

#

initaially | trouble | and | like 
1          | 1       | 1   | 0   
0          | 0       | 1   | 1
0          | 0       | 0   | 1

#

something like this

serene scaffold Jul 18, 2021, 2:27 PM

#

!docs sklearn.preprocessing.MultiLabelBinarizer

arctic wedgeBOT Jul 18, 2021, 2:27 PM

#

sklearn.preprocessing.MultiLabelBinarizer


class sklearn.preprocessing.MultiLabelBinarizer(*, classes=None, sparse_output=False)```
Transform between iterable of iterables and a multilabel format.

Although a list of sets or tuples is a very intuitive format for multilabel data, it is unwieldy to process. This transformer converts between this intuitive format and the supported multilabel format: a (samples x classes) binary matrix indicating the presence of a class label.

hoary wigeon Jul 18, 2021, 2:27 PM

#

Multilabel ?

serene scaffold Jul 18, 2021, 2:27 PM

#

@hoary wigeon keep in mind that you'd end up with as many columns as the number of words in your vocabulary, and that will probably become very sparse

hoary wigeon Jul 18, 2021, 2:28 PM

#

there are 15754 different word

#

so it will generate 15754 columns ?

serene scaffold Jul 18, 2021, 2:29 PM

#

hoary wigeon so it will generate `15754` columns ?

yes. so you should probably decide how many words you want to be of interest

#

like, the top 100, or something

hoary wigeon Jul 18, 2021, 2:30 PM

#

cool

#

lemme try once

serene scaffold Jul 18, 2021, 2:31 PM

#

hoary wigeon

you can tokenize the reviews.text column and then make a list of all the tokens (including duplicates). then you can count them

hoary wigeon Jul 18, 2021, 2:33 PM

#

i did not understand this

serene scaffold Jul 18, 2021, 2:34 PM

#

hoary wigeon i did not understand this

do you know what tokenizing is?

hoary wigeon Jul 18, 2021, 2:34 PM

#

nope

serene scaffold Jul 18, 2021, 2:34 PM

#

hoary wigeon nope

it's where you break the text down into discrete units, which are basically words

hoary wigeon Jul 18, 2021, 2:35 PM

#

#

why only 62 cols ?

serene scaffold Jul 18, 2021, 2:35 PM

#

hoary wigeon why only 62 cols ?

I don't think you tokenized the text, so it did something other than what you expected.

hoary wigeon Jul 18, 2021, 2:36 PM

#

can u pls write a code or pseudocode ?

serene scaffold Jul 18, 2021, 2:36 PM

#

hoary wigeon can u pls write a code or pseudocode ?

do you know about the string manipulation methods for dataframes?

hoary wigeon Jul 18, 2021, 2:37 PM

#

yes

serene scaffold Jul 18, 2021, 2:37 PM

#

hoary wigeon yes

so you need to split up all the sentences into individual words

hoary wigeon Jul 18, 2021, 2:38 PM

#

mmm..

serene scaffold Jul 18, 2021, 2:39 PM

#

In [21]: s
Out[21]: 
0    Hello world!
1     I like cake
2     Pie is good
dtype: object

In [22]: s.str.split()
Out[22]: 
0    [Hello, world!]
1    [I, like, cake]
2    [Pie, is, good]
dtype: object

In [23]: mlb.fit(_)
Out[23]: MultiLabelBinarizer()

In [24]: mlb.classes_
Out[24]: array(['Hello', 'I', 'Pie', 'cake', 'good', 'is', 'like', 'world!'], dtype=object)

hoary wigeon Jul 18, 2021, 2:40 PM

#

eager imp Jul 18, 2021, 2:40 PM

#

i have three lists of 1D-arrays (training & validation), representing three different categories and i'd like to use keras Sequential to identify the different cases, how can i feed them into a model.fit call? i'm also looking at keras Dataset, but can't figure out how to use those lists of arrays as labeled data..

hoary wigeon Jul 18, 2021, 2:40 PM

#

ohh i was using str()

#

serene scaffold Jul 18, 2021, 2:41 PM

#

hoary wigeon ohh i was using str()

yeah, str is an accessor

hoary wigeon Jul 18, 2021, 2:42 PM

#

now im using multilabelbin. after learning SVM i will try with vectorizer

#

which one is good method to create a review rating prediction ?

serene scaffold Jul 18, 2021, 2:42 PM

#

hoary wigeon which one is good method to create a review rating prediction ?

this sounds like a variation on sentiment analysis. which is known for being annoyingly difficult

#

but look into approaches for sentiment analysis with product reviews.

hoary wigeon Jul 18, 2021, 2:47 PM

#

i still have a doubt

#

even if i train model and test it

#

how can i test from out of samples

serene scaffold Jul 18, 2021, 2:47 PM

#

hoary wigeon how can i test from out of samples

you can find another dataset with similar data and see how it goes

hoary wigeon Jul 18, 2021, 2:48 PM

#

but

#

if i want to use gui for input

#

model will find unexpected text shape

#

😐.

serene scaffold Jul 18, 2021, 2:49 PM

#

the binarizer will ignore tokens it's not familiar with

hoary wigeon Jul 18, 2021, 2:50 PM

#

i still dint get this

#

there are 15754 cols

#

for testing i will need array with size 15754cols

#

but user input may be like This was a great product i ever had.

#

i never deployed a model before this

#

i want to deploy this one

grave breach Jul 18, 2021, 3:14 PM

#

@hoary wigeon Let's assume there are only 3 words

#

"Hello", "I" and "am"

#

"Hello" would be

#

{1, 0, 0}

#

"I" would be

#

{0, 1, 0}

#

And "am" would be

#

{0, 0, 1}

#

That means that your sentence will become

#

{{1, 0, 0}, {0, 1, 0}, {0, 0 ,1}}

#

(Ignore the curly braces, just used to another programming language)

#

Did you got it now?

hoary wigeon Jul 18, 2021, 3:18 PM

#

grave breach Did you got it now?

but how am will become {0,0,1} ?

#

when i pass it from gui

grave breach Jul 18, 2021, 3:19 PM

#

Let's suppose these are your words {"Gabe", "Hello", "I", "am"}

#

In order to represent a word we create a vector with the same lenght (usually one more, but ignore in this case) as our words

hoary wigeon Jul 18, 2021, 3:20 PM

#

ya thats the point

#

how to create the vector of same length

grave breach Jul 18, 2021, 3:20 PM

#

So, if our words are {"Gabe", "Hello", "I", "am"} we will create a vector with 4 items

#

If we want to represent the name "Gabe"

#

Since Gabe is the first element in the list

#

We will do

#

{1, 0, 0, 0}

hoary wigeon Jul 18, 2021, 3:21 PM

#

yeah i understand this

#

but

#

my question is

#

my model has vector record of 15754 items

grave breach Jul 18, 2021, 3:22 PM

#

Ok

hoary wigeon Jul 18, 2021, 3:22 PM

#

and i will pass some random text from browser

#

suppor , this is is a good product

#

how to convert it into same vector length

#

: )

grave breach Jul 18, 2021, 3:23 PM

#

first remove punctuation

#

then stopwords

hoary wigeon Jul 18, 2021, 3:23 PM

#

excluded

#

text_preprocessing are done

grave breach Jul 18, 2021, 3:23 PM

#

then you turn the text into a list of word

hoary wigeon Jul 18, 2021, 3:23 PM

#

yes

#

done

grave breach Jul 18, 2021, 3:24 PM

#

then you create a blank array, that will contains your vectors

#

for every world in the array containing words you append the corresponding vector to the new array

#

So you have your vectorized text

hoary wigeon Jul 18, 2021, 3:24 PM

#

okay

#

this is good with dict : | i guess

#

and then get all values in array type

grave breach Jul 18, 2021, 3:25 PM

#

If you need to represent meaning for better results

#

use something like GloVe or Word2Vec (even bert, but it's too heavy for your needs I guess)

hoary wigeon Jul 18, 2021, 3:26 PM

#

ok

serene scaffold Jul 18, 2021, 3:39 PM

#

grave breach use something like GloVe or Word2Vec (even bert, but it's too heavy for your nee...

bert is prohibitively expensive without a GPU, unfortunately

cedar sun Jul 18, 2021, 3:40 PM

#

guys

#

what models are the best to learn do things alone?

#

the genetic ones?

serene scaffold Jul 18, 2021, 3:40 PM

#

hoary wigeon

try doing df.sum(axis=1). It should correspond to the number of tokens in each sentence. If that's the case, then you're on the right track in terms of how you described your solution.

serene scaffold Jul 18, 2021, 3:41 PM

#

cedar sun what models are the best to learn do things alone?

the best models to learn what, exactly?

cedar sun Jul 18, 2021, 3:41 PM

#

like, if i want a model that plays pacman, for example, a genetic one?

#

to do one task alone

serene scaffold Jul 18, 2021, 3:41 PM

#

cedar sun to do one task alone

most models do one task. for playing a game, you might try reinforcement learning.

cedar sun Jul 18, 2021, 3:42 PM

#

why not genetic ones?

#

i mean, one task alone, not training, where u need data etc

#

just, play

serene scaffold Jul 18, 2021, 3:42 PM

#

cedar sun why not genetic ones?

I don't really know what those are. but one of my coworkers used reinforcement learning to train an AI to play a game and it worked really well.

cedar sun Jul 18, 2021, 3:42 PM

#

if u die, punish

#

the genetic ones are models that start with random values

#

and u penalize them and reward them

#

so imagine, driving a car

#

the longer the car moves, the more rewards

serene scaffold Jul 18, 2021, 3:43 PM

#

sounds a lot like reinforcement learning to me.

cedar sun Jul 18, 2021, 3:43 PM

#

so weights of that model will have more chances to be transmited to the next generation

#

but they all have random weights

#

u initialize like 10 models with random weights, and every generation, u get new weights, randomly, but those who achieve higher scores on previous generation will be more likely to keep some wegiths

#

https://www.youtube.com/watch?v=uwz8JzrEwWY

YouTube

Dollar Akshay

Genetic Algorithm. Learning to walk - OpenAI Gym

An AI that learns to walk on its own after several generations.

Program written using python and the OpenAI Gym framework
This is the Bipedal Walker v2 Environment
Link to Code and Presentation : https://gym.openai.com/evaluations/eval_ujFWHmoqSniDh8cErKCVpA

▶ Play video

#

so i didnt want to tell the model how to play pacman

#

i want it to discover on its own

hoary wigeon Jul 18, 2021, 4:08 PM

#

I have a question

#

does code affects storage ?

cedar sun Jul 18, 2021, 4:11 PM

#

ofc lol

hoary wigeon Jul 18, 2021, 4:13 PM

#

im talking about harddisk storage

#

not about ram

tidal bough Jul 18, 2021, 4:14 PM

#

how's disk storage involved? this error is about lacking RAM for your big array

grave breach Jul 18, 2021, 4:14 PM

#

@cedar sun Try Deep Q-Learning

#

But if you're a beginner I suggest you to start learning from the cross-entropy method

#

For gradually learning math and theory for DQN

hoary wigeon Jul 18, 2021, 4:18 PM

#

tidal bough how's disk storage involved? this error is about lacking RAM for your big array

my my drive c was having 9Gb free and after this error there was 7 gb free

#

after running the notebook again from start

#

there was 8.5 free storage space

grave breach Jul 18, 2021, 4:19 PM

#

Just use StackOverflow

#

https://stackoverflow.com/questions/57507832/unable-to-allocate-array-with-shape-and-data-type

Stack Overflow

Unable to allocate array with shape and data type

I'm facing an issue with allocating huge arrays in numpy on Ubuntu 18 while not facing the same issue on MacOS.

I am trying to allocate memory for a numpy array with shape (156816, 36, 53806)
with...

tidal bough Jul 18, 2021, 4:20 PM

#

hoary wigeon my my drive c was having 9Gb free and after this error there was 7 gb free

perhaps you're using a pagefile as virtual memory and this allocation caused it to get filled somewhat

hoary wigeon Jul 18, 2021, 4:20 PM

#

yeah

grave breach Jul 18, 2021, 4:20 PM

#

@hoary wigeon Based on what they said on stackoverflow

#

It's something related about OS configuration

#

You should run a command

#

But I don't think you would have access to it on a remote instance

#

Just run it on your local device

#

(Try)

hoary wigeon Jul 18, 2021, 4:21 PM

#

?

grave breach Jul 18, 2021, 4:22 PM

#

Try running it on your pc, just to see if it works

#

So you can understand if your problem is the same as the guy on stack overflow

hoary wigeon Jul 18, 2021, 4:23 PM

#

#

yeah

grave breach Jul 18, 2021, 4:23 PM

#

Wait

#

I don't think on windows would work

hoary wigeon Jul 18, 2021, 4:24 PM

#

i can resize it

grave breach Jul 18, 2021, 4:24 PM

#

Usually, remote instances works on POSIX operatives systems

hoary wigeon Jul 18, 2021, 4:24 PM

#

do i need to reduce the amount ?

grave breach Jul 18, 2021, 4:24 PM

#

No, just to change a setting

#

Read the stack overflow thread

hoary wigeon Jul 18, 2021, 4:24 PM

#

ok

grave breach Jul 18, 2021, 4:25 PM

#

cedar sun the genetic ones?

No

hoary wigeon Jul 18, 2021, 4:29 PM

#

  Increasing page file size may help prevent instabilities and crashing in Windows. However, a hard drive read/write times are much slower than what they would be if the data were in your computer memory. Having a larger page file is going to add extra work for your hard drive, causing everything else to run slower. Page file size should only be increased when encountering out-of-memory errors, and only as a temporary fix. A better solution is to adding more memory to the computer.```

arctic wedgeBOT Jul 18, 2021, 6:14 PM

#

@lime berry Please don't try to ping @everyone or @here. Your message has been removed. If you believe this was a mistake, please let staff know!

eager imp Jul 18, 2021, 7:45 PM

#

i have data in form of snippets of shape (100000,), labelled for 3 categories (currently i have 3 arrays containing those snippets). i'd like to use keras to categorize them, so i built a sequential model, but i am not sure what the input parameters should be to make this work or whether i need to reshape the date somehow.. any advice?

desert oar Jul 18, 2021, 7:45 PM

#

eager imp i have data in form of snippets of shape (100000,), labelled for 3 categories (c...

so you have 3 arrays, and the sum of the 3 array lengths is 100_000?

eager imp Jul 18, 2021, 7:46 PM

#

no, each array is 100 snippets long

desert oar Jul 18, 2021, 7:46 PM

#

what are the snippets? code snippets?

eager imp Jul 18, 2021, 7:46 PM

#

so the sum is (300, 100000)

#

1D float data

desert oar Jul 18, 2021, 7:47 PM

#

oh, each one is already a vector of... 100k elements? what the heck?

eager imp Jul 18, 2021, 7:47 PM

#

multi electrode array

desert oar Jul 18, 2021, 7:47 PM

#

so effectively you have a (300, 100_000) dataset and 300 labels, right?

eager imp Jul 18, 2021, 7:47 PM

#

3 labels

desert oar Jul 18, 2021, 7:47 PM

#

yes, but the label vector has 300 elements

eager imp Jul 18, 2021, 7:47 PM

#

yes

desert oar Jul 18, 2021, 7:48 PM

#

so do that

#

just shuffle the rows before feeding into the NN

lapis sequoia Jul 18, 2021, 7:48 PM

#

i want to make a piano tiles ai but i need help i got the clicking part done using win32api code = def click():
win32api.mouse_event(win32con.MOUSEEVENTF_LEFTDOWN,0,0)
time.sleep(0.001)
win32api.mouse_event(win32con.MOUSEEVENTF_LEFTUP,0,0)

while keyboard.is_pressed('esc') == False:
click()

desert oar Jul 18, 2021, 7:48 PM

#

@lapis sequoia this is better asked in a help channel, because it's more about win32api than about "AI" as such. see #❓｜how-to-get-help for instructions

lapis sequoia Jul 18, 2021, 7:48 PM

#

ok thanks

desert oar Jul 18, 2021, 7:48 PM

#

or are you asking about "the rest of it"?

lapis sequoia Jul 18, 2021, 7:49 PM

#

im new to this server

desert oar Jul 18, 2021, 7:49 PM

#

i.e. not the windows part?

#

as in, you need help building a reinforcement learning agent or something like that?

eager imp Jul 18, 2021, 7:50 PM

#

desert oar so do that

cnn_model.add(Conv1D(filters=16, 
                        kernel_size=80,
                        padding="same"))

#

i need help with these parameters

#

ah, wrong one

#

cnn_model.add(
    Conv1D(
        input_shape=(100000, 1),
        strides=2,
        filters=16,
        kernel_size=80,
        padding="same",
    )
)

#

that's my first one

#

input_shape should be (300, 100000) or something else?

desert oar Jul 18, 2021, 7:53 PM

#

i think (100_000, 1) makes sense

#

"100k time steps, 1 vector"

#

that's how the docs says it should be interpreted

eager imp Jul 18, 2021, 7:54 PM

#

hmm.. okay, let me try

desert oar Jul 18, 2021, 7:54 PM

#

cnn_model is a keras.Sequential right?

eager imp Jul 18, 2021, 7:54 PM

#

yeah

#

it gets into the first epoch, but raises a ValueError: Input 0 of layer sequential is incompatible with the layer: : expected min_ndim=3, found ndim=2. Full shape received: (None, 100000)

#

@desert oar any idea?

lapis sequoia Jul 18, 2021, 8:06 PM

#

ive got a question in machine learning
so the sigmoid function compresses the values of the sum into the range 0 to 1
but you can use other activation functions, such as ReLU or tanh, whos ranges are (0, inf), and (-1,1) respectively
how do you factor for this different range?
do you normalise for example in ReLU by dividing by the largest activation in each layer
and like (A+1)/2 for tanh, to bring it into the (0,1) range

eager imp Jul 18, 2021, 8:08 PM

#

i think that depends strongly on what the data mean

#

where they are from

#

imagine your largest activation in your training data could be tiny compared to what "RL input" might cover, your whole assumptions could be invalid if you don't account for the full range (sensor data maybe)

#

(somehow that makes me think of autism sensory overload..)

desert oar Jul 18, 2021, 8:13 PM

#

lapis sequoia ive got a question in machine learning so the sigmoid function compresses the va...

"inside" the NN, between hidden layers, the model would learn different weights to adjust for the different activations

#

batch normalization is also used to help control the gradients "inside" the model

desert oar Jul 18, 2021, 8:14 PM

#

eager imp it gets into the first epoch, but raises a ``` ValueError: Input 0 of layer s...

maybe you need (100_000,) as the conv1d input shape then

lapis sequoia Jul 18, 2021, 8:14 PM

#

could you explain that to me plz salt rock lamp

#

because theres nothing limiting an activation of being >1 right

#

if ReLU is used

desert oar Jul 18, 2021, 8:15 PM

#

yeah, that in and of itself isn't a problem

lapis sequoia Jul 18, 2021, 8:15 PM

#

you could just continue, and then normalize at the end?

desert oar Jul 18, 2021, 8:15 PM

#

wdym?

lapis sequoia Jul 18, 2021, 8:15 PM

#

so continue the process with those activations being >1, and only normalising once you get to the output layer?

desert oar Jul 18, 2021, 8:16 PM

#

yes, even if you normalize between every layer you still need to apply sigmoid or softmax to the output layer, if the output needs to be in a certain range

eager imp Jul 18, 2021, 8:16 PM

#

desert oar maybe you need `(100_000,)` as the conv1d input shape then

could the strides=2 parameter cause the issue?

lapis sequoia Jul 18, 2021, 8:16 PM

#

hmmmm ok

#

thank you

#

ill look more into it

desert oar Jul 18, 2021, 8:17 PM

#

doubt it @eager imp

#

afaik the stride is just the "step size" between convolution passes right?

lapis sequoia Jul 18, 2021, 8:17 PM

#

you seem like a very intelligent fellow, any nice resources for learning more about NN and machine learning?

eager imp Jul 18, 2021, 8:17 PM

#

i know strides from numpy, but i'm a bit at a loss with keras

desert oar Jul 18, 2021, 8:18 PM

#

@lapis sequoia we have a bunch of resources in the pinned messages

desert oar Jul 18, 2021, 8:18 PM

#

eager imp i know strides from numpy, but i'm a bit at a loss with keras

https://machinelearningmastery.com/padding-and-stride-for-convolutional-neural-networks/

Machine Learning Mastery

A Gentle Introduction to Padding and Stride for Convolutional Neura...

The convolutional layer in convolutional neural networks systematically applies filters to an input and creates output feature maps. Although the […]

eager imp Jul 18, 2021, 8:18 PM

#

ty

lapis sequoia Jul 18, 2021, 8:18 PM

#

thanks ❤️

sleek dagger Jul 18, 2021, 10:01 PM

#

Is torch or tensorflow Faster for cpu

grave frost Jul 18, 2021, 10:48 PM

#

sleek dagger Is torch or tensorflow Faster for cpu

none

#

for training models, CPU is pretty bad

#

if you talking about CPU ops, they are rarely a bottleneck and don't really differ in any significant perf

sleek dagger Jul 18, 2021, 11:02 PM

#

I am trying to use Cpu because my Gpu (GT 1030) Is pretty slow, I dont think I can use the gpu version

desert oar Jul 18, 2021, 11:33 PM

#

@sleek dagger https://medium.com/@samnco/using-the-nvidia-gt-1030-for-cuda-workloads-on-ubuntu-16-04-4eee72d56791 it will technically work but the low GPU memory will be limiting for "big" workloads

Medium

Using the nVidia GT 1030 for CUDA workloads on Ubuntu 16.04

Recently nVidia released a new low-end card, the nVidia GT1030. Its specs are so low that it is not even listed on the official CUDA…

#

I imagine that it would still perform better than CPU

sleek dagger Jul 18, 2021, 11:34 PM

#

I see, Thank you! This info is a really big help

desert oar Jul 18, 2021, 11:34 PM

#

But it looks like even a toy model will be close to maxing out the VRAM

#

Might be better than nothing, and for all we know newer versions of TF are a lot more optimized

sleek dagger Jul 18, 2021, 11:36 PM

#

I see

#

I was wondering because in blender when rendering. Using cpu it is way faster than gpu on optix or Cuda. So I was thinking the same would apply

desert oar Jul 18, 2021, 11:38 PM

#

I guess you can try it, idk!

sleek dagger Jul 18, 2021, 11:39 PM

#

I will have to download drivers and see

#

Thank you for the info though this will help alot

cold mantle Jul 19, 2021, 1:36 AM

#

How in the word do I install PyStan?

desert oar Jul 19, 2021, 2:13 AM

#

cold mantle How in the word do I install PyStan?

are you stuck at some point on the instructions?

brisk lintel Jul 19, 2021, 4:50 AM

#

anyone here has experience with streamlit and altair charts?

pastel valley Jul 19, 2021, 5:26 AM

#

what should i learn for data science?

peak ridge Jul 19, 2021, 5:30 AM

#

Same Question (How can i get started and make career in data science)
But my python basics are strong
Have been doing web dev from some months in django and flask
I have capability to handle pressure (i mean to struggle those 10-15 days when we start something new)
I code in double digit hrs
Buy my maths is weak

snow hearth Jul 19, 2021, 6:09 AM

#

peak ridge Same Question (How can i get started and make career in data science) But my py...

you might need ood maths in data science

#

like calculus is needed

#

but you can learn that in like a week or so

snow hearth Jul 19, 2021, 6:10 AM

#

pastel valley what should i learn for data science?

most of the people use python , r and mysql or sql for data science

peak ridge Jul 19, 2021, 6:10 AM

#

I'm 16 umm

snow hearth Jul 19, 2021, 6:10 AM

#

languages are mostly python adn r

#

but database can change

snow hearth Jul 19, 2021, 6:10 AM

#

peak ridge I'm 16 umm

tehn take maths in 11th and 12th

peak ridge Jul 19, 2021, 6:10 AM

#

Ik databases stuff too

snow hearth Jul 19, 2021, 6:10 AM

#

it will be enough

peak ridge Jul 19, 2021, 6:10 AM

#

snow hearth tehn take maths in 11th and 12th

Took

snow hearth Jul 19, 2021, 6:10 AM

#

for data science to start with

peak ridge Jul 19, 2021, 6:10 AM

#

But i hate

#

Studies

snow hearth Jul 19, 2021, 6:10 AM

#

peak ridge Took

noceee tha maths is enough

peak ridge Jul 19, 2021, 6:10 AM

#

Of school

snow hearth Jul 19, 2021, 6:10 AM

#

maths you will need

peak ridge Jul 19, 2021, 6:10 AM

#

Ohh

#

Which which chapter and concepts

snow hearth Jul 19, 2021, 6:11 AM

#

speificall the calculus
part it is very imposrtabt

snow hearth Jul 19, 2021, 6:11 AM

#

peak ridge Which which chapter and concepts

do it overall , anything will be needed

peak ridge Jul 19, 2021, 6:11 AM

#

Ohh

snow hearth Jul 19, 2021, 6:11 AM

#

but mostly it will be limits and stuff , stats and probability

peak ridge Jul 19, 2021, 6:12 AM

#

Stats
Prob
Calcus
And?

snow hearth Jul 19, 2021, 6:12 AM

#

in 12th you will have 5 chapters on calculus so just keep your 11th basics strong

snow hearth Jul 19, 2021, 6:12 AM

#

peak ridge Stats Prob Calcus And?

that will cover major maths for data science

#

rest I am also not sure , I am also learnign about it so

peak ridge Jul 19, 2021, 6:13 AM

#

My maths rip

#

😭😭

snow hearth Jul 19, 2021, 6:13 AM

#

looks like you didn't study past few years cuz the amout of stuff you have written in bio

snow hearth Jul 19, 2021, 6:13 AM

#

peak ridge My maths rip

give it time , it will be fine

peak ridge Jul 19, 2021, 6:13 AM

#

Till now my maths suck

#

Ohk

snow hearth Jul 19, 2021, 6:14 AM

#

just practice on it more

#

you will be good at maths

peak ridge Jul 19, 2021, 6:16 AM

#

Ohk

#

So have to wait more 2 years

#

To complete 11th 12th

#

@snow hearth

snow hearth Jul 19, 2021, 6:18 AM

#

peak ridge So have to wait more 2 years

you can do it rn also

#

if you want

#

there are videos on internet

#

but for data science you can start ai or doing ml

#

which stream you have chosen btw

peak ridge Jul 19, 2021, 6:19 AM

#

Pcm + cs
(I'm not good in studies 60-70% student ) but love tech so even my parents told to take commerce i took science i code whole day

#

@snow hearth

snow hearth Jul 19, 2021, 6:19 AM

#

peak ridge Pcm + cs (I'm not good in studies 60-70% student ) but love tech so even my par...

I would say stop and keep coing at side

#

if you really love tech you can take arts also

#

there are a lot of opportunities with that too

peak ridge Jul 19, 2021, 6:20 AM

#

Ohh

#

Bca and Mca

snow hearth Jul 19, 2021, 6:20 AM

#

like iiit delhi take arts students also

#

in btech programs

snow hearth Jul 19, 2021, 6:20 AM

#

peak ridge Bca and Mca

no it is not the only thing , you have to explore it

#

but if you can try changing it to arts or commerce

#

cuz with pcm you cannot code

#

you need to give time to pcm

#

I also have the same combination so ik how hard it is

peak ridge Jul 19, 2021, 6:21 AM

#

Ohh

#

It's too hard

#

So i have to leave to code?

snow hearth Jul 19, 2021, 6:21 AM

#

I am going to college this year like taking admission

snow hearth Jul 19, 2021, 6:21 AM

#

peak ridge So i have to leave to code?

not leave but it will be very less cuz pcm need a lot of time

peak ridge Jul 19, 2021, 6:21 AM

#

Ohh

snow hearth Jul 19, 2021, 6:22 AM

#

like it is not easy to do pcm and code alot

peak ridge Jul 19, 2021, 6:22 AM

#

Ohh

#

Ohk then

#

I'll study from today and code 3 hrs max to max

snow hearth Jul 19, 2021, 6:22 AM

#

you cange to arts or commerce rn I think cuz it is very easy in 1th

peak ridge Jul 19, 2021, 6:22 AM

#

Rip Passion

snow hearth Jul 19, 2021, 6:22 AM

#

peak ridge I'll study from today and code 3 hrs max to max

do pomodoros

peak ridge Jul 19, 2021, 6:22 AM

#

snow hearth you cange to arts or commerce rn I think cuz it is very easy in 1th

Now i have choosen
I won't step back now

snow hearth Jul 19, 2021, 6:22 AM

#

like do 50 min sessiona dn then 10 min break

peak ridge Jul 19, 2021, 6:23 AM

#

Ohh

snow hearth Jul 19, 2021, 6:23 AM

#

peak ridge Now i have choosen I won't step back now

your wish , but I like the spirit

snow hearth Jul 19, 2021, 6:23 AM

#

peak ridge Ohh

yeah , like start doing pomodoros techniques it is awesome ngl

#

you will study and will not get tired also

#

I use that mostly to study or work

peak ridge Jul 19, 2021, 6:23 AM

#

Umm

#

How much should i study in a day

#

I dont want to clear jee or stuff

#

Not that interested

#

Just need a btech clg

snow hearth Jul 19, 2021, 6:24 AM

#

like study 2 to 4hrs a day to start with ( except classes or tutions)

snow hearth Jul 19, 2021, 6:24 AM

#

peak ridge Just need a btech clg

for that you need good 12th marks or jee

#

both are needed

#

atleast good 12th marks are

peak ridge Jul 19, 2021, 6:25 AM

#

Umm

snow hearth Jul 19, 2021, 6:25 AM

#

but it will be easy for you cuz your syllabus is splitted in half

peak ridge Jul 19, 2021, 6:25 AM

#

What if i study 5-6 hrs a day

peak ridge Jul 19, 2021, 6:25 AM

#

snow hearth but it will be easy for you cuz your syllabus is splitted in half

Is that valid for class 12th. Next session too cez now i just entered 11th

snow hearth Jul 19, 2021, 6:26 AM

#

peak ridge What if i study 5-6 hrs a day

amazing , but be consistent, discipline is what is important , start is easy but to keep it going is difficult

snow hearth Jul 19, 2021, 6:26 AM

#

peak ridge Is that valid for class 12th. Next session too cez now i just entered 11th

I am not sure but for this year 12th it is valid

#

I am not the ont th lol

peak ridge Jul 19, 2021, 6:26 AM

#

XD

#

U lucky peeps

#

12th cancelled

#

My 10th cancelled to xD

snow hearth Jul 19, 2021, 6:26 AM

#

peak ridge U lucky peeps

no , it is hard cuz e don't have any idea about our marks

#

and I need to get good for clg

peak ridge Jul 19, 2021, 6:27 AM

#

Ohh

snow hearth Jul 19, 2021, 6:27 AM

#

your 10th ones didn't really matter tbh

peak ridge Jul 19, 2021, 6:27 AM

#

Ohh

#

I have to make a good time table

#

And have to follow that

snow hearth Jul 19, 2021, 6:27 AM

#

it is only for you to get the stream in 11th rest no one ask for them

snow hearth Jul 19, 2021, 6:27 AM

#

peak ridge I have to make a good time table

yess make todo ist also

#

if you want I can help you with softwares and stuff

#

Ik few good ones and also minimal cuz i am a minimalist

peak ridge Jul 19, 2021, 6:28 AM

#

I have made like many web apps with backend and full logic

#

Django op

snow hearth Jul 19, 2021, 6:28 AM

#

peak ridge Django op

I wanted to learn that but I didn't from day after tomorrow I will start

peak ridge Jul 19, 2021, 6:29 AM

#

Nice I'll help u

snow hearth Jul 19, 2021, 6:29 AM

#

Ik python like coding it for past 2 years but due to 12th didn't make much projects, now I am gonna get back

peak ridge Jul 19, 2021, 6:29 AM

#

I'm lil good at that

#

Web tech

snow hearth Jul 19, 2021, 6:29 AM

#

peak ridge Nice I'll help u

sure i will ask if I need help , tho I have found resurses already

snow hearth Jul 19, 2021, 6:29 AM

#

peak ridge Web tech

I know html and css also jsut gonna get then together and make projects for fun

peak ridge Jul 19, 2021, 6:30 AM

#

snow hearth Ik python like coding it for past 2 years but due to 12th didn't make much proje...

That's what the problem is
I dont want to do this
I want to make progress in my code Also
Cez in clg i want to be different
Cez if i leave to code rn
So after 2 years i have to do concepts again and then again django
My efforts and many months wasted

snow hearth Jul 19, 2021, 6:31 AM

#

peak ridge That's what the problem is I dont want to do this I want to make progress in my ...

don't leave just do it side by side and reduce time

peak ridge Jul 19, 2021, 6:31 AM

#

Nice

#

Ohk

#

Can u suggest me time table plz

#

In which i can arrange

#

3-4 hrs code

snow hearth Jul 19, 2021, 6:32 AM

#

I cannot say like if you good at cod you will be different cuz people really don't care how much you good at it if you only do it to look cool and be different.I really don't understand that thinking

#

I just do cuz I love tech tbh. I only get better than myself eveyday rather than comparing myself to others

peak ridge Jul 19, 2021, 6:33 AM

#

I do cez i love to code

snow hearth Jul 19, 2021, 6:33 AM

#

peak ridge Ohk

I don't have one tbh. It will be different for everyone and for your scheddule

snow hearth Jul 19, 2021, 6:33 AM

#

peak ridge I do cez i love to code

then don't do it for hte sake you wanna be different. everyone is different in their own way not different in what tehy do

#

it looks very unprofessional or very silly to read that

peak ridge Jul 19, 2021, 6:34 AM

#

Ohh

snow hearth Jul 19, 2021, 6:34 AM

#

I wanna be different by doing coding

peak ridge Jul 19, 2021, 6:34 AM

#

Umm

#

I hate indian education system

#

Thats what I'll say in last

snow hearth Jul 19, 2021, 6:35 AM

#

peak ridge I hate indian education system

it is good , you just choose the wrong stream

peak ridge Jul 19, 2021, 6:35 AM

#

And yes
I can't blame them
It's my fault why I'm not dedicated towards studies

snow hearth Jul 19, 2021, 6:35 AM

#

they way you love code , you should have gone with arts

#

you can still get good jobs at fang companies with a btech degreee

#

if you want

peak ridge Jul 19, 2021, 6:35 AM

#

Ohk i have chance still

snow hearth Jul 19, 2021, 6:36 AM

#

yeah you can change even when you enter 12th

#

like alot of people change streams

#

you just have to research and discuss with your family

#

I can only suggest it to you

peak ridge Jul 19, 2021, 6:36 AM

#

I won't change now
I'll manage code
No more chatting
No more calls with gf

#

Nothing

snow hearth Jul 19, 2021, 6:37 AM

#

peak ridge I won't change now I'll manage code No more chatting No more calls with gf

your choice, it is not leaving everything thing , it is just managing time according to your priority

peak ridge Jul 19, 2021, 6:37 AM

#

5-6 hrs study
4 hrs code
That's it
I was managing life with code
Now I'll manage studies+ code with life

snow hearth Jul 19, 2021, 6:37 AM

#

talking and having gf or frinds is alo important otherwise you will burnout

snow hearth Jul 19, 2021, 6:38 AM

#

peak ridge 5-6 hrs study 4 hrs code That's it I was managing life with code Now I'll ma...

that is too much for your brain

#

make coding 2hrs and chat with your gf

#

you will feel more normal and relax everyday

peak ridge Jul 19, 2021, 6:38 AM

#

Ohh

#

Let's try that
From Now not tommorow or later

snow hearth Jul 19, 2021, 6:38 AM

#

6hrs of pcm is very hard buddy and the you cannot code 4hrs . you will burnout in a day or two

#

Start slow and after a mnth when you comfortable with this time table

#

change your time according to yur priorities that time

peak ridge Jul 19, 2021, 6:39 AM

#

Yes cez coding in that high lvl django Databases logical thinking also needs my energy

snow hearth Jul 19, 2021, 6:39 AM

#

peak ridge Yes cez coding in that high lvl django Databases logical thinking also needs my ...

yess , take it slow , and relax your mind also

#

and dang you hav a gf also at 16yrs age

peak ridge Jul 19, 2021, 6:40 AM

#

XD

#

And u?

snow hearth Jul 19, 2021, 6:40 AM

#

peak ridge And u?

no I am not intersted in any gf and things

#

I never have one and I am not interested in making one also

peak ridge Jul 19, 2021, 6:41 AM

#

snow hearth no I am not intersted in any gf and things

That's better

snow hearth Jul 19, 2021, 6:41 AM

#

rn like for atleast this year

peak ridge Jul 19, 2021, 6:41 AM

#

Ohk

#

I'm go buy books

#

XD

snow hearth Jul 19, 2021, 6:41 AM

#

peak ridge I'm go buy books

11th pcm books ?

peak ridge Jul 19, 2021, 6:41 AM

#

Yes

snow hearth Jul 19, 2021, 6:42 AM

#

download it online lol

peak ridge Jul 19, 2021, 6:42 AM

#

I dont have books too rn

snow hearth Jul 19, 2021, 6:42 AM

#

why waste money when it is all online

#

you in any coaching ?

#

offline or online

#

?

peak ridge Jul 19, 2021, 6:42 AM

#

Rn nope
But gonna join

#

Very very soon

#

Ig i can join today too

snow hearth Jul 19, 2021, 6:42 AM

#

peak ridge Rn nope But gonna join

wait I willl suggest you something

peak ridge Jul 19, 2021, 6:42 AM

#

Ohk

snow hearth Jul 19, 2021, 6:42 AM

#

try doing the online videos from youtube

#

or maybe take physiswallah

#

they are very affordable and very good

#

like it is just 5K for whole year

peak ridge Jul 19, 2021, 6:43 AM

#

Yes

#

My friend also took that

#

Damn

snow hearth Jul 19, 2021, 6:43 AM

#

I always say to everyone to go for yt

peak ridge Jul 19, 2021, 6:43 AM

#

I'm gonna but it

snow hearth Jul 19, 2021, 6:43 AM

#

you can learn eveyrhing fo rfree

peak ridge Jul 19, 2021, 6:43 AM

#

Today

peak ridge Jul 19, 2021, 6:44 AM

#

snow hearth like it is just 5K for whole year

My friend too took that lmao

snow hearth Jul 19, 2021, 6:44 AM

#

peak ridge I'm gonna but it

then you don't have to join any institute

#

it is very good

peak ridge Jul 19, 2021, 6:44 AM

#

Yes self study is best

snow hearth Jul 19, 2021, 6:44 AM

#

and you will save lacks of money

snow hearth Jul 19, 2021, 6:44 AM

#

peak ridge Yes self study is best

yeah

#

also watch maths videos from neha mam's channel on yt

#

you will find playlist

peak ridge Jul 19, 2021, 6:44 AM

#

Ohh

snow hearth Jul 19, 2021, 6:44 AM

#

you will understand better

#

for sure

#

trust me you will enjoy maths

peak ridge Jul 19, 2021, 6:45 AM

#

Ohk then

#

I'll give my 101%

snow hearth Jul 19, 2021, 6:45 AM

#

I will send you link wait

peak ridge Jul 19, 2021, 6:45 AM

#

Until i ace it

snow hearth Jul 19, 2021, 6:45 AM

#

peak ridge I'll give my 101%

niceee , just don't burn out

drifting ermine Jul 19, 2021, 6:59 AM

#

can anyone tell me frm where do i have to learn AI ???i mean which course is best for AI

short heart Jul 19, 2021, 7:39 AM

#

When I try to create model inside of
with strategy.scope():
it wont show me metric and loss during training
1/1 [==============================] - 61s 61s/step - loss: nan - accuracy: nan
Is it ok or is it some problem?

lilac geyser Jul 19, 2021, 8:13 AM

#

Can we consider a model with R2 score of 0.70448, As a good model?

#

Please @ me

lapis sequoia Jul 19, 2021, 8:52 AM

#

Is it ok to use binary features for continous regression target ?

dire echo Jul 19, 2021, 9:48 AM

#

Are theano similiar to tf?

#

Ik they diffrent but if i want to learn tf but i learn theano instead is that good?

tidal bough Jul 19, 2021, 9:53 AM

#

Not really?

#

For one, theano is abandoned, I believe.

#

But also, I'm not sure it supports autograd for backpropagation, which is about the biggest feature of PyTorch and TF.

whole hamlet Jul 19, 2021, 10:14 AM

#

https://www.kaggle.com/tiwariayan/what-is-really-happening-in-the-epl

What is really happening in the EPL?

Explore and run machine learning code with Kaggle Notebooks | Using data from English Premier League(2020-21)

native bay Jul 19, 2021, 10:35 AM

#

hey can someone suggest me ideas for projects i really cant find any
i know
RNN
CNN
and Autoencoders

grave frost Jul 19, 2021, 10:36 AM

#

tidal bough But also, I'm not sure it supports autograd for backpropagation, which is about ...

https://tenor.com/view/skeletor-wat-what-gif-18497036

Tenor

#

what else would someone use it for then?

tidal bough Jul 19, 2021, 10:37 AM

#

Theano? For optimizing chains of operations on arrays, as far as I know

grave frost Jul 19, 2021, 10:39 AM

#

ooof

#

poor chaps who tried to build models with it

tidal bough Jul 19, 2021, 10:40 AM

#

like, the way I understand it, it's smart enough to figure out that a*2+b*2 is better evaluated as (a+b)*2, and more complex cases. In general, it I believe compiles the expression you are evaluating into a single optimized function that it then runs on the arrays

#

yeah, looks like it's capable of user-defined arbitrary graph optimizations even: https://theano-pymc.readthedocs.io/en/latest/extending/optimization.html

grave frost Jul 19, 2021, 10:41 AM

#

hmm pithink

tidal bough Jul 19, 2021, 10:42 AM

#

tbh some sections of theano docs make me interested in it, like supporting sparse matrices

#

scipy.sparse IIRC has let me down at times

grave frost Jul 19, 2021, 10:43 AM

#

but can't Pytorch and TF handle sparse tensors very well?

#

even jax too?

tidal bough Jul 19, 2021, 10:53 AM

#

Hmm, do they actually store them sparsely? I might check that out too

#

oh, nice https://pytorch.org/docs/stable/sparse.html

grave frost Jul 19, 2021, 10:56 AM

#

whoa, jax also has their own scipy https://jax.readthedocs.io/en/latest/jax.scipy.html

dire echo Jul 19, 2021, 11:08 AM

#

Are creating AI from scratch a good idea?

#

For chatbot

#

Or mechine learning

lilac geyser Jul 19, 2021, 11:10 AM

#

native bay hey can someone suggest me ideas for projects i really cant find any i know RNN ...

Search kaggle. You will get a lot of projects

cold mantle Jul 19, 2021, 11:50 AM

#

desert oar are you stuck at some point on the instructions?

No, I just can’t install it

dire echo Jul 19, 2021, 12:06 PM

#

Just create a random generate things

#

Using matplotlib

#

grave breach Jul 19, 2021, 1:47 PM

#

dire echo Are creating AI from scratch a good idea?

Depends, what is your level of experience?

dire echo Jul 19, 2021, 1:47 PM

#

Very low

grave breach Jul 19, 2021, 1:47 PM

#

Study first

dire echo Jul 19, 2021, 1:48 PM

#

Yes i will

keen prism Jul 19, 2021, 2:24 PM

#

does anyone know about the Google lens API? I have some questions about it.
Is there a way to get it? Is it a machine learning framework?
I've used it myself in my day-to-day, and because I'm interested in OCR I'd like to know more about it.
also it seems fairly fast.
another thing I think is cool about it is it can see math equations, so I'm curious what's at work there that's allowing it to do that.

grave breach Jul 19, 2021, 2:48 PM

#

keen prism does anyone know about the Google lens API? I have some questions about it. Is t...

You can buy access to it throught Google Cloud

keen prism Jul 19, 2021, 2:49 PM

#

grave breach You can buy access to it throught Google Cloud

I'm not familiar with that. Can you share a direct link? Also is this on github?

grave breach Jul 19, 2021, 2:49 PM

#

https://cloud.google.com/vision/automl/docs

Google Cloud

AutoML Vision documentation | Google Cloud

Enables you to train machine learning models to classify your images according to your own defined labels.

#

You can buy access to pretrained model

#

https://cloud.google.com/vision/automl/docs/reference/rest

Google Cloud

Cloud AutoML API | AutoML Vision | Google Cloud

#

@keen prism Otherwise, if you're not willing to pay, you can access Google free yet less powerful models

#

https://developers.google.com/ml-kit/vision/text-recognition

Google Developers

Text Recognition | ML Kit | Google Developers

#

https://developers.google.com/ml-kit/vision/digital-ink-recognition

Google Developers

Digital Ink Recognition | ML Kit | Google Developers

keen prism Jul 19, 2021, 2:57 PM

#

grave breach <@!237257444198187008> Otherwise, if you're not willing to pay, you can access G...

interesting. are those models stored locally?
or in other words, can i subscribe to google cloud, download the model, and unsubscribe?

grave breach Jul 19, 2021, 2:57 PM

#

The two models that I linked you (I think) are

#

But never used that

#

I don't think you can use them outside a mobile app

keen prism Jul 19, 2021, 2:57 PM

#

i have no idea how big a ml model like this is

keen shadow Jul 19, 2021, 2:57 PM

#

so with nltk
if i have a tokenized list such as [('adjectives', 'NNS'), ('adjoin', 'VBP'), ('adjoined', 'VBN')].
how would i convert it to a CFG?

grave breach Jul 19, 2021, 2:58 PM

#

keen prism i have no idea how big a ml model like this is

They're small

keen prism Jul 19, 2021, 2:58 PM

#

hmm. ok...

grave breach Jul 19, 2021, 2:58 PM

#

But I suggest you the paid google services

hexed walrus Jul 19, 2021, 2:58 PM

#

The big one is the dataset lemon_xd

cedar sun Jul 19, 2021, 3:16 PM

#

how can i use nasnet to see which architecture fits better for my problem?

#

wait, i thouigh nasnet is an ai that makes archs

grave breach Jul 19, 2021, 3:36 PM

#

cedar sun how can i use nasnet to see which architecture fits better for my problem?

Just study and make the architecture by yourself

#

Mate, by searching shortcuts for everything you won't learn anything

#

If you wish I can tell you wich direction to go, so you can learn the subject

cedar sun Jul 19, 2021, 3:38 PM

#

learn to answer what ppl ask, and stop telling them what to do

#

if u are not gonna answer, stay quiet

grave breach Jul 19, 2021, 3:38 PM

#

This is not just a Q&R channel I guess

#

It's for discussing an learning

cedar sun Jul 19, 2021, 3:39 PM

#

then im gonna block u

grave breach Jul 19, 2021, 3:39 PM

#

Why are you so aggressive?

#

Have I done something wrong to you?

short heart Jul 19, 2021, 3:54 PM

#

chill

cedar sun Jul 19, 2021, 3:54 PM

#

im chilling until i read him

#

thats why i blocked

grave breach Jul 19, 2021, 3:55 PM

#

@short heart Hi

#

How's going with the project?

#

I think I've found a very straightforward YOLO implementation that you can use

#

You just have to feed the data in

#

And then pass to the model

#

https://github.com/ultralytics/yolov5

GitHub

ultralytics/yolov5

YOLOv5 🚀 in PyTorch > ONNX > CoreML > TFLite. Contribute to ultralytics/yolov5 development by creating an account on GitHub.

#

https://github.com/ultralytics/yolov5/wiki/Train-Custom-Data

GitHub

ultralytics/yolov5

YOLOv5 🚀 in PyTorch > ONNX > CoreML > TFLite. Contribute to ultralytics/yolov5 development by creating an account on GitHub.

#

Dm me if you need some help

short heart Jul 19, 2021, 4:01 PM

#

grave breach How's going with the project?

horrible, i get headaches trying to figure out the only good code i found on kaggle on this task, right now im doing something but ill come back to this project later

grave breach Jul 19, 2021, 4:16 PM

#

short heart horrible, i get headaches trying to figure out the only good code i found on kag...

When you're back send me the link to the code that you found

#

So maybe I can help you

short heart Jul 19, 2021, 4:23 PM

#

i sent it

short heart Jul 19, 2021, 4:23 PM

#

grave breach When you're back send me the link to the code that you found

already

opaque stratus Jul 19, 2021, 5:10 PM

#

As a general rule, do you train a model for less time when the training loss starts plateauing?

grave frost Jul 19, 2021, 5:34 PM

#

grave breach Mate, by searching shortcuts for everything you won't learn anything

just block him; I had done so ages ago when he started randomly pinging and DM-ing. if he's too rude, report to modmail

quasi sparrow Jul 19, 2021, 5:38 PM

#

You need to chill out, man. That behavior is going to get you kick out of this channel and trust me, ML is going to get waaaay harder without a community to help you.

#

Does anybody know what is the "formal name" of the formula to make data into a supervised learning problem for regression?

#

I'm talking about the process of shifting data and turn features into targets for regression

#

Like this

time, measure1, measure2
1, 0.2, 88
2, 0.5, 89
3, 0.7, 87
4, 0.4, 88
5, 1.0, 90

#

into this:

#

X1, X2, X3, y
?, ?, 0.2 , 88
0.2, 88, 0.5, 89
0.5, 89, 0.7, 87
0.7, 87, 0.4, 88
0.4, 88, 1.0, 90
1.0, 90, ?, ?

#

Just need to know the name of the "formula" so I can add it to my paper

true beacon Jul 19, 2021, 5:42 PM

#

So is AI pretty much data. You have data and then you use that data to determine what the likelihood of something happening would be?

quasi sparrow Jul 19, 2021, 5:45 PM

#

I don't understand the question very well but, yeah. That's pretty much what machine learning is\

#

I'm working on a time series project

#

So I shifted data to train the model. The problem is that I don't know what the formal definition ot formula is called and I need to type the mathematical expression of basically shifting data to train a model

#

It's for school

lapis sequoia Jul 19, 2021, 5:47 PM

#

You mean like lag in time series ?

quasi sparrow Jul 19, 2021, 5:47 PM

#

No, not lag. Its just data shift to train a model

quasi sparrow Jul 19, 2021, 5:48 PM

#

quasi sparrow Like this ```py time, measure1, measure2 1, 0.2, 88 2, 0.5, 89 3, 0.7, 87 4, 0....

Here, I have a data set time 5 timestamps and 2 features.

quasi sparrow Jul 19, 2021, 5:49 PM

#

quasi sparrow ```py X1, X2, X3, y ?, ?, 0.2 , 88 0.2, 88, 0.5, 89 0.5, 89, 0.7, 87 0.7, 87, 0....

My data preprocessing for the model includes data shift like this because I want to train a regressor

#

I want to predict measure 2, so I shift my data 1 timestamp and turn the variable 2 into a target

#

Remove the first row and the last row and now it's a time serie.
What is the formal name of this practice?

#

I found it, they call it "sliding window"!

#

Sorry for the spam

uncut orbit Jul 19, 2021, 6:08 PM

#

how can i do model.predict for a tensorflow model that classifies images?

serene scaffold Jul 19, 2021, 6:17 PM

#

quasi sparrow ```py X1, X2, X3, y ?, ?, 0.2 , 88 0.2, 88, 0.5, 89 0.5, 89, 0.7, 87 0.7, 87, 0....

it looks like you might be pivoting the table, but I can't tell exactly what the transformation is.

#

oh no, I see

#

Sorry, I'm not familiar with a name for that.

#

~~here's how you can express it in code, I guess~~

#

In [6]: pd.concat((df.iloc[1:], df.iloc[:-1]), axis=1)
Out[6]: 
   time  measure1  measure2  time  measure1  measure2
0   NaN       NaN       NaN   1.0       0.2      88.0
1   2.0       0.5      89.0   2.0       0.5      89.0
2   3.0       0.7      87.0   3.0       0.7      87.0
3   4.0       0.4      88.0   4.0       0.4      88.0
4   5.0       1.0      90.0   NaN       NaN       NaN

ripe forge Jul 19, 2021, 6:20 PM

#

It's just "lagged features" or time series features. I don't think you'll need to describe it further.

keen shadow Jul 19, 2021, 6:51 PM

#

Guys I have problem with nltk nltk.CFG.fromstring(pre_model) and nltk.parse.generate.generate(self._cfg). So basically my problem is that its not generating any sentences on call. It keeps returning an empty list everytime I call it

quasi sparrow Jul 19, 2021, 6:54 PM

#

ripe forge It's just "lagged features" or time series features. I don't think you'll need t...

Yeah, it's just the professor wants the math written in the paper lol

quasi sparrow Jul 19, 2021, 6:54 PM

#

ripe forge It's just "lagged features" or time series features. I don't think you'll need t...

Thanks, I'll call it lagged features.

candid wraith Jul 19, 2021, 7:25 PM

#

how can i convert a column in my df from printing my values as "[value]" to "value"

grave breach Jul 19, 2021, 7:35 PM

#

grave frost just block him; I had done so ages ago when he started randomly pinging and DM-i...

I don't know... maybe he deserves a second chanche

grave breach Jul 19, 2021, 7:37 PM

#

opaque stratus As a general rule, do you train a model for less time when the **training loss**...

As more data you give as more your model will be flexible

#

Even tough you the loss will stay pretty much the same, your model will learn to generalize better

short chasm Jul 19, 2021, 8:01 PM

#

Hello everyone. For example, I need to write a program that will detect the roads in this photo and take actions accordingly. Is opencv library suitable for this job?

whole hamlet Jul 19, 2021, 8:51 PM

#

native bay hey can someone suggest me ideas for projects i really cant find any i know RNN ...

Search for datasets on kaggle, it might help.

fleet hull Jul 19, 2021, 8:53 PM

#

hello

grave frost Jul 19, 2021, 8:55 PM

#

grave breach I don't know... maybe he deserves a second chanche

you are too soft 🙂

silver sun Jul 19, 2021, 9:11 PM

#

Does anyone know how to filter a csv file based on a certain entry in Pandas? I want all the data except for the column Name that has Test User.

ripe forge Jul 19, 2021, 9:16 PM

#

Load it in memory. Then filter.

silver sun Jul 19, 2021, 9:17 PM

#

ripe forge Load it in memory. Then filter.

Could you explain sorry?

ripe forge Jul 19, 2021, 9:22 PM

#

Basically read the csv into a dataframe first. As is.

#

Then your question changes to "how do I use all data except for column name that has Test User from a df"

#

So, that's a simple filter step. Something like df[df[colname]!="Test User"]

#

(ps. I'm on phone. Typing code is a pain. Forgive the spacing)

silver sun Jul 19, 2021, 9:24 PM

#

ripe forge So, that's a simple filter step. Something like `df[df[colname]!="Test User"]`

Thank you so much!

silver sun Jul 19, 2021, 9:42 PM

#

ripe forge (ps. I'm on phone. Typing code is a pain. Forgive the spacing)

One more thing can I delete multiple data this way? Like if I have additional names other than Test User.

ripe forge Jul 19, 2021, 9:43 PM

#

Yep. Think of it in terms of filtering

#

There's an .isin method that let's you compare against multiple names at once.

grave breach Jul 19, 2021, 9:45 PM

#

short chasm Hello everyone. For example, I need to write a program that will detect the road...

The road seems to be of an unique color

#

Use it

#

Or, use an edge detection algorithm

#

But, what I suggest is

#

Use its color to get only the rows, then use the "thinnification" algorithm to get just a list of points

#

And then maybe fit a line on those

magic dune Jul 19, 2021, 11:41 PM

#

# importing the library
import numpy as np
import matplotlib.pyplot as plt
import math

def diff(x,y,cx,cy,txt,list,dict):
    for i in range(len(x)):

        diffrence=math.sqrt((x[i]-cx)**2+(y[i]-cy)**2)

        dict[x[i],y[i]] = diffrence



# 2.23606797749979
#
# data to be plotted
y = np.array([-2,2,4,0])
x = np.array([-2,4,2,0])

k=[-2,1]
k1=[3,1]
print(k)

l1=[]
l2=[]
dictionary2={}
dictionary1={}
# plotting
plt.title("K Means")
plt.xlabel("X axis")

plt.ylabel("Y axis")
c1=plt.scatter(k[0],k[1], color="black")

c2=plt.scatter(k1[0],k1[1], color="black")
plt.scatter(x, y, color='b')

diff(x,y,k[0],k[1],txt="1 ",list=l1,dict=dictionary1)
diff(x,y,k1[0],k1[1],txt="2 ",list=l2,dict=dictionary2)

plt.show()

#

help me

serene scaffold Jul 20, 2021, 12:13 AM

#

magic dune ```py # importing the library import numpy as np import matplotlib.pyplot as plt...

what is wrong

umbral ferry Jul 20, 2021, 12:28 AM

#

did y'all see that machine learning aimbot?

#

any thoughts?

#

literally undetectable by current anti-cheat software

dire echo Jul 20, 2021, 1:02 AM

#

If it act like human then they neef a human mod to detect it

magic dune Jul 20, 2021, 1:40 AM

#

umbral ferry literally undetectable by current anti-cheat software

Good idea

sullen crescent Jul 20, 2021, 2:06 AM

#

I actually built my own aimbot with object detection, tracking inside ROI, and automatically changed my cursor value to follow the objects, I only tested that in old offline fps game though

#

it would be amazing if I can make it works in latest or maybe online fps games but Im not that great at making undetectable anti-cheat program

rigid zodiac Jul 20, 2021, 2:26 AM

#

Hi everyone, I have a quick question. How do you convert between csv file toward json. Thanks a lot

serene scaffold Jul 20, 2021, 2:30 AM

#

rigid zodiac Hi everyone, I have a quick question. How do you convert between csv file toward...

you can load the CSV in pandas and then write it back to file as a json

#

pd.read_csv(...).to_json(...)

rigid zodiac Jul 20, 2021, 2:31 AM

#

let me try

serene scaffold Jul 20, 2021, 2:31 AM

#

Speaking of, how efficient is DataFrame.query as compared to doing the same operations in the main environment?

rigid zodiac Jul 20, 2021, 2:32 AM

#

tbh I havent do query for a while and today I was tasked to do with athena

rigid zodiac Jul 20, 2021, 2:32 AM

#

serene scaffold Speaking of, how efficient is `DataFrame.query` as compared to doing the same op...

I was able to get some data, but the issue is athena only spit out csv somehow

serene scaffold Jul 20, 2021, 2:33 AM

#

rigid zodiac I was able to get some data, but the issue is athena only spit out csv somehow

My question about DataFrame.query was a general question and was not directed specifically at you. It's unrelated to your csv -> json question.

rigid zodiac Jul 20, 2021, 2:33 AM

#

serene scaffold ```py pd.read_csv(...).to_json(...) ```

what should I put inside the to_json( ..... )

serene scaffold Jul 20, 2021, 2:33 AM

#

!docs pandas.DataFrame.to_json

arctic wedgeBOT Jul 20, 2021, 2:33 AM

#

pandas.DataFrame.to\_json

DataFrame.to_json(path_or_buf=None, orient=None, date_format=None, double_precision=10, force_ascii=True, date_unit='ms', default_handler=None, lines=False, compression='infer', ...)```
Convert the object to a JSON string.

Note NaN’s and None will be converted to null and datetime objects will be converted to UNIX timestamps.

rigid zodiac Jul 20, 2021, 2:35 AM

#

thank you so much I'mma try it

serene scaffold Jul 20, 2021, 3:10 AM

#

rigid zodiac thank you so much I'mma try it

how'd it go?

rigid zodiac Jul 20, 2021, 3:12 AM

#

serene scaffold how'd it go?

it quite well, @lapis sequoia has show me other way. My Pc cant handle your way. it crashed 3 times some unknown reason

royal crest Jul 20, 2021, 3:32 AM

#

i'm a bit stumped on ways to approach this.

i'm trying to convert a large number of proprietary survey data to a python-compatible format (namely pandas). At the same time, i want to be able to retrieve each participant's data (such as their responses) I'm thinking about using either inheritance/composite OOP and then somehow converting it to a pandas dataframe -- would you say this is an alright way to go about this?

serene scaffold Jul 20, 2021, 3:49 AM

#

royal crest i'm a bit stumped on ways to approach this. i'm trying to convert a large numbe...

I'm not following. Is this data in a database or something?

royal crest Jul 20, 2021, 3:49 AM

#

not a database, it's thousands of sheets of paper

serene scaffold Jul 20, 2021, 3:49 AM

#

royal crest not a database, it's thousands of sheets of paper

literal sheets of paper that have to be scanned?

royal crest Jul 20, 2021, 3:50 AM

#

some manual entry needed but no free text response

#

no OCR

serene scaffold Jul 20, 2021, 3:50 AM

#

royal crest some manual entry needed but no free text response

so, in what format will the data be? text files (.txt or otherwise) with a predictable format?

royal crest Jul 20, 2021, 3:51 AM

#

simplest would be csv files - index by the participant identifier and the rest of the columns being their responses

serene scaffold Jul 20, 2021, 3:51 AM

#

royal crest simplest would be csv files - index by the participant identifier and the rest o...

if the data is already in a CSV, you are good to go.

royal crest Jul 20, 2021, 3:52 AM

#

sadly no, but i think manual entry as csv might be the quickest

#

sigh

serene scaffold Jul 20, 2021, 3:52 AM

#

royal crest sadly no, but i think manual entry as csv might be the quickest

So again I ask, what format is the data in currently?

royal crest Jul 20, 2021, 3:52 AM

#

none, just sheets of papers with responses

serene scaffold Jul 20, 2021, 3:53 AM

#

So, literal sheets of paper? If you want to manually enter them into excel (or any other common spreadsheet software), you'll be fine. Pandas can open those.

royal crest Jul 20, 2021, 3:54 AM

#

serene scaffold So, literal sheets of paper? If you want to manually enter them into excel (or a...

cheers pensiveBlobThumbsUp

serene scaffold Jul 20, 2021, 4:15 AM

#

@royal crest generally speaking, what was your plan for using "inheritance/composite OOP" for this problem?

#

generally, I think OOP is intended to make large systems more maintainable. Data science stuff tends to be declarative.

royal crest Jul 20, 2021, 4:20 AM

#

serene scaffold <@!220148284168077312> generally speaking, what was your plan for using "inherit...

well since i'm new to OOP thought it'd be good practice. I was thinking about making a base class named Respondent with composite classes for each attributes and their manipulation thereof

#

In hindsight that sounds incredibly inefficient for this particular task

wanton sleet Jul 20, 2021, 7:31 AM

#

Q8. Subtract the mean of each row of the given 2D array, “arr8”, from the values in the array. Set the updated array in “transformed_arr8”.

#

ans : arr8 = np.random.rand(3, 10)

transformed_arr8 = arr8 - arr8.mean(axis=1, keepdims = True)

#

how is this valid?

#

for rows axis = 0 , but the above solution is shown right how ?

#

anyone

hard hound Jul 20, 2021, 7:36 AM

#

Has anyone read Deep learning by Ian Goodfellow ?

mighty patio Jul 20, 2021, 7:39 AM

#

wanton sleet Q8. Subtract the mean of each row of the given 2D array, “arr8”, from the values...

As I understand it, the question asks you for the mean of all columns in that particular row, so axis=1 makes sense to me

wanton sleet Jul 20, 2021, 7:40 AM

#

mighty patio As I understand it, the question asks you for the mean of all columns in that pa...

It has said that " Subtract the mean of each row

#

that means we have to calcute mean of each row and subtract it with the values in array

#

right?

mighty patio Jul 20, 2021, 7:44 AM

#

Ye when i read it again you are right, you should email whomever made the assignment

wispy wolf Jul 20, 2021, 8:00 AM

#

hello, I have a 1GB dataframe of all the projects on pypi.org, their dists (filenames) and their hashes and I'm looking to filter it by any filenames that don't start with their project name: but I'm getting this Traceback: https://gist.github.com/graingert/9a74f1a9d605caa37e078e33ef0e3091#gistcomment-3819689

Gist

gist:9a74f1a9d605caa37e078e33ef0e3091

GitHub Gist: instantly share code, notes, and snippets.

#

>>> df
                                    date           name                              filename                                               hash
3154138        2005-03-21 15:59:25+00:00        ZConfig                   ZConfig-2.2.tar.bz2  d394f3b3574dbe4b9ca564b078d54dede5dbd3c47b8cfd...
739804         2005-03-22 22:19:10+00:00        roundup                  roundup-0.8.2.tar.gz  7e4ea1cce7a1148b5843519134f1722f7d566257143e5d...
802124         2005-03-22 22:48:40+00:00         pygenx                   pygenx-0.5.3.tar.gz  473c10ea014b2da31405acfccc0ae141bf31dc148c2eb4...
739805         2005-03-24 18:52:12+00:00        roundup               roundup-0.8.2.win32.exe  adbefd79254f103dad5385e34707b29f22e47e84fd4df2...
3024342        2005-03-30 16:50:07+00:00   rlcompleter2              rlcompleter2-0.96.tar.gz  a69241a3a21cbca2c1c57870e4aac620841c3f013577aa...
...                                  ...            ...                                   ...                                                ...
3373212 2021-07-19 07:27:31.247181+00:00  proguard-rate  proguard_rate-0.0.6-py3-none-any.whl  4e93ac1ad615dbe123f0f730ffc1b9faa4e252271a4809...
3373213 2021-07-19 07:27:32.341999+00:00  proguard-rate            proguard_rate-0.0.6.tar.gz  bb67428728825bb0cfc6193673658fad81646d0d664ed4...
4205003 2021-07-19 07:28:27.967307+00:00     bridgecrew   bridgecrew-2.0.281-py3-none-any.whl  c96034720fb65d3ec10d4b55b4d1c2d686069578232126...
4205004 2021-07-19 07:28:30.409115+00:00     bridgecrew             bridgecrew-2.0.281.tar.gz  5c35831ee859889e4f8364e4f09fe66bb9696832cd6dcd...
3834333 2021-07-19 07:31:41.385804+00:00    MachineData              MachineData-0.1.2.tar.gz  e3b342bd6c50e511887fe761c4ec4e6c5013f20c85a417...

#

oh that looks terrible on discord

#

traceback here though:

>>> df[~df.filename.str.startswith(df.name)]
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/graingert/.virtualenvs/testing39/lib/python3.9/site-packages/pandas/core/generic.py", line 1529, in __invert__
    new_data = self._mgr.apply(operator.invert)
  File "/home/graingert/.virtualenvs/testing39/lib/python3.9/site-packages/pandas/core/internals/managers.py", line 325, in apply
    applied = b.apply(f, **kwargs)
  File "/home/graingert/.virtualenvs/testing39/lib/python3.9/site-packages/pandas/core/internals/blocks.py", line 382, in apply
    result = func(self.values, **kwargs)
TypeError: ufunc 'invert' not supported for the input types, and the inputs could not be safely coerced to any supported types according to the casting rule ''safe''
>>>

short chasm Jul 20, 2021, 8:09 AM

#

grave breach The road seems to be of an unique color

thank you

desert oar Jul 20, 2021, 8:15 AM

#

@wispy wolf that is... weird

#

The only debugging step i can imagine is to write df['filename'] instead of df.filename

wispy wolf Jul 20, 2021, 8:20 AM

#

@desert oar is there a way to send you a sample of this DF?

#

>>> dfs = df.sample(100)
>>> dfs[~dfs.filename.str.startswith(dfs.name)]
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/graingert/.virtualenvs/testing39/lib/python3.9/site-packages/pandas/core/generic.py", line 1529, in __invert__
    new_data = self._mgr.apply(operator.invert)
  File "/home/graingert/.virtualenvs/testing39/lib/python3.9/site-packages/pandas/core/internals/managers.py", line 325, in apply
    applied = b.apply(f, **kwargs)
  File "/home/graingert/.virtualenvs/testing39/lib/python3.9/site-packages/pandas/core/internals/blocks.py", line 382, in apply
    result = func(self.values, **kwargs)
TypeError: ufunc 'invert' not supported for the input types, and the inputs could not be safely coerced to any supported types according to the casting rule ''safe''
>>>

#

it breaks on a sample

#

I've attached a sample.csv now https://gist.github.com/graingert/9a74f1a9d605caa37e078e33ef0e3091#file-sample-csv

Gist

sample.csv

GitHub Gist: instantly share code, notes, and snippets.