#data-science-and-ml | Python | Page 282

misty flint Feb 1, 2021, 12:40 AM

#

DoggoKek

rotund dagger Feb 1, 2021, 12:41 AM

#

im really not sure i think his point is to get me familiar with using mapping and reduction in pandas, but its throwing me for a loop

#

this is what i get with the max() applied

📎 unknown.png

misty flint Feb 1, 2021, 12:42 AM

#

so his questions is..."for each race, find the states with the highest density" ?

#

confuseddog

rotund dagger Feb 1, 2021, 12:42 AM

#

this is close to what i need it to display

📎 unknown.png

#

this is the exact question he is asking verbatim. Find the State with the highest density of each of the race categories (e.g.Hispanic, White, Black, Native, Asian, Pacific) - (6 answers). Please note that "Puerto Rico" is not a state even though it is in the data.

misty flint Feb 1, 2021, 12:44 AM

#

surely theres census data that just has it by state

#

instead of counties

#

that would make it much easier

rotund dagger Feb 1, 2021, 12:44 AM

#

it only displays buy state if i use groupby(['State'])

#

but he said not to do that. im not sure why

misty flint Feb 1, 2021, 12:45 AM

#

maybe he wants you to calculate the density in your algorithm

#

ID_BoomKek

#

yikes

rotund dagger Feb 1, 2021, 12:46 AM

#

without it it looks like this:

📎 unknown.png

misty flint Feb 1, 2021, 12:46 AM

#

im still confused as to what youre getting returned

#

confuseddog

#

why did you use median

rotund dagger Feb 1, 2021, 12:49 AM

#

without it median it displays the highest density of a county within a state so i would get 100, but alabama doest fully contain 100 percent of a race

#

median takes the average of all counties

misty flint Feb 1, 2021, 12:49 AM

#

ah thats how youre using it

#

but wait

#

median isnt the average technically

#

pithink

rotund dagger Feb 1, 2021, 12:50 AM

#

mean is close in this usage if i use it i get the same answer

misty flint Feb 1, 2021, 12:50 AM

#

why did you use mean()

#

oh i see

#

well its good that its normally distributed

#

DoggoKek

rotund dagger Feb 1, 2021, 12:51 AM

#

so this is almost the perfect answer, its just missing the name of the state

#

for instance, new mexico is the highest hispanic density, of 43.5

#

but it fails to display new mexico

#

it just shows that 43.5 is the highest density

📎 unknown.png

misty flint Feb 1, 2021, 12:52 AM

#

but you cant use group by?

#

weird

#

i would ask someone who knows pandas more than me

rotund dagger Feb 1, 2021, 12:53 AM

#

i would just assign a title in print to each, or use a dictionary to do so, but he said it cant be hard typed. he didnt say i couldnt use groupby he just stated that i would never get the answer to work if i use groupby

misty flint Feb 1, 2021, 12:53 AM

#

your data frame, what are the row names?

#

if its states you can do this https://stackoverflow.com/questions/26640145/how-do-i-get-the-name-of-the-rows-from-the-index-of-a-data-frame

Stack Overflow

How do I get the name of the rows from the index of a data frame?

Consider a data frame with row names that aren't a column of their own per se, such as the following:

    X  Y

Row 1 0 5
Row 2 8 1
Row 3 3 0
How would I extract the name of these row...

#

DoggoKek

rotund dagger Feb 1, 2021, 12:54 AM

#

the row names are just index 0 - 74000

misty flint Feb 1, 2021, 12:54 AM

#

ahhh

rotund dagger Feb 1, 2021, 12:55 AM

#

this is why group by makes sense to me

📎 unknown.png

misty flint Feb 1, 2021, 12:56 AM

#

maybe you can just return it using indexing

rotund dagger Feb 1, 2021, 12:57 AM

#

📎 unknown.png

#

sorry sum is better here than.count

#

makes better sense

📎 unknown.png

#

so i suppose that density = Hispanic/TotalPop for each state

misty flint Feb 1, 2021, 1:00 AM

#

^

#

thats what i would do

#

at least from the beginning

rotund dagger Feb 1, 2021, 1:05 AM

#

even with that info i get lost still lol yikes

misty flint Feb 1, 2021, 1:07 AM

#

@velvet thorn do you understand

#

we are both still noobs

#

DoggoKek

rotund dagger Feb 1, 2021, 1:25 AM

#

well thank you for taking the time to look through it with me @misty flint i appreciate it greatly.

misty flint Feb 1, 2021, 1:44 AM

#

np sad i couldnt help more

#

tbf i just started coding not long ago

#

DoggoKek

#

i think its more of a data science question rather than a pandas question tho

#

but idk

lapis sequoia Feb 1, 2021, 2:29 AM

#

SciPy's solve_ivp documentation and examples use time t as the independent variable such as dy/dt = f(t, y). But as far as I can tell, the solver can be used to solve ODE systems for space/position such as dy/dx = f(x, y). Is this true or is the solver restricted to ODEs in the time domain? Here's a link to Scipy's docs: https://docs.scipy.org/doc/scipy/reference/generated/scipy.integrate.solve_ivp.html

rotund dagger Feb 1, 2021, 2:33 AM

#

im unfamiliar with scipy

misty flint Feb 1, 2021, 2:56 AM

#

the way im interpreting the documentation, it looks like t is fine as long as its a 1 dimensional variable..?

lapis sequoia Feb 1, 2021, 3:25 AM

#

Yes, that's how I see it. You can define the function passed to the solver however you like as long as it meets the requirements of the solver. However, the returned solution object will always be in terms of y and t.

lapis sequoia Feb 1, 2021, 4:04 AM

#

Anyone have experience with matplotlib as a way to plot data?

rotund dagger Feb 1, 2021, 4:48 AM

#

im learning that next, but i have minor experience in it

velvet thorn Feb 1, 2021, 5:10 AM

#

rotund dagger well thank you for taking the time to look through it with me <@!446424248479645...

wups I got distracted

#

have you solved your problem

rotund dagger Feb 1, 2021, 5:10 AM

#

i have not

#

i am trying like mad to though its due in a few hours lol if you could help i would greatly appreciate you

#

@velvet thorn forgot to @ you

velvet thorn Feb 1, 2021, 5:12 AM

#

uh

#

can you

velvet thorn Feb 1, 2021, 5:12 AM

#

rotund dagger <@!171929073063297024> forgot to @ you

go through again what you're trying to do and where you are

rotund dagger Feb 1, 2021, 5:13 AM

#

ill start with the question i am trying to solvee

#

Find the State with the highest density of each of the race categories (e.g.Hispanic, White, Black, Native, Asian, Pacific) - (6 answers). Please note that "Puerto Rico" is not a state even though it is in the data.

#

that is verbatim

velvet thorn Feb 1, 2021, 5:14 AM

#

okay

rotund dagger Feb 1, 2021, 5:14 AM

#

so i am using a csv from kaggle

#

https://www.kaggle.com/muonneutrino/us-census-demographic-data

US Census Demographic Data

Demographic and Economic Data for Tracts and Counties

velvet thorn Feb 1, 2021, 5:14 AM

#

sounds like a groupby problem

rotund dagger Feb 1, 2021, 5:14 AM

#

thats what i thought

#

so this is what i tried so far

#

d = df.groupby(['State'])[['Hispanic','White','Black','Native','Asian','Pacific']].mean()
d

#

and i get

#

a data frame with states as an the index and race for columns with mean values per state

📎 unknown.png

velvet thorn Feb 1, 2021, 5:16 AM

#

okay

#

and then

rotund dagger Feb 1, 2021, 5:16 AM

#

then i need to find which of those states are the highest for each race

#

so i apply max() and get this

#

which is the correct answer but not in the format he is requesting

📎 unknown.png

velvet thorn Feb 1, 2021, 5:17 AM

#

what format do you want it to be in

rotund dagger Feb 1, 2021, 5:17 AM

#

he wants it to read:

#

hispanic: New Mexico

#

so new mexico is 43.5 but when i apply max it is now a series and no longer a dataframe

#

so i lose the state column

#

new mexico is 45.3

velvet thorn Feb 1, 2021, 5:19 AM

#

so basically

#

you want the index

#

to be the race

#

and the value to be the state?

rotund dagger Feb 1, 2021, 5:20 AM

#

yea, but the state has to be calculated not hard coded

velvet thorn Feb 1, 2021, 5:20 AM

#

sec

#

>>> df[df['State'] != 'Puerto Rico'].groupby('State')[['Hispanic', 'White', 'Black', 'Native', 'Asian', 'Pacific']].mean().idxmax()
Hispanic              New Mexico
White                    Vermont
Black       District of Columbia
Native                    Alaska
Asian                     Hawaii
Pacific                   Hawaii

#

like this? @rotund dagger

rotund dagger Feb 1, 2021, 5:24 AM

#

yes exactly

velvet thorn Feb 1, 2021, 5:24 AM

#

ye

#

there you go

rotund dagger Feb 1, 2021, 5:24 AM

#

but with puerto rico and district dropped

#

omg your a life saver

velvet thorn Feb 1, 2021, 5:24 AM

#

change to

rotund dagger Feb 1, 2021, 5:24 AM

#

i see how to drop district in that

velvet thorn Feb 1, 2021, 5:25 AM

#

df[~df['State'].isin({'Puerto Rico', 'District of Columbia'})]

rotund dagger Feb 1, 2021, 5:25 AM

#

ive been working on this for 3 days you are absolutely amazing!

#

hopefully i can cruise through the rest of the questions now

velvet thorn Feb 1, 2021, 5:26 AM

#

yw 👋

misty flint Feb 1, 2021, 6:49 AM

#

wow a real superhero

#

MHXwoah

rotund dagger Feb 1, 2021, 6:49 AM

#

yea, absolutely slayed it

misty flint Feb 1, 2021, 6:49 AM

#

you were so close to

#

ID_BoomKek

rotund dagger Feb 1, 2021, 6:50 AM

#

my professor totally tried to throw me off that path too

steel zealot Feb 1, 2021, 9:47 AM

#

my minds blown

#

i found somthing out

velvet thorn Feb 1, 2021, 9:57 AM

#

rotund dagger my professor totally tried to throw me off that path too

hey

#

so I was

#

just thinking about your problem and

#

I feel like the statistical methodology is wrong?

#

because

#

what you're doing

#

is taking the mean of the percentages of each race, right?

#

but that doesn't necessarily represent the percentage of each race for that state

#

because each entry may have a different total population

#

do you get what I mean?

tall trail Feb 1, 2021, 10:11 AM

#

so im trying to filter out a row in my dataframe between 2 minutes ( my dataframe has this kind of timestamp: 2021-01-31 15:46:33 ) and i can not understand how pandas between_time works.
Right now i have this:
peaks = peaks[peaks['time'].between_time('00:30', '00:32')]
which gives me the following error:
TypeError: Index must be DatetimeIndex

if i run df.dtypes it returns the column as datetime64[ns]

what am i doing wrong/do i need to supply more info?

last rivet Feb 1, 2021, 10:13 AM

#

datetime64 !== DatetimeIndex
Do a conversion

tall trail Feb 1, 2021, 10:13 AM

#

does the column datetime need to be the index aswell?

last rivet Feb 1, 2021, 10:14 AM

#

It's a type error, you need to fix the type so pandas recognize it

#

e.g convert ur datetime64 to normal DateTime by calling .tolist() and I think then Pandas should recognize it

tall trail Feb 1, 2021, 10:16 AM

#

thanks, ill try that

velvet thorn Feb 1, 2021, 10:44 AM

#

tall trail so im trying to filter out a row in my dataframe between 2 minutes ( my datafram...

no

#

between_time implicitly filters on the index

#

so you want to set_index first

tall trail Feb 1, 2021, 10:45 AM

#

ya i figured, did this now df['time'] = pd.DatetimeIndex(df['time'])
df.set_index(keys='time', inplace=True, drop=False)

lapis sequoia Feb 1, 2021, 1:02 PM

#

plss help in this... I am a begginer and new here... so cant access the voice ... plsss help me in this... and have to give a voice message coz... the problem was long to write

📎 Rev.mp3

atomic obsidian Feb 1, 2021, 1:56 PM

#

is pandas a good library to begin learning data science?

rigid ledge Feb 1, 2021, 2:13 PM

#

hello guys

#

can anyone plz help me in installing yolov4 on ubuntu VM

#

?

#

any tutorial is appreciated thx

ripe forge Feb 1, 2021, 3:50 PM

#

Yes, check this repo. https://github.com/qfgaohao/pythorch-ssd read near the end of readme

#

It has the option of fine tuning too, so make sure to read the params and change it as needed if you want training from scratch. (ps. I don't recommend training from scratch)

supple minnow Feb 1, 2021, 3:54 PM

#

Hello all,
I have a question based on feature selection. Based on this picture(mutual info) what is the best approach when we need to decide what feature we gonna take and which we gonna drop? Like is it ok if I take the first 7(including age) features or should I just take the first two since they have better results?

📎 mi.png

ripe forge Feb 1, 2021, 4:20 PM

#

Don't choose the number of features directly from plot. You can instead decide on some threshold, say cumulative 0.9 or cutoff 0.005

#

Then take whatever n you get from that approach

#

Note that while first two features seem to be clearly stronger, it doesn't automatically make other features bad. There's still information there.

crisp gazelle Feb 1, 2021, 5:18 PM

#

Would any of you guys know a way to visualize neural networks like with a library similar to sns ? I have seen ann_visualizer, but I am working on my own neural net without using Keras so i am not sure if that will work

undone kelp Feb 1, 2021, 5:37 PM

#

you want to get the nodes and weightings as a visualisation?

#

https://networkx.org/documentation/stable/auto_examples/drawing/plot_multipartite_graph.html#sphx-glr-auto-examples-drawing-plot-multipartite-graph-py

ripe forge Feb 1, 2021, 5:59 PM

#

  File "/mnt/disks/sdb/superai/ai/objectrecognition/vision/datasets/open_images.py", line 17, in __init__
    self.data, self.class_names, self.class_dict = self._read_data()
  File "/mnt/disks/sdb/superai/ai/objectrecognition/vision/datasets/open_images.py", line 63, in _read_data
    class_names = ['BACKGROUND'] + sorted(list(annotations['ClassName'].unique()))
``` focus on this part of the traceback. this is giving you a clue about where to look

#

see if you can read the code in that place to figure out why this error is happening

#

My first guess would be it expects the class names to be provided in a certain format, yeah

dusty anchor Feb 1, 2021, 6:00 PM

#

ehy guys can i ask here for tensorflow/keras questions?

ripe forge Feb 1, 2021, 6:00 PM

#

yep, you can

dusty anchor Feb 1, 2021, 6:03 PM

#

so ive a few questions, first, im working on a image segmentation project for the first time using the cityscape dataset, ive 2 folders one containing the images, and one containing the masks, ive made a function to create a list containing the path of all the images, can i convert these lists in a keras dataset?

delicate crane Feb 1, 2021, 6:25 PM

#

Can I get some ideas for a project using machine learning

lapis sequoia Feb 1, 2021, 6:44 PM

#

For a project to practice you can use the covid data sets. I used the data set as well for my data science lecture class. The source code of the lecture is free on GITHUB: https://github.com/kienlef/Lecture_Covid_19_data_analysis

GitHub

kienlef/Lecture_Covid_19_data_analysis

Content Material for the lecture Covid-19 data analysis - kienlef/Lecture_Covid_19_data_analysis

#

Python Projects: How to Build a Simple Trading Bot Skeleton in Python | Episode 1 by Third Eye Cyborg Podcast • A podcast on Anchor. This podcast episode goes into the code of a basic Python project. Let me know what you think, I am always open to feedback! https://anchor.fm/thirdeyecyborg/episodes/Python-Projects-How-to-Build-a-Simple-Trading-Bot-Skeleton-in-Python--Episode-1-epplkg

Anchor FM Inc.

Python Projects: How to Build a Simple Trading Bot Skeleton in Python | Episode 1 by Third Eye Cyborg Podcast

Python Projects: How to Build a Simple Trading Bot Skeleton in Pyth...

I will be using the knowledge that is covered in the Python Basics Series to conduct several Python Projects in the Episodes of this Podcast. In this episode I will be going over building a basic trading bot skeleton in the Python Programming Language. I also plan to go into other programming languages and technologies in future episodes.
Check ...

atomic obsidian Feb 1, 2021, 6:53 PM

#

should i start learning pandas or mysql first

cerulean spindle Feb 1, 2021, 7:17 PM

#

atomic obsidian should i start learning pandas or mysql first

I'd say pandas

loud osprey Feb 1, 2021, 7:31 PM

#

hello, can anyone recommend me good online visualization tools which can fetch data from an api

quick veldt Feb 1, 2021, 7:49 PM

#

Hey, I'm currently collecting some data from Twitter Streaming API and I need to run the script all the time. What are my options in terms of free script hosting?

cerulean spindle Feb 1, 2021, 11:43 PM

#

Does anyone how a sub-par GPU affects tensorflow training?

austere swift Feb 2, 2021, 2:06 AM

#

theres really 2 main things about the gpu you need to worry about

#

first is vram

#

if you have a very low amount of vram then some larger models won't be able to train

#

or you'd have to lower the parameters of the model or the batch size to get it to fit

#

second is the gpus actual speed

#

this, unlike the vram, wouldn't completely stop you from training the models

#

it would just make it slower/faster

carmine finch Feb 2, 2021, 2:38 AM

#

hey does anybody know python well i need help

austere swift Feb 2, 2021, 3:09 AM

#

this is a python server

#

a lot of people know python well

misty flint Feb 2, 2021, 3:24 AM

#

dont ask to ask meme

#

DoggoKek

foggy fern Feb 2, 2021, 5:53 AM

#

Hi I'm having trouble with some data visualizing I'm not seeing the whole behavior of contours

#

I'm just getting a chunk of it

misty flint Feb 2, 2021, 5:54 AM

#

i understand that feeling

#

ID_BoomKek

foggy fern Feb 2, 2021, 5:54 AM

#

do you know how to resolve it?

misty flint Feb 2, 2021, 5:54 AM

#

matplotlib?

foggy fern Feb 2, 2021, 5:54 AM

#

yeah

misty flint Feb 2, 2021, 5:54 AM

#

well depends

#

whats your code

foggy fern Feb 2, 2021, 5:55 AM

#

phi_0 = np.linspace(0, 0,400)
phi_1 = np.linspace(0, 0,400)
m= np.linspace(-5, 0,400)
w= np.linspace(-.0002,.0008,400)
[M,W] =np.meshgrid(m,w)
z0 = 0
zf =1000
N = 400 # Number of Runge-Kutta steps
h = (zf - z0)/N
def f(p, z ):
x = p[0]
U = p[1]
dx = U
dU= -(2./(1.+z)+(3..03/(2.(1.+z)**2.(.3(1.+z)*3.+.7))))p[1] +2.w.3np.exp(-2.x)((1.+z)/(.3(1.+z)**3.+.7))-(np.power(10.,m))x/((z+1.)**2.((1.+z)**3.+.7))

return array([dx, dU], float)

zpoints = arange(z0, zf, h)
ypoints = []
vpoints = []
p = array([phi_0, phi_1], float)
for z in zpoints:
ypoints.append(p[0])
vpoints.append(p[1])
k1 = h * f(p, z)
k2 = h * f(p + 0.5k1, z + 0.5h)
k3 = h * f(p + 0.5k2, z + 0.5h)
k4 = h * f(p + k3, z + h)
p = p + (k1 + 2k2 + 2k3 + k4)/6
z0q=1000 #interesting redshift values in QSO data(initial)
zfq=1100 #interesting redshift values in QSO data(final)
i = (zfq - z0q)/N #step size
def f(q, z ):
x = q[0]
U = q[1]
dx = U
dU= -(2./(1.+z)+(3..03/(2.(1.+z)**2.(.3(1.+z)*3.+.7))))p[1] +2.w.3np.exp(-2.x)((1.+z)/(.3(1.+z)**3.+.7))-(np.power(10.,m))x/((z+1.)**2.((1.+z)**3.+.7))

return array([dx, dU], float)

zqpoints = arange(z0q, zfq, i)
xqpoints = []
vqpoints = []
q = array([ypoints[399], phi_1], float)

for z in zqpoints:
xqpoints.append(q[0])
vqpoints.append(q[1])
k1 = h * f(q, z)
k2 = h * f(q + 0.5k1, z + 0.5h)
k3 = h * f(q + 0.5k2, z + 0.5h)
k4 = h * f(q + k3, z + h)
q = q + (k1 + 2k2 + 2k3 + k4)/6

misty flint Feb 2, 2021, 5:55 AM

#

oh no can you put in in markdown

#

nvm

#

phi_0 = np.linspace(0, 0,400)
phi_1 = np.linspace(0, 0,400)
m= np.linspace(-5, 0,400)
w= np.linspace(-.0002,.0008,400)
[M,W] =np.meshgrid(m,w)
z0 = 0
zf =1000
N = 400        # Number of Runge-Kutta steps
h = (zf - z0)/N 
def f(p, z ):
    x = p[0]
    U = p[1]
    dx = U
    dU= -(2./(1.+z)+(3..03/(2.(1.+z)2.(.3(1.+z)3.+.7))))p[1] +2.w.3np.exp(-2.x)((1.+z)/(.3(1.+z)**3.+.7))-(np.power(10.,m))x/((z+1.)2.*((1.+z)3.+.7))

    return array([dx, dU], float)
zpoints = arange(z0, zf, h)
ypoints = []
vpoints = []
p = array([phi_0, phi_1], float)
for z in zpoints:
    ypoints.append(p[0])
    vpoints.append(p[1])
    k1 = h * f(p, z)
    k2 = h * f(p + 0.5k1, z + 0.5h)
    k3 = h * f(p + 0.5k2, z + 0.5h)
    k4 = h * f(p + k3, z + h)
    p = p + (k1 + 2k2 + 2k3 + k4)/6
z0q=1000    #interesting redshift values in QSO data(initial)
zfq=1100   #interesting redshift values in QSO data(final)
i = (zfq - z0q)/N #step size
def f(q, z ):
    x = q[0]
    U = q[1]
    dx = U
    dU= -(2./(1.+z)+(3..03/(2.(1.+z)2.(.3(1.+z)3.+.7))))p[1] +2.w.3np.exp(-2.x)((1.+z)/(.3(1.+z)**3.+.7))-(np.power(10.,m))x/((z+1.)2.*((1.+z)3.+.7))

    return array([dx, dU], float)
zqpoints = arange(z0q, zfq, i)
xqpoints = []
vqpoints = []
q = array([ypoints[399], phi_1], float)

for z in zqpoints:
    xqpoints.append(q[0])
    vqpoints.append(q[1])
    k1 = h * f(q, z)
    k2 = h * f(q + 0.5k1, z + 0.5h)
    k3 = h * f(q + 0.5k2, z + 0.5h)
    k4 = h * f(q + k3, z + h)
    q = q + (k1 + 2k2 + 2k3 + k4)/6

#

!e

arctic wedgeBOT Feb 2, 2021, 5:56 AM

#

You are not allowed to use that command here. Please use the #bot-commands channel instead.

misty flint Feb 2, 2021, 5:56 AM

#

ah dang it

#

ill just pull up a notebook rq

#

what were the libraries you used

#

numpy

#

matplotlib

foggy fern Feb 2, 2021, 5:59 AM

#

import numpy as np
from scipy.integrate import odeint
import matplotlib.pyplot as plt
%matplotlib inline
from numpy import array, arange

misty flint Feb 2, 2021, 5:59 AM

#

thanks

#

its giving me an invalid syntax

foggy fern Feb 2, 2021, 6:00 AM

#

plt.rcParams["font.family"] = "serif"
fig, ax = plt.subplots()
level = [ -.0026, -0.0021,-.0016,-.0011,-.0005]
levels = [ -0.0000875,-0.0000701,-0.0000576,-0.0000450, -0.0000285]
plt.contour(W,M,xqpoints,10, cmap='jet');
#CS=ax.contour(Q,P,xpoints, levels, colors='black')
plt.colorbar()
#CS=ax.contour(W,M,xqpoints, level, colors='green')
#plt.ylim([-5,-2])
#ax.set_ylabel("$ α_{had,0} $")
#ax.set_xlabel("$φ'_{0}$")
#ax.clabel(CS, inline=1, fmt='%1.9f')
#ax.yaxis.grid(True, zorder=0)
#ax.xaxis.grid(True, zorder=0)
plt.show()

#

this is what I'm doing for the contour

misty flint Feb 2, 2021, 6:02 AM

#

dU= -(2./(1.+z)+(3..03/(2.(1.+z)2.(.3(1.+z)3.+.7))))p[1] +2.w.3np.exp(-2.x)((1.+z)/(.3(1.+z)**3.+.7))-(np.power(10.,m))x/((z+1.)2.*((1.+z)3.+.7))```

#

this line

#

theres problems with the parentheses

foggy fern Feb 2, 2021, 6:04 AM

#

is there?

#

i dont get any errors

#

i mean ((1.+z)3.+.7)) has 2 parenthesis

misty flint Feb 2, 2021, 6:04 AM

#

the code look the same to you?

#

or is it different

foggy fern Feb 2, 2021, 6:04 AM

#

but this doesnt change anything

#

same

misty flint Feb 2, 2021, 6:05 AM

#

ah 2 instead of 4?

#

its not liking the 4

foggy fern Feb 2, 2021, 6:05 AM

#

📎 Screen_Shot_2021-02-02_at_1.05.23_AM.png

#

it works fine for me

misty flint Feb 2, 2021, 6:06 AM

#

weird

#

might just be me then

foggy fern Feb 2, 2021, 6:06 AM

#

 dU= -(2./(1.+z)+(3.*.03/(2.*(1.+z)**2.*(.3*(1.+z)**3.+.7))))*p[1] +2.*w*.3*np.exp(-2.*x)*((1.+z)/(.3*(1.+z)**3.+.7))-(np.power(10.,m))*x/((z+1.)**2.*((1.+z)**3.+.7))

#

copy paste it directly maybe?

misty flint Feb 2, 2021, 6:07 AM

#

oh yeah there is some difference

#

you raised to the power of two at one part

#

i dont have that in the original code

foggy fern Feb 2, 2021, 6:09 AM

#

phi_0 = np.linspace(0, 0,400)
phi_1 = np.linspace(0, 0,400)
m= np.linspace(-5, 0,400)
w= np.linspace(-.0002,.0008,400)
[M,W] =np.meshgrid(m,w)
z0 = 0           
zf =1000          
N = 400        # Number of Runge-Kutta steps
h = (zf - z0)/N 
def f(p, z ):
    x = p[0]
    U = p[1]
    dx = U  
    dU= -(2./(1.+z)+(3.*.03/(2.*(1.+z)**2.*(.3*(1.+z)**3.+.7))))*p[1] +2.*w*.3*np.exp(-2.*x)*((1.+z)/(.3*(1.+z)**3.+.7))-(np.power(10.,m))*x/((z+1.)**2.*((1.+z)**3.+.7))
     
    return array([dx, dU], float)
zpoints = arange(z0, zf, h)
ypoints = []
vpoints = []
p = array([phi_0, phi_1], float)
for z in zpoints:
    ypoints.append(p[0])
    vpoints.append(p[1])
    k1 = h * f(p, z)
    k2 = h * f(p + 0.5*k1, z + 0.5*h)
    k3 = h * f(p + 0.5*k2, z + 0.5*h)
    k4 = h * f(p + k3, z + h)
    p = p + (k1 + 2*k2 + 2*k3 + k4)/6
z0q=1000    #interesting redshift values in QSO data(initial)
zfq=1100   #interesting redshift values in QSO data(final)
i = (zfq - z0q)/N #step size
def f(q, z ):
    x = q[0]
    U = q[1]
    dx = U     
    dU= -(2./(1.+z)+(3.*.03/(2.*(1.+z)**2.*(.3*(1.+z)**3.+.7))))*p[1] +2.*w*.3*np.exp(-2.*x)*((1.+z)/(.3*(1.+z)**3.+.7))-(np.power(10.,m))*x/((z+1.)**2.*((1.+z)**3.+.7))
     
    return array([dx, dU], float)
zqpoints = arange(z0q, zfq, i)
xqpoints = []
vqpoints = []
q = array([ypoints[399], phi_1], float)

for z in zqpoints:
    xqpoints.append(q[0])
    vqpoints.append(q[1])
    k1 = h * f(q, z)
    k2 = h * f(q + 0.5*k1, z + 0.5*h)
    k3 = h * f(q + 0.5*k2, z + 0.5*h)
    k4 = h * f(q + k3, z + h)
    q = q + (k1 + 2*k2 + 2*k3 + k4)/6

#

this is the code

misty flint Feb 2, 2021, 6:09 AM

#

oh yeah its dif than the previous one

#

keeps giving me errors

#

like its not multiplying the k1 and k2 in the previous code

#

let me use this one

foggy fern Feb 2, 2021, 6:09 AM

#

foggy fern phi_0 = np.linspace(0, 0,400) phi_1 = np.linspace(0, 0,400) m= np.linspace(-5, 0...

i think when you copied from this it got messed up

misty flint Feb 2, 2021, 6:10 AM

#

yeah

#

📎 gcW4uh1xEVNEREREAREXMUVEREQUEJHcIyIiIgqISO4RERERBUQk94iIiIgCIpJ7RERERAERyT0iIiKigIjkHhEREVFARHKPiIiI.png

#

this what you got?

#

DoggoKek

foggy fern Feb 2, 2021, 6:11 AM

#

yeah

#

i want to see their full behavior

#

like the whole contours

misty flint Feb 2, 2021, 6:13 AM

#

i think the key here is changing what pass in in these lines:

#

zpoints = arange(z0, zf, h)

#

zqpoints = arange(z0q, zfq, i)

#

i believe

#

i could be wrong tho

#

so let me try

foggy fern Feb 2, 2021, 6:13 AM

#

those are the initial and ending values though

#

for where I'm trying to calculate ode

misty flint Feb 2, 2021, 6:14 AM

#

pithink

#

oh

#

well then i guess its just changing the linspace then

#

no?

foggy fern Feb 2, 2021, 6:15 AM

#

well if i do huge linspace then i can't see my paramters well enough

misty flint Feb 2, 2021, 6:16 AM

#

thats a problem

#

bc i cant think of a better solution

foggy fern Feb 2, 2021, 6:18 AM

#

were you able to make the full contours anyhow though

misty flint Feb 2, 2021, 6:18 AM

#

kept giving me errors

#

ID_BoomKek

foggy fern Feb 2, 2021, 6:18 AM

#

what error?

misty flint Feb 2, 2021, 6:19 AM

#

oh wait

#

this looks interesting

#

📎 FPjIfJAAAAABJRU5ErkJggg.png

#

woah

#

📎 efRl4NfBwUvdRxtd1Hl5G3XfxrJrjqvrpNK4r79ggJsMwDAexQUyGYRgOYuJuGIbhICbuhmEYDmLibhiG4SAm7oZhGA5i4m4YhuE.png

foggy fern Feb 2, 2021, 6:19 AM

#

it does look interesting

misty flint Feb 2, 2021, 6:20 AM

#

the problem here is if you increase the linspace parameters too much you get exponent overflow error

#

i think theres a mathematical solution instead

foggy fern Feb 2, 2021, 6:20 AM

#

yeah mass is a power 10

misty flint Feb 2, 2021, 6:20 AM

#

maybe using log or something

#

and then youll visualize it that way instead

#

how much can you change the equation

#

i would start there maybe

foggy fern Feb 2, 2021, 6:21 AM

#

what do you mean changing the equation

misty flint Feb 2, 2021, 6:22 AM

#

what is dU here

#

or what does it represent

foggy fern Feb 2, 2021, 6:22 AM

#

second derivative

misty flint Feb 2, 2021, 6:22 AM

#

second derivative of what

#

of just U?

foggy fern Feb 2, 2021, 6:22 AM

#

2nd derivative of x

misty flint Feb 2, 2021, 6:22 AM

#

oh im dumb

#

lol

foggy fern Feb 2, 2021, 6:23 AM

#

first derivative of U

#

no you're good

misty flint Feb 2, 2021, 6:23 AM

#

have you messed at all with the delta?

foggy fern Feb 2, 2021, 6:24 AM

#

which delta?

misty flint Feb 2, 2021, 6:24 AM

#

ah nvm it didnt work

#

DoggoKek

#

if only gm was here

#

they would know what to do

#

DoggoKek

foggy fern Feb 2, 2021, 6:25 AM

#

what's/who's gm

misty flint Feb 2, 2021, 6:25 AM

#

@velvet thorn

#

can you help us please if you are free

#

shiroGomen

#

wild

📎 AcibPnY3qeJcAAAAAElFTkSuQmCC.png

foggy fern Feb 2, 2021, 6:27 AM

#

whoa!!!!

#

there's a dip!

#

i didn't expect this

misty flint Feb 2, 2021, 6:28 AM

#

phi_0 = np.linspace(0, 0,400)
phi_1 = np.linspace(0, 0,400)
m= np.linspace(-10, 3,400)
w= np.linspace(-.0002,.0008,400)

#

dunno if its actually supposed to be there or if its matplotlib

#

you can try it

foggy fern Feb 2, 2021, 6:30 AM

#

this is helpful thanks i still want to see the full behavior though 😦

misty flint Feb 2, 2021, 6:30 AM

#

same

#

sorry bud

#

i am still noob

#

DoggoKek

foggy fern Feb 2, 2021, 6:31 AM

#

no worries thanks for your help

misty flint Feb 2, 2021, 6:31 AM

#

ngl i thought your issue was going to be easier

#

like mine 2 weeks ago

#

DoggoKek

#

np

velvet thorn Feb 2, 2021, 6:31 AM

#

misty flint can you help us please if you are free

you should not

#

tag specific people

#

for help

misty flint Feb 2, 2021, 6:31 AM

#

were all struggling here

#

sorry

#

RunFail

lapis sequoia Feb 2, 2021, 6:45 AM

#

Write a short Python program which, given an array of integers, a, calculates an array of the same length, p, in which p[i] is the product of all the integers in a except a[i].

#

Can someone help me with a question?

velvet thorn Feb 2, 2021, 6:47 AM

#

foggy fern no worries thanks for your help

what's your problem

lapis sequoia Feb 2, 2021, 6:47 AM

#

Write a short Python program which, given an array of integers, a, calculates an array of the same length, p, in which p[i] is the product of all the integers in a except a[i].

foggy fern Feb 2, 2021, 6:50 AM

#

velvet thorn what's your problem

I can't visualize the whole contour

#

I'm getting it partially

velvet thorn Feb 2, 2021, 6:51 AM

#

foggy fern I can't visualize the whole contour

like

#

there's stuff

#

outside the bounds of the box

#

that you want to see?

foggy fern Feb 2, 2021, 6:51 AM

#

yes

velvet thorn Feb 2, 2021, 6:51 AM

#

honestly there's way too much code + discussion there than I care to wade through

#

but

#

a good start would be

#

ax.axis, which lets you set the viewport bounds

#

if your range is very big

#

I would suggest

#

a different scale

#

log scale, in particular

foggy fern Feb 2, 2021, 6:54 AM

#

the problem with that is I want to see the change in parameter in really small scales like order of 10^-5 if i do log scale it would just be -3,-4,-5...

#

but i want to see for values like .00005 and .00008 because I want to compare it with another set of data

velvet thorn Feb 2, 2021, 6:59 AM

#

foggy fern the problem with that is I want to see the change in parameter in really small s...

think

#

you might want to visualise subsets

#

use axis to move the viewport

#

to parts you want to focus on

#

sounds too big otherwise

lapis sequoia Feb 2, 2021, 8:19 AM

#

I have a weird question

#

Do you guys fully memorize plot methods or you do it copy pasta from google

tall trail Feb 2, 2021, 8:21 AM

#

mostly google here

fleet heath Feb 2, 2021, 8:27 AM

#

lapis sequoia Do you guys fully memorize plot methods or you do it copy pasta from google

You don't need to memorize anything...it comes with practice

#

And still if you're stuck somewhere, google is always there

lapis sequoia Feb 2, 2021, 8:39 AM

#

I mean, do professionals do like this too?

#

like, google most of the time

tall trail Feb 2, 2021, 8:40 AM

#

im doing an internship right now, here they mostly document their google findings and present them to each other to teach everyone if its usefull

lapis sequoia Feb 2, 2021, 8:46 AM

#

Wow

#

Thats impressive

#

What do you do as an intern

tall trail Feb 2, 2021, 9:06 AM

#

i need to build a dashboard which visualizes analysed data that suits the company application landscape

#

and i need to do the analyzing part myself too

#

and write a massive report about it too

lapis sequoia Feb 2, 2021, 9:12 AM

#

I envy you

dusty anchor Feb 2, 2021, 9:43 AM

#

hey guys if i use the tf.data.Dataset.list_files can i specify to import only images that have a particular word in the name?

velvet thorn Feb 2, 2021, 9:56 AM

#

lapis sequoia like, google most of the time

I'd say what you search for changes

#

like as you get better

#

usually you know what you want to do, just not how to do it in a specific context

#

so for example

#

for a certain problem, a beginner might search "how to find text with dynamic prefix"

#

which is something you can do with a regex

#

and you might just have forgotten the syntax

#

so you might instead look for "python lookbehind regex"

#

experience also helps you know what to search for

#

asking the right question is very important

ripe forge Feb 2, 2021, 9:58 AM

#

I personally mostly Google things as well, there's too many other things to think about rather than worrying about memorizing specific syntax or parameters of an api call

velvet thorn Feb 2, 2021, 9:58 AM

#

over time, when presented with new problems, even if you don't know exactly how to solve it, you'll have a sense of what kind of approach is more likely to work

#

there is stuff that can be looked up easily (what a parameter is called) and stuff that cannot (how to reduce a set of complex business requirements to a viable technical architecture).

#

you want to be the kind of person who is good @ the latter.

#

to draw one final analogy: the best Scrabble players are not the best writers.

lapis sequoia Feb 2, 2021, 10:38 AM

#

velvet thorn to draw one final analogy: the best Scrabble players are not the best writers.

Thanks for your amazing input

#

I agree

lapis sequoia Feb 2, 2021, 1:25 PM

#

4 for loops and it takes forever to execute lol

📎 4f.png

lapis sequoia Feb 2, 2021, 1:48 PM

#

so it's very slow, right?

#

hmm

fleet heath Feb 2, 2021, 1:50 PM

#

Why would it be ^?

#

Yepp

lapis sequoia Feb 2, 2021, 1:52 PM

#

well I just need to split it into country wise then

#

it takes forever

#

iterating over 1.75mil is ridiculous?

hasty grail Feb 2, 2021, 1:58 PM

#

You can take the line country_city_list = df['City'].unique() out of the loop

#

Also, use df.loc instead of df when boolean masking, that way you avoid making copies of the dataframe

lapis sequoia Feb 2, 2021, 2:04 PM

#

Thanks guys

#

I'll try changing it

#

seems like data are too large to do the 4 for loops

sullen hull Feb 2, 2021, 2:26 PM

#

figure = plt.Figure(figsize=(0.5 * res_width / 100, 0.75 * res_height / 100), facecolor="#67676b")
figure.add_subplot(fc="#15151c").plot(df["Close time"], df["Close"].astype(float), "-" + colour)
figure.add_subplot(fc="#15151c").plot(df["Close time"], moving_avg, "-w")

error: MatplotlibDeprecationWarning: Adding an axes using the same arguments as a previous axes currently reuses the earlier instance. In a future version, a new instance will always be created and returned. Meanwhile, this warning can be suppressed, and the future behavior ensured, by passing a unique label to each axes instance.
figure.add_subplot(fc="#15151c").plot(df["Close time"], moving_avg, "-w")

#

How do I properly add another line to my figure

#

the graph comes out correctly but I wish to remove the deprecation warning

📎 unknown.png

glad night Feb 2, 2021, 2:29 PM

#

Hi guys! I have a question! I have a dataset that contains unique orders in it for a number of accounts. So the first column in each row is account number, and the rest are info on a specific order for that account. Hence, any account may have a number of rows for different things they have ordered - all of them containing the unique account number. I would like to pull a list of unique account numbers that have NOT ordered any from a short list of items, which I would identify by keyword... Any ideas? You're saving a life here. Thank you, hope you're keeping safe

#

PS. Apologies if this is the wrong space for this query...

serene scaffold Feb 2, 2021, 2:53 PM

#

glad night Hi guys! I have a question! I have a dataset that contains unique orders in it f...

How is the dataset stored? (Please ping to reply for every message directed at me, even if you think I'm here.)

glad night Feb 2, 2021, 2:54 PM

#

@serene scaffold appreciate you getting in touch Stelercus. Dataset stored in csv/xls.

serene scaffold Feb 2, 2021, 2:54 PM

#

glad night <@!253696366952316929> appreciate you getting in touch Stelercus. Dataset stored...

can you show an example of the first few rows of the csv?

glad night Feb 2, 2021, 3:05 PM

#

@serene scaffold Happy to. So I am attaching an example and will talk a bit about it

bleak fox Feb 2, 2021, 3:05 PM

#

glad night Hi guys! I have a question! I have a dataset that contains unique orders in it f...

Hi, please share data sample,

glad night Feb 2, 2021, 3:05 PM

#

@serene scaffold

📎 unknown.png

serene scaffold Feb 2, 2021, 3:05 PM

#

@glad night please share it as text so that I can use it in a program.

#

namely as a csv

#

For future reference, text is the best format for anything pertaining to a question. Anything you can share as text and not as a screenshot, do that.

glad night Feb 2, 2021, 3:06 PM

#

@serene scaffold @bleak fox Of course guys, give me a second. Just to comment on this - the idea is that if I run whatever code we come up with successfully, my 'exclusion criteria' would be the keywords 'voucher' and 'coupon', and in this specific case the one that does not contain them is AB000005

#

@serene scaffold Apologies - Discord turned it into a png! Attaching now

#

@serene scaffold @bleak fox

📎 Example_-_greekthenick_-_Sheet1.csv

serene scaffold Feb 2, 2021, 3:08 PM

#

glad night <@!253696366952316929> <@!731198409838690355> Of course guys, give me a second. ...

!code

arctic wedgeBOT Feb 2, 2021, 3:08 PM

#

Here's how to format Python code on Discord:

```py
print('Hello world!')
```

These are backticks, not quotes. Check this out if you can't find the backtick key.

serene scaffold Feb 2, 2021, 3:08 PM

#

for future reference, share text as text in the chat. However I'll download it this time around.

glad night Feb 2, 2021, 3:09 PM

#

@serene scaffold Thanks man I will.

serene scaffold Feb 2, 2021, 3:09 PM

#

glad night <@!253696366952316929> Thanks man I will.

Do you have pandas installed?

glad night Feb 2, 2021, 3:09 PM

#

@serene scaffold Yes!

serene scaffold Feb 2, 2021, 3:12 PM

#

glad night Hi guys! I have a question! I have a dataset that contains unique orders in it f...

So for this dataframe, "unique customer number" is the index. And you'd like to get every index for which "coupon" and "vounter" are NOT a substring of the value in "product name", yes?

glad night Feb 2, 2021, 3:13 PM

#

@serene scaffold Exactly that.

serene scaffold Feb 2, 2021, 3:15 PM

#

glad night <@!253696366952316929> Exactly that.

>>> df['Product Name'].str.contains('coupon')
Unique customer number
AB00001    False
AB00001    False
AB00001    False
AB00001    False
AB00001    False
AB00003    False
AB00003     True
AB00003    False
AB00005    False
AB00005    False
AB00005    False

#

this is part of the solution

#

@glad night the things you need to learn are how to use pd.Series.str.contains, how to do boolean logic with pandas, and how to select from a dataframe using a boolean dataframe.

#

!docs pandas.Series.str.contains

arctic wedgeBOT Feb 2, 2021, 3:19 PM

#

`pandas.Series.str.contains`

Series.str.contains(pat, case=True, flags=0, na=None, regex=True)```
Test if pattern or regex is contained within a string of a Series or Index.

Return boolean Series or Index based on whether a given pattern or regex is contained within a string of a Series or Index.

Parameters  **pat**strCharacter sequence or regular expression.

**case**bool, default TrueIf True, case sensitive.

**flags**int, default 0 (no flags)Flags to pass through to the re module, e.g. re.IGNORECASE.

**na**scalar, optionalFill value for missing values. The default depends on dtype of the array. For object-dtype, `numpy.nan` is used. For `StringDtype`, `pandas.NA` is used.

**regex**bool, default TrueIf True, assumes the pat is a regular expression.

If False, treats the pat as a literal string.

Returns  Series or Index of boolean valuesA Series or Index of boolean values indicating whether the given pattern is contained within the string of each element of the Series or Index.

See also... [read more](https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.Series.str.contains.html#pandas.Series.str.contains)

glad night Feb 2, 2021, 3:24 PM

#

@serene scaffold Thanks a lot man - I will do some reading after work tonight.

dusty anchor Feb 2, 2021, 3:27 PM

#

hello guys, im trying to import images into a tensorflow dataset, i need the dataset for image segmentation and i cant understand which is the correct way to create teh dataset, does it need to have couples (image-mask)? or i need 2 datasets one containing the images and one containing the masks?

tall trail Feb 2, 2021, 4:06 PM

#

lapis sequoia I envy you

u'll get there, im not that special

#

anyone ever worked with a datalake and databricks togheter? how do you check for new available data? does a notebook save variables if you run it as a task?

alpine mountain Feb 2, 2021, 5:43 PM

#

📎 IMG_20210202_231028.jpg

#

Can anybody help me with this?

misty flint Feb 2, 2021, 5:59 PM

#

i just got an interview for a DS internship

#

Praise

solid aurora Feb 2, 2021, 6:04 PM

#

So I have this numpy array as follows:

#

(I'm substituting in small numbers for the dimensions to make it easier to explain)

#

It's shape is (10, 10, 5, 5, 3)

#

it's a 10x10 array of 5x5x3 image tiles

#

I need to recombine it into a single 50x50x3 image

#

what's the best way to do this?

#

Obviously I could do it with two for-loops but I'd rather have a more efficient vectorized way

hazy flax Feb 2, 2021, 6:20 PM

#

Good afternoon, can you tell me where a data science programmer can work?

#

Can you work in physics and chemistry labs?

#

I wanted to go to Astrophysics College to work with python, but I don't know if there is a library for that.

#

😮

#

ehueuehue

#

In Brazil python is very popular, they use it a lot to create websites with back end and facial recognition programs.

#

Okay, thanks for helping me.

sturdy musk Feb 2, 2021, 8:34 PM

#

hoe to hak naza

hazy flax Feb 2, 2021, 9:44 PM

#

sturdy musk hoe to hak naza

You will need blue dye and the white house heuheuhueheeuheu

idle cave Feb 2, 2021, 10:25 PM

#

Does anyone here use pytorch or tensorflow? If so why do you use each one respectively and which would you prefer to use for a personal major web project that incorporates Django? What are the learning curves?

stray ingot Feb 2, 2021, 10:41 PM

#

Hello! I'm currently working on a project where I need to convert the adjective form of a country name into its noun. For example, convert Italian to Italy, Spanish to Spain, etc...

#

Does anyone know how to achieve this? Its hard to search this on google, because the default search returns currency converters lol

misty flint Feb 2, 2021, 11:13 PM

#

ngl i looked at this for forever until i realized its supposed to be read bottom-up

📎 OCR-2.png

velvet thorn Feb 2, 2021, 11:14 PM

#

stray ingot Hello! I'm currently working on a project where I need to convert the adjective ...

depends

#

if you can guarantee

#

no misspellings

#

and a fixed + known set of source words

#

you can just use regex + replacement

#

otherwise things get more complex

misty flint Feb 2, 2021, 11:14 PM

#

pithink

velvet thorn Feb 2, 2021, 11:14 PM

#

solid aurora I need to recombine it into a single `50x50x3` image

in what order are the tiles?

#

depending on structure

#

some combination of np.concatenate and np.stack

solid aurora Feb 2, 2021, 11:17 PM

#

Ooh perfect, thanks!

fair shoal Feb 3, 2021, 12:22 AM

#

Is it ok if I post a referral link for $15 off of a dataquest subscription? I get a free year if someone signs up with it, but I don't want to break any rules regarding spam or solicitation. Someone here might be able to make use of it.

fair shoal Feb 3, 2021, 4:55 AM

#

Seems like it shouldn't be an issue. I've found them useful for brushing up on R and Python for data manipulation. Full transparency: if four people sign up through the link I get a free year. I hope someone is able to make use of it. Link: app.dataquest.io/referral-signup/p2d1jh5t/

lapis sequoia Feb 3, 2021, 7:24 AM

#

Take Developer Ecosystem Survey by JetBrains
https://t.co/S8US6FsJAQ?amp=1

Developer Ecosystem Survey 2021

Share your coding expertise with the professional community. Take part in the survey, win prizes, and get personalized your results.
#DevEcosystem2021

earnest widget Feb 3, 2021, 8:19 AM

#

Not really a Python related question, more of a data pre-processing question. I have a dataset which is about 600,000+ rows, would it make sense to remove rows to make it easier to work with or would that give me bias or incorrect results?

hasty grail Feb 3, 2021, 8:32 AM

#

Depends on how your select the rows to be removed

#

If you do it in a completely random manner it should be reasonably fine.

earnest widget Feb 3, 2021, 8:35 AM

#

Yeah I'm not deleting selected rows or such, it's all random.

hasty grail Feb 3, 2021, 8:56 AM

#

However you should note that the more data you have, the easier it is for your model to generalize

#

Still, using less data may be sensible if you just want a proof-of-concept

rugged comet Feb 3, 2021, 10:46 AM

#

If anyone here has a solution to this issue, please tell me.
https://github.com/tensorflow/tensorflow/issues/46247

GitHub

Unexpected Events CUDA_ERROR_ILLEGAL_ADDRESS and CUDA_ERROR_LAUNCH_...

System information Have I written custom code (as opposed to using a stock example script provided in TensorFlow): I have followed this tutorial using my own data. Tutorial: https://stackabuse.com/...

dusty anchor Feb 3, 2021, 10:53 AM

#

Hey guys ive this error when executing my script : Shapes (None, 256, 512, 3) and (None, 256, 512, 4) are incompatible.

#

how can i check what is giving it?

hasty grail Feb 3, 2021, 11:12 AM

#

dusty anchor Hey guys ive this error when executing my script : Shapes (None, 256, 512, 3) an...

Find the line where the error occurs, which should tell you which two tensors have incompatible shapes

dusty anchor Feb 3, 2021, 11:18 AM

#

hasty grail Find the line where the error occurs, which should tell you which two tensors ha...

it says that happens when i train my model so i cant understand with part of my model is wrong

hasty grail Feb 3, 2021, 11:21 AM

#

Can you paste the error log?

#

!paste

arctic wedgeBOT Feb 3, 2021, 11:21 AM

#

Pasting large amounts of code

If your code is too long to fit in a codeblock in discord, you can paste your code here:
https://paste.pydis.com/

After pasting your code, save it by clicking the floppy disk icon in the top right, or by typing ctrl + S. After doing that, the URL should change. Copy the URL and post it here so others can see it.

dusty anchor Feb 3, 2021, 11:22 AM

#

hasty grail Can you paste the error log?

here it is https://paste.pythondiscord.com/esudututeg.sql

hasty grail Feb 3, 2021, 11:23 AM

#

Try compiling your model with run_eagerly=True

#

Should make it easier to debug

dusty anchor Feb 3, 2021, 11:23 AM

#

ok, where should i put this parameter?

hasty grail Feb 3, 2021, 11:23 AM

#

in model.compile

#

Did it work?

dusty anchor Feb 3, 2021, 11:28 AM

#

it takes a while cuz ive a big dataset

#

https://paste.pythondiscord.com/imuxocazab.sql

hasty grail Feb 3, 2021, 11:28 AM

#

If you need to debug your model you can just take the first few elements of the dataset to save time

#

Are you using the correct loss function? Seems that the error is occurring there

dusty anchor Feb 3, 2021, 11:30 AM

#

im honestly going blind right now, i tried both sparse_ catergorical_cross and categorical cross

#

i keep receiving a different error everytime

hasty grail Feb 3, 2021, 11:31 AM

#

What is the output shape of your model?

#

Also the same for your ground truth

dusty anchor Feb 3, 2021, 11:32 AM

#

this is my model, https://paste.pythondiscord.com/kesijaxipu.apache

#

as input i use pictures rgb of size 256,512

hasty grail Feb 3, 2021, 11:33 AM

#

Can you display the model summary?

dusty anchor Feb 3, 2021, 11:33 AM

#

yes

#

this is my first try at image segmentation and im a bit lost tbh

#

https://paste.pythondiscord.com/boyakewuve.md here is the summary

hasty grail Feb 3, 2021, 11:37 AM

#

How about the ground truth?

dusty anchor Feb 3, 2021, 11:47 AM

#

u mean my metric?

#

i use categorical accuracy

hasty grail Feb 3, 2021, 11:49 AM

#

Ground truth = The correct "answer" you're supposed to predict

dusty anchor Feb 3, 2021, 11:54 AM

#

hasty grail Ground truth = The correct "answer" you're supposed to predict

as u could understand im a beginner, so idont know exactly how to check the ground truth, i have two numpy arrays containing my images and my mask, i create them a dataset with the from tensor slices function and i feed it to my model just to check if my dataset was correct, but probably my model is not correct for the dataset im using

hasty grail Feb 3, 2021, 11:55 AM

#

print the dataset directly, it should tell you the shapes of its components

dusty anchor Feb 3, 2021, 11:57 AM

#

PrefetchDataset shapes: ((None, 256, 512, 3), (None, 256, 512, 3)), types: (tf.float32, tf.float32)>

hasty grail Feb 3, 2021, 11:59 AM

#

ok so your ground truth is the second element

#

with shape (None, 256, 512, 3)

dusty anchor Feb 3, 2021, 11:59 AM

#

oh ok, they should be my masks

hasty grail Feb 3, 2021, 11:59 AM

#

that doesn't fit your model output

#

your model is outputting 4 masks

#

but the ground truth only has 3

#

as indicated by the last element of its shape

dusty anchor Feb 3, 2021, 12:01 PM

#

conv2d_23 (Conv2D) (None, 256, 512, 4) this part of the summary?

hasty grail Feb 3, 2021, 12:02 PM

#

yes

dusty anchor Feb 3, 2021, 12:04 PM

#

i changed the last conv2d so now the output correspond , but i still get this : https://paste.pythondiscord.com/pumuvalote.sql this is probably given by my loss function right?

hasty grail Feb 3, 2021, 12:05 PM

#

Can you describe the image segmentation task you are trying to achieve?

#

Taking a step back ^

dusty anchor Feb 3, 2021, 12:08 PM

#

, i have the cityscape dataset and i need to make and train a model that can fit into a microcrontroller (1.5MB) so what i want to achieve is that the model is able to generate a mask that can classify the objects on the pictures, with an accuracy of 80%+

hasty grail Feb 3, 2021, 12:10 PM

#

How are you generating the dataset?

dusty anchor Feb 3, 2021, 12:12 PM

#

ive a function that get all the paths of the pictures and the masks and give me back 2 lists containing all them sorted, i then generate 2 numpy array from the path that contain the images and the masks

hasty grail Feb 3, 2021, 12:13 PM

#

and I suppose there are only 3 classes that have to be identified?

#

are they mutually exclusive?

dusty anchor Feb 3, 2021, 12:13 PM

#

teh calsses are 30 in the dataset

hasty grail Feb 3, 2021, 12:13 PM

#

then why is the shape (..., 3)?

dusty anchor Feb 3, 2021, 12:13 PM

#

road , sky , person car, etc...

hasty grail Feb 3, 2021, 12:14 PM

#

if there are 30 classes it should be (..., 30)

dusty anchor Feb 3, 2021, 12:14 PM

#

i probably did some mistakes then

#

i think the 3 is about the rgb of the picture

hasty grail Feb 3, 2021, 12:14 PM

#

yeah

#

also your model should be outputting 30 channels, 1 per class

#

not 4

dusty anchor Feb 3, 2021, 12:19 PM

#

i see, seems logic, so now i need to sort some things out, first i need to import the class correctly i guess, so that i have all the 30 classes

#

i have json files containing the objects and the masks poligons

hasty grail Feb 3, 2021, 12:21 PM

#

yeah

dusty anchor Feb 3, 2021, 12:22 PM

#

i guess i need to parse the json for every single mask right?

hasty grail Feb 3, 2021, 12:22 PM

#

Not sure how it's formatted, it's on you to parse it

dusty anchor Feb 3, 2021, 12:24 PM

#

i just dont understand one thing, how can the model understand which pixel correspond to a class during the training?

hasty grail Feb 3, 2021, 12:58 PM

#

It doesn't need to

#

You just have to ensure that the classes in your dataset are consistent, so that channel N always corresponds to class N

#

Then, whenever the model predicts a high value for channel N, the same is said for class N.

#

The model only needs to learn to predict a value for each channel (like in other tasks), which is implicitly the class (in this case)

dusty anchor Feb 3, 2021, 2:00 PM

#

hasty grail The model only needs to learn to predict a value for each channel (like in other...

ok, i have a json for every image/mask it is like : https://paste.pythondiscord.com/egiraqiqoj.json

hasty grail Feb 3, 2021, 2:01 PM

#

you'll have to convert the labels into indices for the masks (channels)

dusty anchor Feb 3, 2021, 2:02 PM

#

u mean a list containing all the classes?

hasty grail Feb 3, 2021, 2:02 PM

#

e.g. car = 0, person = 1

#

I think a dictionary would be more appropriate

dusty anchor Feb 3, 2021, 2:10 PM

#

hasty grail I think a dictionary would be more appropriate

i made a dic containing all the classes like car:0 etc...

hasty grail Feb 3, 2021, 2:11 PM

#

you can lookup the dict to get the index from the label name then

dusty anchor Feb 3, 2021, 2:12 PM

#

so now i just need to find a way to assign labels to the masks i guess

hasty grail Feb 3, 2021, 2:14 PM

#

^

dusty anchor Feb 3, 2021, 2:50 PM

#

hasty grail ^

the dataset give me 3 masks, which one should i use? can i post here them?

hasty grail Feb 3, 2021, 2:52 PM

#

only 3 masks? which ones?

#

I have to go soon so maybe someone else can help

dusty anchor Feb 3, 2021, 2:54 PM

#

hasty grail only 3 masks? which ones?

a fully colorized one, named color.png, one that has car higlighted called instancedlds and one that seems to be greyscale called labellds

dusty anchor Feb 3, 2021, 2:54 PM

#

hasty grail I have to go soon so maybe someone else can help

dont worry u helped me a lot already

#

3 masks for every image

azure leaf Feb 3, 2021, 3:58 PM

#

anyone got a good guide on picking a good alpha value?

#

            ('clf', OneVsRestClassifier(MultinomialNB(alpha=0.1, fit_prior=True, class_prior=None))),
            ])```

fallen yoke Feb 3, 2021, 4:11 PM

#

What's the difference between fit and transform? I still don't get it :/

quiet elk Feb 3, 2021, 4:28 PM

#

Hi all, i'm not sure if anyone will see this but while having a relaxing walk I had an idea to create and machine learning algorithm which will detect sequences in data e.g. Arithmetic sequences, geometric sequences using an example data set. Now i'm home i've attempted to find a example dataset but I can't find one and I wondered if there was any way you guys could help me find/create one to use? My template was going to be the pattern formula in the first column then the pattern from 0n to 10n. Thank you sooo much to anyone that can help me !😁

woeful leaf Feb 3, 2021, 4:30 PM

#

do anyone know how to import numpy from the cloned github numpy repo?
I have cloned the numpy github repo to my local.
then in terminal i did import numpy as np
but I'm not able to access numpy method. for ex. np.sin()

#

getting error AttributeError: module 'numpy' has no attribute 'sin'

woeful hamlet Feb 3, 2021, 4:59 PM

#

what can i do when my data set doesnt have the same ammount of images per class?
the confusion matrix is a shiit xd every class (almost) filled with 0's

austere swift Feb 3, 2021, 5:00 PM

#

try changing the class weights

#

so having some classes that have a lower amount of images be weighted more

#

thats assuming you don't wanna do augmentation or anything which would be a better idea

woeful hamlet Feb 3, 2021, 5:01 PM

#

i did augmentation

austere swift Feb 3, 2021, 5:02 PM

#

damn python bot has no scikit learn docs

#

https://scikit-learn.org/stable/modules/generated/sklearn.utils.class_weight.compute_class_weight.html

#

that's how you would do the balancing class weights

woeful hamlet Feb 3, 2021, 5:02 PM

#


       001_Class       0.00      0.00      0.00        18
       002_Class       0.00      0.00      0.00        17
       003_Class       0.00      0.00      0.00        23
       004_Class       0.00      0.00      0.00        18
       005_Class       0.00      0.00      0.00        16
       006_Class       0.00      0.00      0.00        22
       007_Class       0.00      0.00      0.00        20```

#

it looks like this almost all the matrix

azure leaf Feb 3, 2021, 6:29 PM

#

anyone here familliar with the multinomialNB

#

and how to pick good alpha vlaues

abstract zealot Feb 3, 2021, 8:11 PM

#

what are you simulating @azure leaf

#

you should really just benchmark your test set and plot for different values of alpha, this is usually a good way to determine optimal hyperparamaters, unless you have an underlying idea of how much shift or smoothing you would like

austere swift Feb 3, 2021, 9:06 PM

#

How would i find all rows that have a certain value within one of their columns in a pandas dataframe? for example if i had a dataframe that has one column that contains a class, but the column can contain more than one class in it, how would i get all the rows that have a certain class in that column

#

the classes within the column are structured like "class1|class2|class3" etc

#

and they can have different amounts of classes as well, or just one single class

lapis sequoia Feb 3, 2021, 9:13 PM

#

pandas.DataFrame.eval?

austere swift Feb 3, 2021, 9:17 PM

#

no i figured it out, it was pandas.Series.str.contains

#

I knew that was a thing i didnt know it supported checking for multiple items

#

but thank you anyways

lapis sequoia Feb 3, 2021, 9:30 PM

#

can someone help me convert unix to datetime?

#

df1['datetime'] = df1['unix'].apply(lambda x: datetime.utcfromtimestamp(x).strftime('%Y-%m-%d %H:00'))

#

this is my try. but it returns just OSError: [Errno 22] Invalid argument

#

the unix column looks like this in the dataframe:

0        1.612310e+12
1        1.612307e+12
2        1.612303e+12
3        1.612300e+12
4        1.612296e+12
             ...     
33016    1.502957e+09
33017    1.502953e+09
33018    1.502950e+09
33019    1.502946e+09
33020    1.502942e+09
Name: unix, Length: 33021, dtype: float64

lapis sequoia Feb 3, 2021, 9:50 PM

#

nevermind, figured it out with some help

ashen nacelle Feb 3, 2021, 9:50 PM

#

Hi guys

lapis sequoia Feb 3, 2021, 9:50 PM

#

it is in milliseconds and not normal seconds. just had to divide the x with 1000

ashen nacelle Feb 3, 2021, 9:51 PM

#

I was trying to launch anaconda navigator using cli on Linux

#

But for some reason it is not working

#

Anyone knows how to solve this issue?

austere swift Feb 3, 2021, 10:00 PM

#

i think my model's doing pretty well

📎 unknown.png

#

lol i messed up on the accuracy reporting

woeful hamlet Feb 3, 2021, 10:42 PM

#

what is the seed argument for on flow_from_directory?

#

I have a ImageDataGenerator object

#

like this

#

    rescale=1./255,
    rotation_range=20,
    width_shift_range=0.2,
    height_shift_range=0.2,
    shear_range=0.2,
    zoom_range=0.2,
    horizontal_flip=True,
    validation_split=0.975)```

#

validation split is so high for resting purposes

#

and then i do this

#

    directory=data_dir, target_size=dimensions[:2],
    seed=seed, subset='training')```

#

but when i print train_generator.filenames

#

It always prints the same ones

#

no matter the seed

remote summit Feb 4, 2021, 4:21 AM

#

Hygy

old veldt Feb 4, 2021, 6:40 AM

#

hi ! What would be the best approach to handling missing data in time series of cryptocurrencies data? I want to predict etherum prices but most others currencies didn't exist at the time etherum was invented so I have a lot of missing data in the bottom of my table. What is the proper way to handle this? Nice day to all of you! 🌟

sterile saddle Feb 4, 2021, 8:16 AM

#

Can anybody help out with opencv? 🙂 Thank you

dusty anchor Feb 4, 2021, 9:19 AM

#

hey guys which is the fastest model i can use for image segmentation? i need low accuracy (80%)

lapis sequoia Feb 4, 2021, 12:08 PM

#

i have a list of 106 tokens
when i analyzed them i got this analysis , how can i predict the next token?

📎 unknown.png

#

what i mean is to generate a few tokens which follows the similar pattern

#

📎 unknown.png

#

ping me up if anyone decides to help

warm bane Feb 4, 2021, 12:58 PM

#

Dense(6,activation="sigmoid",kernel_initializer='glorot_uniform')(x)

what is the meaning of this source code? whereas so far sigmoid has only been used for binary classification

azure leaf Feb 4, 2021, 1:09 PM

#

man ML is so hard

#

https://towardsdatascience.com/multi-label-text-classification-5c505fdedca8 is this a good article to learn off?

Medium

Multi-Label Text Classification

Assign labels to movies based on descriptions

tall basin Feb 4, 2021, 2:40 PM

#

https://twitter.com/Br3Sc/status/1357328840431910913?s=20
Sup guys, pls if y'all don't mind take a look in my tweet about learn data science for free

Pierre de Fermat (@Br3Sc)

Free resources to learn #MachineLearning and #DataScience for free🆓:

I've been gathering resources from these wonderful people:
@svpino @PrasoonPratham

#100DaysOfCode #Python #codingtips #CodeNewbie #pythonprogramming #Python3 #Algorithm #data #pythonlearning #code #Tips #ML

olive tinsel Feb 4, 2021, 6:04 PM

#

hello data science people. I have a csv file and one of the column names is "Activities", under this column are a bunch of records where this value is "Sitting", which I dont want

#

how do I drop all records where the Activity value is "Sitting"? please 🙂

olive tinsel Feb 4, 2021, 6:43 PM

#

if anyone could help, pls pm me 🙂

stray owl Feb 4, 2021, 6:45 PM

#

df_filtered = df[df['Activities'] != 'Sitting']

olive tinsel Feb 4, 2021, 6:49 PM

#

stray owl df_filtered = df[df['Activities'] != 'Sitting']

How would you make that work for multiple fields? right now I have this, and it doesnt work:
`NotUsedActivites = unseen_df[unseen_df['Activity'] == 'Vibration' | (unseen_df['Activity'] == 'Drop_n_Pickup')].index

unseen_df.drop(NotUsedActivites, inplace = True)`

stray owl Feb 4, 2021, 6:56 PM

#

df_filtered = df[df['Activity'].isin(['Vibration', 'Drop_n_Pickup'])]

#

I think this is what you're looking for

#

@olive tinsel

olive tinsel Feb 4, 2021, 6:58 PM

#

that worked 😄 thank you!!!!

#

MU!!!! to you too!

lapis sequoia Feb 4, 2021, 7:09 PM

#

hi

#

umhm

#

sorry but mind if i ask something?

olive tinsel Feb 4, 2021, 7:11 PM

#

sure 🙂

lapis sequoia Feb 4, 2021, 7:14 PM

#

ummm...i have these 318 samples of a token when i analyzed them i got this as output

📎 unknown.png

#

📎 unknown.png

#

so can i generate a few similar patterns of token using this data?

#

like predicting what the next token would be

#

@olive tinsel

olive tinsel Feb 4, 2021, 7:16 PM

#

oh my... I am no where near that good at python

#

im sorry, I thought it would be simple 😢

lapis sequoia Feb 4, 2021, 7:17 PM

#

;-;

#

datascience is not my subject as well

#

thats why asked

olive tinsel Feb 4, 2021, 7:18 PM

#

same here 😦

#

I only started python 1 week ago

lapis sequoia Feb 4, 2021, 7:18 PM

#

ping me up anyone if u would like to help, thanks

analog kiln Feb 4, 2021, 9:17 PM

#

anyone here have experience with optimizing a streamlit app so that a weak server can run it? i'm having some issues and can't seem to get it to just... not crash hah

misty flint Feb 4, 2021, 9:44 PM

#

ID_BoomKek

#

gl

analog schooner Feb 4, 2021, 9:58 PM

#

what would you choose?

📎 unknown.png

abstract zealot Feb 4, 2021, 10:12 PM

#

2 and 6 maybe?

#

what do you think?

#

@analog schooner

analog schooner Feb 4, 2021, 10:14 PM

#

2 is obvious choice for me

#

but not sure about another point

abstract zealot Feb 4, 2021, 10:14 PM

#

hmmm

analog schooner Feb 4, 2021, 10:14 PM

#

leaning to 1. option

abstract zealot Feb 4, 2021, 10:15 PM

#

lots of rows?

analog schooner Feb 4, 2021, 10:15 PM

#

yeah, few columns reduce accuracy quickly

abstract zealot Feb 4, 2021, 10:15 PM

#

thats true

#

and last 4 are kind of vague

analog schooner Feb 4, 2021, 10:15 PM

#

anwer to all questions is: "well, it depends"

abstract zealot Feb 4, 2021, 10:16 PM

#

EXACTLY

#

incorrectly labelled data but if you have low dimensionality data then this probably is an easy fix

#

yikes

#

rough one

analog schooner Feb 4, 2021, 10:17 PM

#

with labels it also depends whether you realize it during your analysis

abstract zealot Feb 4, 2021, 10:17 PM

#

trueeeeee, do they give you a 2000 word essay as an answer xd

analog schooner Feb 4, 2021, 10:18 PM

#

it is a question in a job application form

#

input your date, upload resume, answer this question 😄

abstract zealot Feb 4, 2021, 10:19 PM

#

maybe answer with the ones that are most appropriate for the job youre applying for and the data youre working with?

analog schooner Feb 4, 2021, 10:19 PM

#

junior data scienstist for DS/AI consulting company

#

just submitted by application, picked the first and second option and wrote it the comment below that "it depends".

#

thanks for your help!

misty flint Feb 4, 2021, 10:57 PM

#

good luck dude!

#

cattohug

twin moth Feb 4, 2021, 11:00 PM

#

Heya, any chance to get some help with Selenium?

#

I am trying to filter HTML elements by using multiple filters, not sure how to accomplish that though

abstract zealot Feb 4, 2021, 11:11 PM

#

not sure bro

velvet thorn Feb 4, 2021, 11:28 PM

#

twin moth I am trying to filter HTML elements by using multiple filters, not sure how to a...

find by CSS selector?

twin moth Feb 4, 2021, 11:32 PM

#

velvet thorn find by CSS selector?

Thanks for the hasty reply, was already able to fetch it using xpath -

    for img in driver.find_elements_by_xpath("//img[contains(@data-automation, 'mosaic-grid-cell-image')]"):

#

But now I have another issue

#

The site I'm trying to fetch the data from loads the images when the images are close to being on screen

#

I tried scrolling slowly to the bottom of the screen and then fetch the links but seems like it doesn't work because the browser is in the background.
Got any idea how to get it to work with a headless setup?

#

Would love some pointers regarding it, tried moving the focus of the driver between the windows, no change whatsoever

velvet thorn Feb 4, 2021, 11:46 PM

#

twin moth I tried scrolling slowly to the bottom of the screen and then fetch the links bu...

hm

#

haven't worked with this in a long time, honestly

#

don't really remember, sorry

#

but

twin moth Feb 4, 2021, 11:47 PM

#

Well, I really appreciate you trying though 🙂

velvet thorn Feb 4, 2021, 11:47 PM

#

I am a bit sceptical that focus would matter

#

do you know this to be the case?

#

or is it due to the scrolling behaviour itself

twin moth Feb 4, 2021, 11:48 PM

#

I tried it a couple of times, once I tried running the script when the FireFox opened in the background and the other I tried executing it I was just looking at the browser - the two had whole different outcomes

boreal summit Feb 4, 2021, 11:56 PM

#

Hello everyone, is there like a website or directory to know what version of Tensorflow would work with your laptop?

velvet thorn Feb 4, 2021, 11:56 PM

#

twin moth I tried it a couple of times, once I tried running the script when the FireFox o...

put it in the background but in a position where you can visually inspect it

boreal summit Feb 4, 2021, 11:57 PM

#

I've been practicing on Google's colab but would like to run some stuffs on my PC (hp elite book 8440p).

#

The latest versions of TF are giving me issues, so I'm thinking of installing a lesser version.

#

Like 1.X versions. Thanks.

velvet thorn Feb 4, 2021, 11:59 PM

#

boreal summit Hello everyone, is there like a website or directory to know what version of Ten...

why do you think an older version would work?

boreal summit Feb 5, 2021, 12:00 AM

#

velvet thorn why do you think an older version *would* work?

Actually, I'm thinking an older version would work better as the latest versions (2.X) are not running on my PC.

#

I'm getting different issues which I can't resolve.

velvet thorn Feb 5, 2021, 12:01 AM

#

boreal summit Actually, I'm thinking an older version would work better as the latest versions...

not running how

boreal summit Feb 5, 2021, 12:01 AM

#

It's saying some stuff about DLL import error and stuff.

velvet thorn Feb 5, 2021, 12:01 AM

#

your dependencies

#

might not be set up properly

#

hard to say

boreal summit Feb 5, 2021, 12:01 AM

#

So I'm thinking maybe 1.X versions would work with my laptop since it's old.

velvet thorn Feb 5, 2021, 12:02 AM

#

velvet thorn hard to say

hard to say how to fix it, but that seems to be the case

boreal summit Feb 5, 2021, 12:02 AM

#

Okay, I'll see what I can do.

#

Thanks.

twin moth Feb 5, 2021, 12:08 AM

#

velvet thorn put it in the background but in a position where you can visually inspect it

ummm I'll try, but don't forget that I strive to achieve a headless script

#

And I'm using a tiling WM, not sure how I'd be able to do so fast enough

velvet thorn Feb 5, 2021, 12:09 AM

#

twin moth ummm I'll try, but don't forget that I strive to achieve a headless script

no, I mean, to verify your hypothesis

#

namely, that it's focus that's causing the problem

twin moth Feb 5, 2021, 12:11 AM

#

velvet thorn no, I mean, to verify your hypothesis

Huh, seems like you were right, or rather that I was wrong

#

I had it in view but not under focus

#

Yet I was able to extract all needed elements

#

Got any idea how to deal with it though? It only raises more questions

twin moth Feb 5, 2021, 12:30 AM

#

I'm heading to bed, if you guys have any idea how to handle that beast I'd be more than happy to hear
Thanks! 🙂

woeful hamlet Feb 5, 2021, 1:18 AM

#

Following this tuto

#

https://docs.opencv.org/3.4/dc/dc3/tutorial_py_matcher.html

#

How (coding) can i say if a match between 2 images is good or not?

opal sleet Feb 5, 2021, 1:35 AM

#

Is someone familiar with plotly?

misty flint Feb 5, 2021, 1:41 AM

#

i decided to apply for a Computational LInguistics/NLP minor bc im a clown

#

Clown2

rotund dagger Feb 5, 2021, 1:46 AM

#

hi guys, i have a question about finding start to end times of a data frame in days months years. this i what i have tried df['Date] = pd.to_datetime(df['Date'] to change to a datetime object. then i did df['Date'].max().day - df['Date].min().day. but i now see that this is wrong because what it is doing is going to the last date.day and subtracting the min date.day from it but that is not actually number of days. for instance '2010-27-02' - '2000-01-02' it will return 26

#

when really what i am trying to find "this data span is from 9 years, 3 months and 28 days".

abstract zealot Feb 5, 2021, 2:06 AM

#

bruh @rotund dagger can u send sc of pd df

rotund dagger Feb 5, 2021, 2:06 AM

#

@abstract zealot

📎 unknown.png

#

the correct answer is 9 years, 7 months, and 25 days. im just not sure how to get there

#

via pandas that is

abstract zealot Feb 5, 2021, 2:08 AM

#

gimme one sec

rotund dagger Feb 5, 2021, 2:09 AM

#

thank you

abstract zealot Feb 5, 2021, 2:09 AM

#

youre basically trying to find the difference between 1st and last date?

#

or am i wrong

rotund dagger Feb 5, 2021, 2:09 AM

#

yea

abstract zealot Feb 5, 2021, 2:29 AM

#

sorry bro back now

#

because you sort the column there are a couple ways you could do it

#

calculating the difference like ```py
start = pd.to_datetime(df['Date'][0], format='%Y-%d-%m')
end = pd.to_datetime(df['Date'].iloc[-1], format='%Y-%d-%m')
difference = end - start

#

this will probs return a timedelta in days??? although im not sure

#

i cant remember much about it

#

if it does you can just manipulate it a bit to get the required format @rotund dagger

rotund dagger Feb 5, 2021, 2:33 AM

#

thank you i will mess around with that. i dont have to sort the column necessarily i was just messing with functions. im looking into relativedelta from the dateutil libray at the moment

abstract zealot Feb 5, 2021, 2:34 AM

#

yea so if you print(difference) it should return number of days between those two dates

#

is it right? lol

rotund dagger Feb 5, 2021, 2:34 AM

#

it kind of does but its a bit skewed.

abstract zealot Feb 5, 2021, 2:34 AM

#

wym?

rotund dagger Feb 5, 2021, 2:34 AM

#

i get 10 years 25 days and 5 months.

#

it should be 9 years 7 monts and 25 days

abstract zealot Feb 5, 2021, 2:35 AM

#

are you years-day-month?

#

yikes

#

youre years-month-day

#

start = pd.to_datetime(df['Date'][0], format='%Y-%m-%d')
end = pd.to_datetime(df['Date'].iloc[-1], format='%Y-%m-%d')
difference = end - start

#

try that

rotund dagger Feb 5, 2021, 2:36 AM

#

ok , sec ill let you know

#

@abstract zealot

📎 unknown.png

#

i start.day rather lol in the diffday line, but still stlightly wrong

#

📎 unknown.png

abstract zealot Feb 5, 2021, 2:41 AM

#

what i wrote should return that, i just tried

#

ahhhhh

#

it might not work because you need to sort values

rotund dagger Feb 5, 2021, 2:42 AM

#

ohhhhhh i just realized i missed some values of what you had duh

#

sec

abstract zealot Feb 5, 2021, 2:42 AM

#

try putting df = df.sort_values('Date')

#

so ```py
df = df.sort_values('Date')
start = pd.to_datetime(df['Date'][0], format='%Y-%m-%d')
end = pd.to_datetime(df['Date'].iloc[-1], format='%Y-%m-%d')
difference = end - start
print(difference)

supple minnow Feb 5, 2021, 2:44 AM

#

can someone explain me why boxplot doesn't working?

📎 why.png

rotund dagger Feb 5, 2021, 2:44 AM

#

this is my output

📎 unknown.png

abstract zealot Feb 5, 2021, 2:46 AM

#

bruh

#

ok, what might be happening is whenever you sort the date column, pandas sorts by year, and doesnt know how to handle datetime objects, take away the sort_values

#

or convert the entire column to datetime objects and then sort it

#

sorting the column is what messed it up, that code should return the exact number of days between the first and last date in the column

rotund dagger Feb 5, 2021, 2:48 AM

#

true

#

let me retry without sort

abstract zealot Feb 5, 2021, 2:49 AM

#

you will need to rerun the block of code where you defined df

#

since you overwrote it when you said df = df.sort_values

rotund dagger Feb 5, 2021, 2:50 AM

#

so this is what i have and what ive re ran

#

📎 unknown.png

abstract zealot Feb 5, 2021, 2:50 AM

#

run the top one where you say df=read_csv

#

then run my code

#

and see if it works

#

lmao

#

im struggling today xd

#

@supple minnow do you have repeats for those categories?

#

if you only do something once it appears like that

rotund dagger Feb 5, 2021, 2:52 AM

#

@abstract zealot all 3 implementations show 3128

#

📎 unknown.png

#

except for 1 is not calculating the last day in the mix

abstract zealot Feb 5, 2021, 2:53 AM

#

omg im so sorry, you will need to put df['Date'] = pd.to_datetime(df['Date'])

#

then df.sort_values('Date')

#

take away all the other code, just make sure youre defining the dataframe, converting the date column to datetime objects, sorting the column, then using my code

rotund dagger Feb 5, 2021, 2:57 AM

#

📎 unknown.png

abstract zealot Feb 5, 2021, 2:58 AM

#

idk man, i literally replicated this on my own system and got 3524

rotund dagger Feb 5, 2021, 2:59 AM

#

dam, let me try with just hardcoding those 2 dates

abstract zealot Feb 5, 2021, 2:59 AM

#

thats a good idea actually

#

lst = ['2007-11-01', '2016-02-01', '2020-02-10', '2017-06-25']
df = pd.DataFrame(lst, columns=['Date'])
start = pd.to_datetime(df['Date'][0], format='%Y-%m-%d')
end = pd.to_datetime(df['Date'].iloc[-1], format='%Y-%m-%d')
difference = end - start
print(difference)

#

this prints 3524 and is just simulating your situation

#

although your indexing is crazy on that df

rotund dagger Feb 5, 2021, 3:03 AM

#

yea index runs from 0 - 142193

#

but unsorted lol

abstract zealot Feb 5, 2021, 3:03 AM

#

try the df = df.sort_values('Date') again and print df

#

i dont think it reindexes it

#

or call it org = df.sort_values('Date') so you dont overwrite

rotund dagger Feb 5, 2021, 3:05 AM

#

📎 unknown.png

abstract zealot Feb 5, 2021, 3:05 AM

#

anyway, ill make new code that shouldnt need you to sort anything

#

nah it doesnt

#

i think its an indxing thing

#

are you using publicly available data?

rotund dagger Feb 5, 2021, 3:06 AM

#

yea ill send you the link

#

https://www.kaggle.com/jsphyg/weather-dataset-rattle-package

Rain in Australia

Predict next-day rain in Australia

abstract zealot Feb 5, 2021, 3:08 AM

#

i dont have acc can you send me data in dm

#

its only 4mb

rotund dagger Feb 5, 2021, 3:08 AM

#

yea np

nova kelp Feb 5, 2021, 3:18 AM

#

how do i rename images like photo0,photo1,photo2.... using csv file that contains the name? Thanks in advance!

astral path Feb 5, 2021, 3:28 AM

#

if I have a pandas dataframe that has two different index values but multiple elements with each index, how would I split that up into multiple dataframes? Is there a better way to do this?

#

📎 unknown.png

rotund dagger Feb 5, 2021, 3:52 AM

#

@astral path i think there is a way to do it let me try

astral path Feb 5, 2021, 3:53 AM

#

thank you!

rotund dagger Feb 5, 2021, 3:55 AM

#

to clarify, you want a dataframe with url = abc, and values 1 12 3 4, and a second dataframe with url = def, and val = -190, -4, -5

astral path Feb 5, 2021, 3:56 AM

#

yes, correct

rotund dagger Feb 5, 2021, 3:59 AM

#

so it will be something like this

finite harness Feb 5, 2021, 3:59 AM

#

Ye

rotund dagger Feb 5, 2021, 3:59 AM

#

df1 = pd.dataframe([])

#

df2 = pd.dataframe([])

#

then load the values you need in the dataframe, i have to leave for a sec, when i come back i will try to load it up and show displays

astral path Feb 5, 2021, 4:00 AM

#

ok, thank you! i appreciate this

rotund dagger Feb 5, 2021, 4:02 AM

#

np

astral path Feb 5, 2021, 4:03 AM

#

the bigger issue i have in particular though, is that I have 209 different indices that I want to make into 209 different dataframes

#

i. e.

📎 unknown.png

velvet thorn Feb 5, 2021, 4:09 AM

#

the bigger issue i have in particular though, is that I have 209 different indices that I want to make into 209 different dataframes
@astral path groupby apply

astral path Feb 5, 2021, 4:10 AM

#

what would the apply do?

nocturne plover Feb 5, 2021, 4:28 AM

#

Can anyone suggest me the best model for multiclass classification. I am thinking of Naives Bayes(Gaussian) but is there any better model?

abstract zealot Feb 5, 2021, 4:29 AM

#

@nocturne plover you can model your data by lots of distributions

astral path Feb 5, 2021, 4:29 AM

#

wouldnt it depend on the question youre trying to answer

abstract zealot Feb 5, 2021, 4:30 AM

#

literally check out https://medium.com/@ciortanmadalina/overview-of-data-distributions-87d95a5cbf0a for a flavour of the different ways you can model data

Medium

Overview of data distributions

How to choose the right distribution to model your data

#

@astral path did you solve your problem?>

astral path Feb 5, 2021, 4:33 AM

#

im close

#

colab is just being really slow as of right now so i dont know yet

abstract zealot Feb 5, 2021, 4:34 AM

#

lemme know if you need help man

astral path Feb 5, 2021, 4:34 AM

#

will do and thank you!

lapis sequoia Feb 5, 2021, 4:35 AM

#

I did a df.groupby(["a", "b"]).x.mean() and I ended up with a,b as multiindex with my x-mean column, I'm trying to plot a separate plot for each a with b being on the x axis and x being on the y-axis

astral path Feb 5, 2021, 4:35 AM

#

for more context just to let you guys know

lapis sequoia Feb 5, 2021, 4:35 AM

#

not sure how to do that

astral path Feb 5, 2021, 4:35 AM

#

im looping over every game from this NBA season and am trying to split each game into a new dataframe

#

im solving it a different way though so far:

vals = pd.DataFrame([])

df.set_index(keys=['URL'], drop=False,inplace=True)
urls = df['URL'].unique().tolist()

display(df.loc[df.URL==urls[0]])

for thisURL in urls:
  gamestats = df.loc[df.URL==thisURL]
  display(gamestats['ShotDist'])

abstract zealot Feb 5, 2021, 4:37 AM

#

are you getting this from dictionaries?

astral path Feb 5, 2021, 4:38 AM

#

no, from a csv

#

https://docs.google.com/spreadsheets/d/16iQVdYP4saDmZ5eOXpsDd-0pVbNm2kXH-IZ2hMBb5gc/edit#gid=0

Google Sheets - create and edit spreadsheets online, for free.

Create a new spreadsheet and edit with others at the same time -- from your computer, phone or tablet. Get stuff done with or without an internet connection. Use Sheets to edit Excel files. Free from Google.

#

the data looks like this

abstract zealot Feb 5, 2021, 4:39 AM

#

can you post screenshot im not logged into google sadge

astral path Feb 5, 2021, 4:39 AM

#

yea sure

#

https://www.dropbox.com/s/794jaaou1tep0kr/NBA_PBP_2020-21.csv - Sheet1.csv?dl=1
also theres this

#

📎 unknown.png

#

lol didnt mean to screenshot both screens

abstract zealot Feb 5, 2021, 4:40 AM

#

np

#

so youre trying to create a new df for each unique URL?

astral path Feb 5, 2021, 4:41 AM

#

yeah thats what im doing

#

however instead, i'm looping over each URL and using df.loc to get the rows with that URL instead

abstract zealot Feb 5, 2021, 4:45 AM

#

emm

#

you could try something like ```py
for a, b in df.groupby(by='URL'):
print(a)
print(b)
break

#

this would just give you an example thats why theres a break but basically it takes each element from url and makes a df out of it

#

🙂 @astral path

astral path Feb 5, 2021, 4:47 AM

#

right now i'm doing this

#

vals = pd.DataFrame([])

df.set_index(keys=['URL'], drop=False,inplace=True)
urls = df['URL'].unique().tolist()

display(df.loc[df.URL==urls[0]])

for thisURL in urls:
  gamestats = df.loc[df.URL==thisURL]
  shotDists = gamestats['ShotDist']
  shotOutcomes = gamestats['ShotOutcome']
  for i in enumerate(shotOutcomes):
    if(shotOutcomes[i] == 'miss'):

#

(incomplete)

#

what im trying to do specifically is get the ShotDists for both teams in a specific game, do an analysis with each teams separate series of ShotDists which returns a float, and then send that float to a separate dataframe for storage

abstract zealot Feb 5, 2021, 4:49 AM

#

ahhhhhh okay i get you

#

looks pretty good, keep it up!

astral path Feb 5, 2021, 4:49 AM

#

thank you!

#

i'll update if i need help or if i finish it

warm bane Feb 5, 2021, 5:06 AM

#

anyone know how to convert keras model into pytorch model ?

rotund dagger Feb 5, 2021, 5:09 AM

#

oh good they got you @astral path

astral path Feb 5, 2021, 5:13 AM

#

yeah but

#

i have an error

#

vals = pd.DataFrame([])

df.set_index(keys=['URL'], drop=False,inplace=True)
urls = df['URL'].unique().tolist()

display(df.loc[df.URL==urls[0]])

for thisURL in urls:
  gamestats = df.loc[df.URL==thisURL] 
  homestats = gamestats.loc[gamestats.HomePlay==gamestats['HomePlay'].dropna()]
  awaystats = gamestats['AwayPlay'].dropna()
  #homedists = homestats['ShotDist']
  #awaydists = awaystats['ShotDist']
  #homeoutcomes = homestats['ShotOutcome']
  #awayoutcomes = awaystats['ShotOutcome']
  display(homestats)

#

ValueError: Can only compare identically-labeled Series objects

#

i'm trying to make a new dataframe (homestats) which uses .loc on gamestats with the parameter of all gamestats elements in the column 'HomePlay' which are not NaN

#

@abstract zealot @rotund dagger any ideas?

nocturne plover Feb 5, 2021, 5:20 AM

#

abstract zealot <@!763299991850057729> you can model your data by lots of distributions

Hmm I'll try that

misty flint Feb 5, 2021, 5:23 AM

#

i have no ideas but you should follow ken jee on YT

#

hes a big sports data science guy

#

DoggoKek

astral path Feb 5, 2021, 5:24 AM

#

i'll check him out!

misty flint Feb 5, 2021, 5:25 AM

#

Praise

#

https://youtube.com/playlist?list=PL2zq7klxX5ATB60CpT2Ls3pbsBL-Z2IWq

YouTube

Sports Analytics

astral path Feb 5, 2021, 5:27 AM

#

thank you! will def be checking this out

#

i chose nba analytics for my focus in my DS class this semester

rotund dagger Feb 5, 2021, 5:36 AM

#

im looking it over now so far nothing jumps out

#

in this line gamestats.loc[gamestats.HomePlay==gamestats['HomePlay'].dropna()].... is the gamestats.HomePlay the same as saying gamestats['HomePlay']?

#

HomePlay is a column, but the environment might be confusing it with a method call

#

so could you do gamestats.loc[gamestats.['HomePlay']==gamestats['HomePlay'].dropna()]

#

im fairly new to this so i could be entirely off base

#

@abstract zealot @astral path

astral path Feb 5, 2021, 5:44 AM

#

hmm i dont know

#

i mean to get gameStats I did gamestats = df.loc[df.URL==thisURL]

#

i'll try it

nocturne plover Feb 5, 2021, 5:46 AM

#

warm bane anyone know how to convert keras model into pytorch model ?

Recoding it is the only way.. if I'm not wrong

warm bane Feb 5, 2021, 5:47 AM

#

nocturne plover Recoding it is the only way.. if I'm not wrong

u have recoding it?
wew I need more time to spend up

nocturne plover Feb 5, 2021, 5:48 AM

#

There's no other way... If you need to use same model in PyTorch cause both have separate backends (incompatible)

#

But there's something at GH called PyTorch Lighting which makes it compatible on coding it in PyTorch. I'm not sure if it works but maybe yes.

warm bane Feb 5, 2021, 6:08 AM

#

Incompatible backend?
Easy to understand, thanks for the info!

austere swift Feb 5, 2021, 6:09 AM

#

you can easily convert pytorch to keras by using onnx as a middle man but pytorch doesnt allow loading onnx models

warm bane Feb 5, 2021, 6:10 AM

#

so now can you use onnx to convert from keras to pytorch?

austere swift Feb 5, 2021, 6:10 AM

#

pytorch doesnt allow loading onnx models

warm bane Feb 5, 2021, 6:12 AM

#

so can't ?

#

okay

astral path Feb 5, 2021, 7:00 AM

#

rotund dagger <@!467031448940052480> <@!478676609914765322>

its the same error when i try that

astral path Feb 5, 2021, 7:18 AM

#

so i think i know why

#

when i do homestats = gamestats.loc[gamestats.HomePlay==gamestats['HomePlay'].dropna()]

#

its trying to compare gamestats.HomePlay and gamestats['Homeplay'].dropna()

#

which are both of different lengths because .dropna() is removing the rows with NaN as the value for HomePlay

#

so how would I return a df with all the rows where gamestats['HomePlay'] is not NaN?

#

thanks!

velvet thorn Feb 5, 2021, 7:44 AM

#

astral path so i think i know why

gamestats[gamestats['HomePlay'].notna()]

soft mango Feb 5, 2021, 7:50 AM

#

How do you color your output?

#

Also, if any of you are learning data science here is a good website to start. app.dataquest.io/referral-signup/inxant5f/

earnest forge Feb 5, 2021, 8:05 AM

#

someone could help me with syntax of keras and tensorflow, they seem having a conflict in my code

#

who can I dm about it?

heavy bay Feb 5, 2021, 9:23 AM

#

How much math do I need to know before learning tensorflow?

dusty anchor Feb 5, 2021, 9:47 AM

#

hey guys how can i convert my rgb channels into classes in tensorflow?

supple minnow Feb 5, 2021, 10:07 AM

#

Does anybody know why is plotting like this?

📎 why.png

abstract zealot Feb 5, 2021, 12:02 PM

#

Bruh show code @supple minnow

bleak fox Feb 5, 2021, 12:10 PM

#

heavy bay How much math do I need to know before learning tensorflow?

If you just wanna use TF apis as default.... 0 math.... but if you want to tune your model for better results th defiantly calculus, probability topics are required!!!

bleak fox Feb 5, 2021, 12:10 PM

#

earnest forge someone could help me with syntax of keras and tensorflow, they seem having a co...

DM Me

heavy bay Feb 5, 2021, 1:07 PM

#

bleak fox If you just wanna use TF apis as default.... 0 math.... but if you want to tune ...

Ok, Thanks 👍

supple minnow Feb 5, 2021, 1:12 PM

#

abstract zealot Bruh show code <@265947570709200896>

sry for the late reply I just notice it. Code is in the picture: sns.boxplot(x='Fjob',y='repeated',data = data)

woeful hamlet Feb 5, 2021, 1:45 PM

#

valid_datagen = ImageDataGenerator(
    rescale=1. / 255,
    validation_split=0.2)

valid_generator = valid_datagen.flow_from_directory(
    directory=data_dir, target_size=dimensions[:2],
    seed=seed, subset='validation')```

#

How can i get X_test and Y_test from this object?

#

I need it to plot confusion matrix

bold olive Feb 5, 2021, 2:22 PM

#

How exactly do you select features from an image (pixels) for subsequent input (X,y) to classifier/neural network?

austere swift Feb 5, 2021, 2:57 PM

#

normally you'd just take the pixels and put them directly into an array which you can later convert to a tensor

#

you can use something like opencv to read the image

woeful hamlet Feb 5, 2021, 3:17 PM

#

why when i use plot_confusion_matrix i get "only classifiers supported"?

astral path Feb 5, 2021, 4:44 PM

#

velvet thorn `gamestats[gamestats['HomePlay'].notna()]`

thank you! worked

elfin stream Feb 5, 2021, 6:13 PM

#

Is this also a channel I can ask for help?

#

because I needed some help with matplotlib

abstract zealot Feb 5, 2021, 6:19 PM

#

go ahead

elfin stream Feb 5, 2021, 6:22 PM

#

so whenever I use FuncAnimation, anything I return from its animate function seems to draw on top of everything

abstract zealot Feb 5, 2021, 6:23 PM

#

and you divide your axis with ```py
fig, ax = plt.subplots(<number rows>, <number columns>)

elfin stream Feb 5, 2021, 6:24 PM

#

I used fig.add_axes

abstract zealot Feb 5, 2021, 6:24 PM

#

can you show code?

elfin stream Feb 5, 2021, 6:25 PM

#

which part specifically?

abstract zealot Feb 5, 2021, 6:26 PM

#

maybe just the part where youre trying to plot?

elfin stream Feb 5, 2021, 6:26 PM

#

what I'm trying to do btw is make it not draw over everything

abstract zealot Feb 5, 2021, 6:27 PM

#

yes i know

elfin stream Feb 5, 2021, 6:29 PM

#

Idk what part still, mean my animation code?

abstract zealot Feb 5, 2021, 6:29 PM

#

you can dm me the code and i can suggest a fix if you dont want to post it here

elfin stream Feb 5, 2021, 6:29 PM

#

there is a lot of code

#

should I post the whole thing?

abstract zealot Feb 5, 2021, 6:30 PM

#

100 ?

elfin stream Feb 5, 2021, 6:30 PM

#

?

abstract zealot Feb 5, 2021, 6:30 PM

#

100 lines

elfin stream Feb 5, 2021, 6:31 PM

#

166 in total

abstract zealot Feb 5, 2021, 6:32 PM

#

you just need to show the part where you start using matplotlib, if i need more ill letcha know 😄

elfin stream Feb 5, 2021, 6:33 PM

#

the entire code is just matplotlib really

abstract zealot Feb 5, 2021, 6:33 PM

#

dm me the code then xd

elfin stream Feb 5, 2021, 6:33 PM

#

okay

lapis sequoia Feb 5, 2021, 6:37 PM

#

anyone here work with h5py files

untold raft Feb 5, 2021, 6:37 PM

#

привет

#

русских нет?

slow grove Feb 5, 2021, 6:52 PM

#

Aye could someone point me in the right direction? How could i use a pattern of data from a set of users to find similar users? At this point i don't even know what to google. I've got a pretty massive amount of users, and a decent sized subset of them that I know fit, i just need a way to find other users like them in the entire group. Sorry if this doesnt make sense

astral path Feb 5, 2021, 7:48 PM

#

if I have a dataframe with two columns and each row is an integer from 3 to 37, how could I figure out the exact number of times a specific combination of values appears in the dataframe?

#

so like col1 and col2 are the two columns i want to analyze

📎 unknown.png

#

and combo count contains the number of times a row appears

barren prism Feb 5, 2021, 7:54 PM

#

slow grove Aye could someone point me in the right direction? How could i use a pattern of ...

This sounds like a task for a typical recommender system. There are some simple algorithms for this, and advanced stuff too. Maybe look at this link for some simple starters: https://www.kdnuggets.com/2019/09/machine-learning-recommender-systems.html

KDnuggets

An Easy Introduction to Machine Learning Recommender Systems - KDnu...

Recommender systems are an important class of machine learning algorithms that offer "relevant" suggestions to users. Categorized as either collaborative filtering or a content-based system, check out how these approaches work along with implementations to follow from example code.

slow grove Feb 5, 2021, 7:55 PM

#

ty cheers m8

abstract zealot Feb 5, 2021, 8:09 PM

#

What is your combo of numbers @astral path

astral path Feb 5, 2021, 8:11 PM

#

in the example, 3 and 32

#

is that what you're asking?

abstract zealot Feb 5, 2021, 8:15 PM

#

try ```py
number = len(df[(df['col1'] == '3') & (df['col2'] == '32')])

woeful hamlet Feb 5, 2021, 8:19 PM

#

How can i use plot_confusion_matrix from sklearn.metrics with the images on an ImageDataGenerator from keras?

astral path Feb 5, 2021, 8:20 PM

#

will do! thank you

astral path Feb 5, 2021, 8:20 PM

#

abstract zealot try ```py number = len(df[(df['col1'] == '3') & (df['col2'] == '32')]) ```

although how would I apply it to an entire dataframe?

abstract zealot Feb 5, 2021, 8:21 PM

#

depends what combos youre looking for

astral path Feb 5, 2021, 8:21 PM

#

so rather than df['col1'] =='3', it would be df['col1'] == col1vals

abstract zealot Feb 5, 2021, 8:21 PM

#

yea

astral path Feb 5, 2021, 8:21 PM

#

how would that work?

abstract zealot Feb 5, 2021, 8:21 PM

#

so you want to do this for every row

#

?

astral path Feb 5, 2021, 8:22 PM

#

yeah

#

im making a scatterplot where the size of the point is dependent on the frequency that the x value and y value combination appears

#

in seaborn

abstract zealot Feb 5, 2021, 8:25 PM

#

maybe then try a different strategy like ```py
for e, i in df.groupby(by=['col1', 'col2']):
print(f'The combo {e} appears {len(i)} times ')

#

does that work?

astral path Feb 5, 2021, 8:25 PM

#

lemme check

abstract zealot Feb 5, 2021, 8:26 PM

#

instead of printing them then, just put them into a dictionary which you can use to form your plot

woeful hamlet Feb 5, 2021, 8:27 PM

#

How can i use plot_confusion_matrix from sklearn.metrics with the images on an ImageDataGenerator from keras?

abstract zealot Feb 5, 2021, 8:28 PM

#

not sure man

#

@astral path any success?

astral path Feb 5, 2021, 8:30 PM

#

yep it worked!

#

now i just have to work it into the vis

abstract zealot Feb 5, 2021, 8:31 PM

#

nice, goodluck man 😄

astral path Feb 5, 2021, 8:31 PM

#

ty!

woeful hamlet Feb 5, 2021, 9:01 PM

#

How can i use plot_confusion_matrix from sklearn.metrics with the images on an ImageDataGenerator from keras?

rancid ruin Feb 5, 2021, 9:14 PM

#

hey

#

can anybody help

#

@client.event
async def on_guild_join(user, guild, ctx):
await ctx.send(f"{user} joined {guild}!")