#databases | Python | Page 53

hallow coral Oct 8, 2018, 10:29 AM

#

just some random stuff

terse stump Oct 8, 2018, 10:30 AM

#

well.. it is good practice in any case..

hallow coral Oct 8, 2018, 10:30 AM

#

alright

terse stump Oct 8, 2018, 10:30 AM

#

how is your database connection setup?

hallow coral Oct 8, 2018, 10:31 AM

#

📎 unknown.png

terse stump Oct 8, 2018, 10:32 AM

#

so, you get the username from somewhere, you should run something like this

#

not sure your users table is called users

hallow coral Oct 8, 2018, 10:35 AM

#

Would this work or would I have to do " + input_username + "?

#

📎 unknown.png

terse stump Oct 8, 2018, 10:37 AM

#

are you using python 3.6+?

hallow coral Oct 8, 2018, 10:37 AM

#

Yup, python 3.6 😃

terse stump Oct 8, 2018, 10:38 AM

#

then you should use f-strings, not + username +

#

!t f-strings

delicate fieldBOT Oct 8, 2018, 10:38 AM

#

f-strings

In Python, there are several ways to do string interpolation, including using %s's and by using the + operator to concatenate strings together. However, because some of these methods offer poor readability and require typecasting to prevent errors, you should for the most part be using a feature called format strings.

In Python 3.6 or later, we can use f-strings like this:

snake = "Pythons"
print(f"{snake} are some of the largest snakes in the world")

In earlier versions of Python or in projects where backwards compatibility is very important, use str.format() like this:

snake = "Pythons"

# With str.format() you can either use indexes
print("{0} are some of the largest snakes in the world".format(snake))

# Or keyword arguments
print("{family} are some of the largest snakes in the world".format(family=snake))

hallow coral Oct 8, 2018, 10:39 AM

#

Ah I see, haven't used Python in ages. Thanks.

terse stump Oct 8, 2018, 10:40 AM

#

you should also consider not storing the password in the database

viral crag Oct 8, 2018, 10:40 AM

#

@terse stump wait what

#

no f-strings

#

this is a database

#

SQL injection :P

hallow coral Oct 8, 2018, 10:40 AM

#

oh well, only for me to mess around with 😃

terse stump Oct 8, 2018, 10:40 AM

#

oh.. damn.. i did not look at the screenshot

viral crag Oct 8, 2018, 10:40 AM

#

No, do it right or don't do it

hallow coral Oct 8, 2018, 10:41 AM

#

How else do you do it?

terse stump Oct 8, 2018, 10:41 AM

#

i only took the string concatination question

#

thanks gdude, my bad..

viral crag Oct 8, 2018, 10:41 AM

#

You can use the variable insertion provided by whatever SQL driver you're using

hallow coral Oct 8, 2018, 10:41 AM

#

mysql.connector

terse stump Oct 8, 2018, 10:41 AM

#

here i am trying to get you to not store password and open a SQL injection...

viral crag Oct 8, 2018, 10:41 AM

#

cursor.execute("... WHERE x = %s", (username, ))

terse stump Oct 8, 2018, 10:42 AM

#

I did link the correct way above..

#

SELECT * FROM Users WHERE name=?

hallow coral Oct 8, 2018, 10:43 AM

#

Right I've got accountdetails.execute("SELECT * FROM accountinformation WHERE username=f{input_username}") at the moment. How would I change it to the other version?

viral crag Oct 8, 2018, 10:44 AM

#

Look at my example and try to adapt it

hallow coral Oct 8, 2018, 10:44 AM

#

Alright

viral crag Oct 8, 2018, 10:44 AM

#

@terse stump I don't think the MySQL connector uses ?

#

Maybe you can use both though

#

I dunno

terse stump Oct 8, 2018, 10:45 AM

#

I think i have used it in mysql.. but then again, I do not use mysql if i do not have to

viral crag Oct 8, 2018, 10:45 AM

#

I mean, I wouldn't write SQL if I had the choice of an ORM

hallow coral Oct 8, 2018, 10:56 AM

#

📎 unknown.png

#

📎 unknown.png

viral crag Oct 8, 2018, 10:57 AM

#

I guess it does use ? instead of %s then

hallow coral Oct 8, 2018, 10:59 AM

#

That doesn't seem to work either.

full geyser Oct 8, 2018, 11:04 AM

#

This is mysqlclient? (import MySQLdb ?)

#

Ohh

#

Cuz input_username needs to be a triple

#

*tuple

#

(input username, )

#

Make sure u include the ,

#

@hallow coral

hallow coral Oct 8, 2018, 11:06 AM

#

Ah okay

full geyser Oct 8, 2018, 11:07 AM

#

Should work, if it doesn’t I have no idea 😂

#

(Make sure u use %s as the placeholder not ? for MySQLdb)

pallid zinc Oct 8, 2018, 11:23 AM

#

@terse stump sorry for late response i was on meeting

#

what I am trying to do is connect the sqlite with discord

#

so user would apply username to database

#

and than inside different commands there would be a username check

#

if user has already applied or not

#

now when I run a check command

#

it works and I get returned what it was stored in udner my username

#

however if the table is empty

#

or my username not stored in one of the rows

#

nothing is returned

#

but the idea would be that bot would send message for example

#

You need to sign in first

viral crag Oct 8, 2018, 11:25 AM

#

well yes, nothing is returned if there's nothing matching

#

you just need to check that

pallid zinc Oct 8, 2018, 11:25 AM

#

yeah

#

but i want that if there is no match in database

#

bot would respond back

hallow coral Oct 8, 2018, 12:07 PM

#

All working 😃

#

accountdetails.execute("SELECT * FROM accountinformation WHERE username=%s", (input_username, ))

rancid pier Oct 8, 2018, 1:32 PM

#

@hallow coral Just chiming in to say that this

accountdetails.execute("SELECT * FROM accountinformation WHERE username=f{input_username}")  # SUPER BAD```
is security **suicide**. Don't ever use string interpolation (`+` operator, `.format`, f-strings) to build queries, but do indeed use the % placeholders of an execute or SQL building statement of your ORM

hallow coral Oct 8, 2018, 1:33 PM

#

Yup, already been made aware of that. Thank though!

rancid pier Oct 8, 2018, 1:33 PM

#

what if input_username = "'john' OR 1=1; DROP TABLE accountinformation; --"

#

😉

#

https://www.xkcd.com/327/
the tale of Bobby Tables, any SQL database user should remember this comic by heart 😄

full geyser Oct 8, 2018, 2:43 PM

#

Aww lovely bobby tables

#

Such a sweet child

torn sphinx Oct 9, 2018, 12:26 AM

#

I'm trying to insert/update records into a table using SQLAlchemy, and my table reflection is not working with the UniqueConstraint that I have passed in, along with the PrimaryKeyConstraint as well. My code looks like this, am I doing something wrong? myTable = Table('TableName', metaTable, PrimaryKeyConstraint('fieldName1', 'fieldName2'), UniqueConstraint('fieldName1', 'fieldName2'), autoload=True, autoload_with=engineDatabase, extend_existing=True)

versed coyote Oct 9, 2018, 2:07 AM

#

mystic surge Oct 9, 2018, 2:34 PM

#

So i'm using MySQL Workbench... and i'm trying to create a datawarehouse

#

Is it possible to run a query..

#

Export Results into another database?

#

Creating it as well...

night charm Oct 10, 2018, 9:49 AM

#

I'm pretty much a noob when it comes to all this, so sorry if something is really obvious, and I'm missing it. I have a program that can output data via ODBC, so I would like to setup a MySQL server to receive the data. I plan on using Python to handle the analysis later. I downloaded the MySQL installer and installed the server. I didn't add any user accounts during the setup phase, so all I have is the root user. I see the ODBC setup in Windows' control panel, but I can't really figure out how it works or how to get it to work. Any help would be much appreciated.

Maybe I shouldn't even be using this "Data Source Administrator" because the MySQL server isn't really the data source?

night charm Oct 10, 2018, 10:45 AM

#

Nevermind! Figured a good chunk of it out on my own.

#

Just took an hour. 😃

brave grotto Oct 10, 2018, 1:01 PM

#

Heyo,
I'm doing the following query on my DB
SELECT * FROM table ORDER BY column_name DESC
The problem is that this query only retrieves 1 line.
How can I retrieve several lines ?

dull scarab Oct 10, 2018, 1:16 PM

#

Are you sure that's the issue as to why you're getting 1 line?

#

That would just mean that table only has 1 row

brave grotto Oct 10, 2018, 1:17 PM

#

here is a screenshot of my code

#

📎 unknown.png

#

And here is one of my DB

📎 unknown.png

#

the Table has 1538 lines

dull scarab Oct 10, 2018, 1:27 PM

#

📎 unknown.png

#

Does that tell you anything special ?

brave grotto Oct 10, 2018, 1:28 PM

#

...

#

oh shit

#

I'm such an idiot

dull scarab Oct 10, 2018, 1:29 PM

#

it happens :P

brave grotto Oct 10, 2018, 1:29 PM

#

--'

#

copy pasting shit from other functions

upper beacon Oct 10, 2018, 4:26 PM

#

Anyone know if python can write to an existing firebase database project?

torn sphinx Oct 10, 2018, 6:00 PM

#

if u have a valid gateway to the specific database then yes

#

but u would have to import a specific module for that ( forgot , u will need to research about that )

upper beacon Oct 10, 2018, 6:45 PM

#

@torn sphinx hmm okay thanks

tender fern Oct 10, 2018, 9:05 PM

#

Where is a great place to start with databases, just learn the basics of SQL?

heady umbra Oct 11, 2018, 12:59 AM

#

I was wondering does anyone know of a good (preferably free) file hosting site that allows uploading through python?

#

this is probably the wrong channel to ask that in

vestal apex Oct 11, 2018, 4:31 AM

#

yeah, certainly not a database question

#

Dropbox and Google Drive both offer that functionality and have some degree of free storage

#

I'm not intimately familiar with any others

rancid pier Oct 11, 2018, 6:07 AM

#

Amazon S3

full geyser Oct 11, 2018, 9:04 AM

#

Dropbox I am intimately familiar with if u want help with that

#

It’s kinda complicated if u wanna upload a file not in a single request

#

@tender fern lookup installing MySQL tutorial on Ubuntu by digitalocean

#

And then also look at digitaloceans tutorial on using MySQL

night charm Oct 11, 2018, 12:30 PM

#

Anyone ever run into an issue like this? I'm trying to add my local SQL Server as a data host, but it isn't coming up in the dropdown menu, and I can't get it to work when I type something like "localhost\MySQL80"

#

📎 unknown.png

wind nexus Oct 11, 2018, 3:48 PM

#

Im having trouble coming up with a good way to store who sent and who recieved a message because i have 2 different user classes

class Chats(db.Model):
    __tablename__ = "chats"
    id = db.Column(INTEGER, primary_key=True)
    talent_id = db.Column(INTEGER(db.ForeignKey('users.id')))
    startup_id = db.Column(INTEGER(db.ForeignKey('startups.id')))
    startup_user_id = db.Column(INTEGER(db.ForeignKey('startup_users.id')))
    messages = db.relationship('Messages')


class Messages(db.Model):
    __tablename__ = "messages"
    id = db.Column(INTEGER, primary_key=True)

boreal carbon Oct 11, 2018, 4:25 PM

#

Hey, I'm kinda getting into using python and databases together but I got a problem, could someone help me with it?

#

cursor = cnxn.cursor()

cursor.execute("INSERT INTO discordusers (username, password, rank, division, discordid) VALUES ('Testing', 'Testing2', 'Testing3', 'Testing4', 'Testing5');")
row = cursor.fetchone() 
``` this is the code and the error I'm getting is No results.  Previous SQL was not a query. But it does work within acces without python

fathom crater Oct 11, 2018, 4:27 PM

#

@wind nexus Rather than having 2 distinct types of user objects why not have 1 user class with a parameter like user_type or user_role to differentiate them.

boreal carbon Oct 11, 2018, 4:29 PM

#

using microsoft acces

fathom crater Oct 11, 2018, 4:31 PM

#

@boreal carbon What are you expecting to get from your query? It's an insertion.

boreal carbon Oct 11, 2018, 4:32 PM

#

I realised it when i heard the beep

#

okay I guess I asked too soon 😄

#

thanks regardless

fathom crater Oct 11, 2018, 4:32 PM

#

Huzzah! 😃

wind nexus Oct 11, 2018, 4:38 PM

#

@fathom crater lthe reason i have different user classes is because im using a relations and the login system is different for both,

fathom crater Oct 11, 2018, 4:41 PM

#

I'm not suggesting you don't have different behaviour but typically with a relational database you want to normalize your data as much as possible. Having 2 different user types in 2 different tables is going to make your life harder. It's just as effective to have 1 table with a isStartup value that you use in your code to switch login behaviour.

wind nexus Oct 11, 2018, 4:42 PM

#

also some people might use same email for being a user and a startup users

#

well i just did this and now i feel dirty

class Chats(db.Model):
    __tablename__ = "chats"
    id = db.Column(INTEGER, primary_key=True)
    talent_id = db.Column(INTEGER, db.ForeignKey('users.id'))
    startup_id = db.Column(INTEGER, db.ForeignKey('startups.id'))
    startup_user_id = db.Column(INTEGER, db.ForeignKey('startup_users.id'))
    sent_by_startup = db.relationship('Startup_Messages')
    sent_by_talent = db.relationship('Talent_Messages')


class Startup_Messages(db.Model):
    __tablename__ = "startup_messages"
    id = db.Column(INTEGER, primary_key=True)
    chat_id = db.Column(INTEGER, db.ForeignKey('chats.id'))
    message = db.Column(TEXT(500))
    date_sent = db.Column(DATETIME, default=datetime.utcnow)
    seen = db.Column(BIT, default = False)
    
class Talent_Messages(db.Model):
    __tablename__ = "talent_messages"
    id = db.Column(INTEGER, primary_key=True)
    chat_id = db.Column(INTEGER, db.ForeignKey('chats.id'))
    message = db.Column(TEXT(500))
    date_sent = db.Column(DATETIME, default=datetime.utcnow)
    seen = db.Column(BIT, default = False)

#

i had a user class a bit like this before :

class Users(UserMixin, db.Model):
    __tablename__ = "users"
    user_id = db.Column(INTEGER, primary_key=True)
    email = db.Column(VARCHAR(255), index=True, unique=True)
    password_hash = db.Column(VARCHAR(255)) 
    is_startup = db.Column(BIT, default=False)
    talent_details = db.relationship('Details', uselist=False, back_populates='user')
    startup = db.relationship('Startups', uselist=False, back_populates='user')
    otp_secret = db.Column(VARCHAR(255))

fathom crater Oct 11, 2018, 4:51 PM

#

All I can say is you asked for a "good way" and the good way is database normalization. Your ORM reflects the database and in this case there's no magic way to make it less "dirty" without refactoring the data first.

wind nexus Oct 11, 2018, 4:52 PM

#

hmm, thats a good point

#

maybe my current user class is too specialized for single type of users

#

well time to refactor again, lets hope i can still meet the deadline next month

#

almost all main func is still missing :/

#

im getting major impostor syndrome fibes atm

#

from myself

lilac turret Oct 11, 2018, 7:09 PM

#

Has anyone here used PonyORM with PostgresQL before? I'm having the issue where everything except object.to_json() works, which errors with pony.orm.core.PermissionError: The current user None which belongs to groups ['anybody'] has no rights to see the object User['4970937813479959015'] on the frontend

pseudo compass Oct 11, 2018, 7:10 PM

#

fix ur db settings

lilac turret Oct 11, 2018, 7:10 PM

#

I tried

#

but I can create tables and insert just fine

#

the password and username should work

pseudo compass Oct 11, 2018, 7:12 PM

#

they dont

#

owait

#

u aren't logging in with a username

lilac turret Oct 11, 2018, 7:13 PM

#

I am

#

db.bind(provider='postgres', **kwargs) with kwargs as py func(user=SETTINGS["DB_USER"], password=SETTINGS["DB_PASS"], database="Sayonika", host=SETTINGS["DB_HOST"])

pseudo compass Oct 11, 2018, 7:14 PM

#

make sure the kwargs are correct, i guess

lilac turret Oct 11, 2018, 7:15 PM

#

📎 unknown.png

pseudo compass Oct 11, 2018, 7:15 PM

#

make sure the value u passed isn't None

lilac turret Oct 11, 2018, 7:16 PM

#

it's not, I set it earlier

#

SETTINGS = {
    # Default
    "DB_HOST": "localhost",
    "DB_USER": "mart",
    "DB_PASS": "my_pass"
}

and password was set with ALTER USER mart WITH PASSWORD 'my_pass';

pseudo compass Oct 11, 2018, 7:17 PM

#

🤔 add more prints lmao

lilac turret Oct 12, 2018, 6:50 AM

#

So apparently I should use to_dict instead

night charm Oct 12, 2018, 10:15 AM

#

I have SQL Server 8.0 installed, but I can't get an instance of it to show up in Microsoft SQL Server Manager

#

Any ideas? Most of the posts I see online are made by people who apparently didn't install SQL Server 8.0

#

But I see that I have an instance running and can login and do stuff like make tables.

#

Nothing shows up in SSCM either

#

I just reinstalled Server 8.0. Same issue

mystic surge Oct 12, 2018, 1:53 PM

#

So i've got a MySQL Database for all my users about 250k rows of data (16-20 Tables)
I've got one data analyst who analyses the data using Microsoft Excel ...

We're planning on moving over to Amazon Redshift

Does anyone have any software reccommendations that can just connect to a database as a Read ONLY but have the ability to adjust/edit the data client-side only?

minor matrix Oct 12, 2018, 2:02 PM

#

Helloooo, I'm confused by your question. So you want read only perms in sql and in your front-end software adjust/edit perms?

brave grotto Oct 12, 2018, 10:35 PM

#

Python Friends, I need advices.
My bot is going to use a Database.
For now (bot is still in dev) i'm using a local SQLite db.
Do you think it's worth to use a GCP database ?
The things i'll be storing in the db are critical for my bot features

#

and i'm afraid that some sort of bug could wipe the SQLite local db

#

does the operation to a GCP db takes time ? or is rather fast (~0.2 sec) ?

#

The bot will be hosted on GCP aswell

lilac turret Oct 12, 2018, 11:51 PM

#

I recommend PostgreSQL because it's always been perfectly reliable for me, GCP not sure

brave grotto Oct 12, 2018, 11:51 PM

#

Okay

#

thx

lilac sundial Oct 13, 2018, 12:48 AM

#

The best module for MySQL is pymysql right ?

night charm Oct 13, 2018, 1:20 AM

#

I'm connecting a program to MS SQL via ODBC. The program that is sending the data to MS SQL is able to see tables on the SQL server, but all it shows are ones in the default "master" database. Is this normal? It says that there are 502 of them.

#

I can't see the "test" database, I made. Perhaps it is being crowded out by all of these tables in the "master" database?

#

They are ones like "master.sys.database_files" and "master.sys.database_audit_specification_details"

#

Tons of them

#

No, I don't think there is a limit to the number of tables I can have. I just created one in the master database, and it is accessible by the program sending the data.

#

So now the issue is that my ODBC connection only sees master tables

night charm Oct 13, 2018, 1:59 AM

#

Nevermind. I figured it out

torn sphinx Oct 13, 2018, 9:51 AM

#

@lilac sundial Yes

tender hollow Oct 13, 2018, 4:41 PM

#

HENLO Anyone use NoSQL/Document store Databases? If so which?

lilac sundial Oct 13, 2018, 6:19 PM

#

Do you guys know any modules for creating SQL statements from objects, for example sometihng like this:

dict = {col1:val1,col2:val2}
print( Statement.InsertInto(dict,"my_db.mytable") ) 
----------------------------------------------------------------
 INSERT INTO my_db.mytable
(`col1`,`col2`)
VALUES
('val1','val2')```

torn sphinx Oct 13, 2018, 8:41 PM

#

google thought they slick hiding the db

#

https://gyazo.com/2ce2cf456261db8c8eac95e47a7eb75b

Gyazo

analog forge Oct 13, 2018, 9:00 PM

#

Anyone have any idea how to install scrapy

#

?

#

Do I need to mess with my environmental path if I’m using conda ?

#

Please dm me thanks

broken linden Oct 13, 2018, 9:03 PM

#

!t no-dms

delicate fieldBOT Oct 13, 2018, 9:03 PM

#

Tag not found.

no-dms is an unknown tag name. Please check the spelling and try again.

broken linden Oct 13, 2018, 9:03 PM

#

!t no-dm

delicate fieldBOT Oct 13, 2018, 9:03 PM

#

no-dm

Can I send you a private message?

No. We do not provide one-on-one tutoring - you can hire someone locally if you really need that. We also prefer that questions are answered in a public channel as it means that everyone else present is able to learn from them. If you're working with code that you are unable to disclose for any reason, you should try to make your question more general and write a separate, small piece of code to illustrate your problem.

broken linden Oct 13, 2018, 9:04 PM

#

furthermore, why dont you just try the normal way and use pip? @analog forge

and even more interesting, why is this related to #databases ?

chilly mesa Oct 13, 2018, 11:40 PM

#

i am facing a problem with pysqlcipher3
i downloaded the clone from github
and i did this
python setup.py build
from terminal
and i have this error: command 'cl.exe' failed: No such file or directory

#

windows 10

torn sphinx Oct 14, 2018, 9:35 AM

#

CL is Visual Studio's compiler from what I remember, I assume you'll to download something that embeds it. Your best bet is to download the VS20xx compiler

night charm Oct 14, 2018, 4:09 PM

#

Any thoughts on pyodbc vs pypyodbc?

naive anvil Oct 14, 2018, 8:33 PM

#

Hey I ran this sql command on sqlite 3

#

f"""SELECT * FROM people WHERE strftime('%m', birth_day) = '{month}'"""

nova hawk Oct 14, 2018, 8:33 PM

#

why are there so many """

naive anvil Oct 14, 2018, 8:33 PM

#

It doesn't give me the results in order

#

its and sql command

#

shouldn't it be like that

#

it works.. it just doesn't give it to me in order.

nova hawk Oct 14, 2018, 8:35 PM

#

you can specify the order

#

where you have *

naive anvil Oct 14, 2018, 8:35 PM

#

what do you mean?

nova hawk Oct 14, 2018, 8:36 PM

#

Select id, name from users

naive anvil Oct 14, 2018, 8:38 PM

#

[('Name 1', 'male', '2005-10-30', 'Null', '00-00-00'), ('Name 2', 'male', '2005-10-04', 'Null', '00-00-00'), ('Name 3', 'male', '2005-10-12', 'Null', '00-00-00')]

#

This is the output it gives when the month is 10

#

I want it to give it out in the order of date.. for example:

nova hawk Oct 14, 2018, 8:39 PM

#

ORDER BY clause

#

ORDER BY name of date field ASC

naive anvil Oct 14, 2018, 8:39 PM

#

oh lemme try that

#

it works!

#

thanks

#

i though there had to be a clause in the WHERE statement that puts it in order

nova hawk Oct 14, 2018, 8:41 PM

#

if you replace ASC with DESC it sorts in the other direction

naive anvil Oct 14, 2018, 8:41 PM

#

thanks for helping me out 😄

nova hawk Oct 14, 2018, 8:41 PM

#

no problem

slate night Oct 15, 2018, 8:38 AM

#

    for column in Cursor.execute ("SELECT * FROM Playersname WHERE username = ?",(Name)"AND password = ?",(password)";"):
                Cursor.execute("sql query"(1)= Name,(2)=Password)```

#

i am trying to insert the variable name into the ? of the dataase query for the Username and i am trying to insert the Variable password into password

#

can someone tell me how to do this

#

it isnt working at all

full geyser Oct 15, 2018, 8:44 AM

#

MySQL?

slate night Oct 15, 2018, 8:44 AM

#

sql lite

#

@full geyser

full geyser Oct 15, 2018, 8:44 AM

#

Ah k nvm idk :/ i’d try and help but I don’t trust myself on 0h sleep 😂

slate night Oct 15, 2018, 8:45 AM

#

okay thanks anyway

chrome fjord Oct 15, 2018, 8:53 AM

#

Hello,
I just encounter something weird in a MySQL db: image directly stored within the dabatase. I mean, not the path to the image, the actual data of the image
Am I missing some genius or the person who did that is a total dummy?

dull scarab Oct 15, 2018, 9:04 AM

#

@slate night That's not quite what i suggested you do last time we went over it.

slate night Oct 15, 2018, 9:05 AM

#

I forgot what you said XD

#

I had to go after you did and I forgot what chat it was

dull scarab Oct 15, 2018, 9:05 AM

#

firstly you weren't supposed to directly write "sql query"

slate night Oct 15, 2018, 9:06 AM

#

Ohhh I just copied and pasted what you said I thought you said to do that

dull scarab Oct 15, 2018, 9:07 AM

#

We don't generally do the work for you.

#

The query is what you had before

slate night Oct 15, 2018, 9:07 AM

#

I know I thought you just said that I know I want to learn

dull scarab Oct 15, 2018, 9:07 AM

#

but the ? i had within the query were meant to be ? s

#

as they are placeholder symbols in sqlite (for python interaction) at least

slate night Oct 15, 2018, 9:08 AM

#

Yeah then how do I replace them

dull scarab Oct 15, 2018, 9:08 AM

#

so you could pass Name and Password as parameters to execute

#

to fill those placeholders

slate night Oct 15, 2018, 9:08 AM

#

That’s what I am stuck on

#

Just replacing the placeholders

dull scarab Oct 15, 2018, 9:08 AM

#

execute("sqlstuff ... name = ?", Name)```

#

Here the first ? gets replaced with the value of Name

slate night Oct 15, 2018, 9:08 AM

#

OHHHHH

#

Then what about password same thing

chrome fjord Oct 15, 2018, 10:02 AM

#

Hello,
I just encounter something weird in a MySQL db: image directly stored within the dabatase. I mean, not the path to the image, the actual data of the image
Am I missing some genius or the person who did that is a total dummy?

full geyser Oct 15, 2018, 5:36 PM

#

He’s a dummy

#

Just because you can do it doesn’t mean u should 😂

#

@chrome fjord

fathom crater Oct 15, 2018, 6:08 PM

#

Be careful about casting stones at other devs, you don't know the particulars. There are reasons (https://www.microsoft.com/en-us/research/publication/to-blob-or-not-to-blob-large-object-storage-in-a-database-or-a-filesystem) (admittedly old) that someone might have chosen to store binary data in a database. I'm not saying @chrome fjord's dev made a good or bad choice but in the absence of information let's encourage systematic review over maligning someone else's intellect.

Microsoft Research

Jim Gray

To BLOB or Not To BLOB: Large Object Storage in a Database or a Fi...

Application designers often face the question of whether to store large objects in a filesystem or in a database. Often this decision is made for application design simplicity. Sometimes, performance measurements are also used. This paper looks at the question of fragmentatio...

full geyser Oct 15, 2018, 6:16 PM

#

To blob or not to blob lmao 😂

#

reminds me of google noto's "The blobs are dead. Long live the blobs!"

torn sphinx Oct 15, 2018, 6:31 PM

#

I usually store BLOBs too

full geyser Oct 15, 2018, 6:31 PM

#

why?

torn sphinx Oct 15, 2018, 6:32 PM

#

More scalable on multi-server instances

#

And most db system feature optimizations for storing BLOBs, so that was never much of an issue, except the size of the DB maybe

full geyser Oct 15, 2018, 6:39 PM

#

hmmm

torn sphinx Oct 15, 2018, 6:44 PM

#

: )

mystic surge Oct 16, 2018, 10:23 AM

#

So i've gotta export a database MySQL into Redshift JDBC every night

I was hoping to do this all with python?
Is it possible to run a python script get all table names then for each table get the data structure?

#

as well as sometimes get the data

#

Want the structure mostly incase the tables change over time

slate night Oct 16, 2018, 11:20 AM

#

Hey @dull scarab just got to my pc

#

i added that thing you said and it worked now getting this error

#

line 15, in querying
    for column in Cursor.execute ("SELECT * FROM Playersname WHERE username = ?",Name, "AND password = ?",Password):
TypeError: function takes at most 2 arguments (4 given)```

#

for column in Cursor.execute ("SELECT * FROM Playersname WHERE username = ?",Name "AND password = ?",Password):

dull scarab Oct 16, 2018, 11:21 AM

#

they are to passed as a tuple at the end, without splitting up the query iirc

slate night Oct 16, 2018, 11:21 AM

#

thats the line

#

that just went straight over my head

#

tuple? iirc?

dull scarab Oct 16, 2018, 11:21 AM

#

("query eruy stuff = ? and thing = ?", (123, "abc")```

#

(1, 2) tuple of 1 and 2

#

iirc, if i remember correctly

slate night Oct 16, 2018, 11:22 AM

#

ohh so at the end

#

oh great that worked

#

so how can i do if the password is wrong it prints invalid password

#

@dull scarab

dull scarab Oct 16, 2018, 11:31 AM

#

Technically, for security reasons you shouldn't give any different response if either username or password is incorrect. But if you want to do it remove password from the query, fetch the username and check manually if the password matches

slate night Oct 16, 2018, 11:32 AM

#

but what if the user wants to know if there password was correct or incorrect?

#

wasnt that how i was doing it before

viral crag Oct 16, 2018, 11:33 AM

#

They shouldn't know

#

Because they'll be able to tell that the user they tried exists

slate night Oct 16, 2018, 11:34 AM

#

ahhhh okay

#

thats ok thanks guys for your help

#

i may ask for more later on but i am ok for now

sick dawn Oct 16, 2018, 11:51 AM

#

Database with python

#

REALLY?

viral crag Oct 16, 2018, 11:53 AM

#

Why not?

#

@sick dawn

sick dawn Oct 16, 2018, 11:57 AM

#

This... some strange

viral crag Oct 16, 2018, 12:01 PM

#

Pretty common task in python

mystic surge Oct 16, 2018, 12:21 PM

#

So i can get list of my table in tuples

#

is it possible to convert to a string list?

nova hawk Oct 16, 2018, 12:23 PM

#

You can use a string like a list in of one character strings in a lot of places

mystic surge Oct 16, 2018, 3:18 PM

#

Is it possible to export MySQL Tables like this:

DROP TABLE IF EXISTS broadcategory CASCADE;

CREATE TABLE broadcategory
(
   broadcategoryid  integer          NOT NULL,
   value            varchar(65535)
);

COMMIT;

Instead of like this:

 CREATE TABLE `broadcategory` (
`BroadCategoryID` int(11) NOT NULL AUTO_INCREMENT,
`Value` varchar(15) DEFAULT '',
PRIMARY KEY (`BroadCategoryID`)
ENGINE=InnoDB AUTO_INCREMENT=11 DEFAULT CHARSET=latin1

mystic surge Oct 16, 2018, 3:43 PM

#

Is it possible to have something in python a long the lines of

#

For x in TableName
print x.column and x.type ?

nova hawk Oct 16, 2018, 3:44 PM

#

you can get results as a dict i think

#

if you do the thing that gets you tuples you could do
for column, type in result

neat reef Oct 16, 2018, 3:57 PM

#

is value a keyword in MySQL?

patent glen Oct 16, 2018, 3:58 PM

#

sql syntax highlighting is weird

neat reef Oct 16, 2018, 3:58 PM

#

i was just asking

patent glen Oct 16, 2018, 3:58 PM

#

i think there are words that have a meaning in some contexts that get highlighted because the syntax parser isn't smart enough

neat reef Oct 16, 2018, 3:58 PM

#

because i named a column value and now i get syntax error

patent glen Oct 16, 2018, 3:59 PM

#

oh then probably

#

just use "value" in double quotes

neat reef Oct 16, 2018, 3:59 PM

#

you mean \value``?

#

i can't

patent glen Oct 16, 2018, 3:59 PM

#

no i meant double quotes

#

` works too in mysql but isn't part of the ansi sql standard so i tend not to recommend it

neat reef Oct 16, 2018, 3:59 PM

#

but value is a column, not a string

patent glen Oct 16, 2018, 3:59 PM

#

strings are in single quotes

#

some sql engines let you sometimes use double quotes for strings, but use as a column/table name takes priority

#

because ansi

#

(but, sure, use `value` if you want

neat reef Oct 16, 2018, 4:01 PM

#

ok

patent glen Oct 16, 2018, 4:01 PM

#

(surround with `` and spaces to code-quote something containing ` in markdown)

neat reef Oct 16, 2018, 4:02 PM

#

oh wait, my problem is with key, not value

patent glen Oct 16, 2018, 4:02 PM

#

that makes more sense, since key is definitely a keyword

#

well maybe

#

oracle doesn't care, oddly enough

#

i think the oracle parser may be more contextual than the mysql one

neat reef Oct 16, 2018, 4:03 PM

#

oh well, thanks anyways

patent glen Oct 16, 2018, 4:05 PM

#

As an aside, to explain what i'm talking about: there are two ways to do parsers for languages that have "keywords" (i.e. identifier-like words that have a special meaning in the syntax) - non-contextual (the lexer classifies the keyword immediately, and it's banned from use as an identifier for everywhere) or contextual (the keyword is only recognized where it makes sense - note that this does not mean a language cannot be a "context free grammar")

neat reef Oct 16, 2018, 4:06 PM

#

i got chu

mystic surge Oct 16, 2018, 4:36 PM

#

@nova hawk ill try with tuples

#

As of now - I've left work
But I've managed to search for the though all types

#

Convert most to varchar

#

Just gotta create a create a query with it for create table

torn sphinx Oct 17, 2018, 6:17 AM

#

http://katski.org/facegen.html

#

Hello everyone, I could use some responses for my data science project

viral crag Oct 17, 2018, 6:19 AM

#

I think that was a near-miss with your click there, #data-science-and-ml :P

#

Maybe explain what your project is about as well

pure cypress Oct 17, 2018, 6:20 AM

#

well it just looks like a survey

viral crag Oct 17, 2018, 6:20 AM

#

It is a survey

mystic surge Oct 17, 2018, 9:15 AM

#

So i have this:

    print("""DROP TABLE IF EXISTS %s CASCADE;
\nCREATE TABLE %s 
(""" % (x,x))
    for x,y in zip(tableFields, Tabletype):
        if(x == (tableFields[-1])):
            print("  "+x+"      "+y)
        else:
            print("  "+x+"      "+y+",")
    print(");\n\n")

#

DROP TABLE IF EXISTS usertoken CASCADE;

CREATE TABLE usertoken
(
  UserTokenID      int(11),
  UsersID      int(11),
  Token      varchar(250),
  Expiry      datetime
);

#

Which gives me that...

#

How can i make that into a Cursor.execute?

pure cypress Oct 17, 2018, 9:21 AM

#

store it as a string instead of printing it

#

then cursor.executescript(my_sql_string)

#

well, that's what it would be for sqlite at least

mystic surge Oct 17, 2018, 9:24 AM

#

How?

#

there's several tables

#

would that be a case of creating a list of strings?

#

stringone[0].append
stringone[0].append
stringone[0].append
Stringone[1].append
Stringone[1].append
Stringone[1].append

#

sorta thing?

pure cypress Oct 17, 2018, 9:46 AM

#

you could create a list of strings or you could just append all to 1 string

mystic surge Oct 17, 2018, 10:42 AM

#

i just did

query_list = [""] * len(str_tables)

wind nexus Oct 17, 2018, 10:45 AM

#

Whats the best data format use on mssql when storing output time.time()?

mystic surge Oct 17, 2018, 10:47 AM

#

Does anyone know how to get SELECT * from table
results

#

so its like

#

INSERT INTO c1,c2,c3,c4 ('3232','434','33','4')

fathom crater Oct 17, 2018, 1:45 PM

#

? Can you rephrase that question.

unique mauve Oct 17, 2018, 1:49 PM

#

Sound like your looking for SELECT INTO @mystic surge

mystic surge Oct 17, 2018, 2:13 PM

#

Thanks @unique mauve Ted

proud stirrup Oct 17, 2018, 9:22 PM

#

hey all

#

is it possible to do a mysql query to find the latest table thats currently or has been updated without providing a specific table name?

brave grotto Oct 17, 2018, 11:47 PM

#

Which module do you guys uses for Postgres DBs ? psycopg2 or pygresql ?

dull scarab Oct 18, 2018, 4:53 AM

#

Asyncpg 02peek

brave grotto Oct 18, 2018, 7:40 AM

#

oh

dull scarab Oct 18, 2018, 9:20 AM

#

But i've mostly only used it in an asynchronous environment

barren glacier Oct 18, 2018, 10:43 AM

#

I'm trying to connect to a local mongodb database, but apparently I can't even get the simplest script to work properly. The error output is AttributeError: 'Post' object has no attribute '_is_document', but that must be from the mongoengine library and I don't know how to fix it.

from mongoengine import *

connect('test')

class Post(Document):
  title = StringField(max_length=120, required=True)
  author = StringField(max_length=120, required=True)
  tags = ListField(StringField(max_length=30))

post1 = Post(title="Hello, world!", author="John", tags=["flask", "mongoengine"])
post1.save()

brave grotto Oct 18, 2018, 11:34 AM

#

@dull scarab Do you know how to use SSL with AsyncPG ?

tawny lodge Oct 18, 2018, 11:47 AM

#

@barren glacier your code is working for me what python do you use?

barren glacier Oct 18, 2018, 11:47 AM

#

3.7

tawny lodge Oct 18, 2018, 11:47 AM

#

installed mongoengine with pip3 ?

barren glacier Oct 18, 2018, 11:48 AM

#

yes. What version of mongodb do you use?

#

Come to think of it, that can't be the problem...the error comes from mongoengine

tawny lodge Oct 18, 2018, 11:51 AM

#

tested it on python 3.5.3 and mongoDB 3.2.11

barren glacier Oct 18, 2018, 11:51 AM

#

Thanks, I'll setup a new environment and test it again. If the code is alright, the problem must be on my side. Thanks for checking

tawny lodge Oct 18, 2018, 11:56 AM

#

what version is your mongoengine btw ?

#

mongoengine had some issues with python 3.7 so maybe just upgrading mongoengine will get it working for you

barren glacier Oct 18, 2018, 11:59 AM

#

I've just installed mongoengine yesterday, must be 0.15.3 then

#

pymongo is 3.7.1 and six is 1.11

tawny lodge Oct 18, 2018, 12:01 PM

#

got the same except pymongo is 3.7.2 for me

barren glacier Oct 18, 2018, 12:05 PM

#

Alright, it works for me with python 3.5.3. I'll test it with 3.6.6 now, see how that goes. Must have been the python version then. Thanks for your help, Quiz
edit: python 3.6.6 works fine as well

tawny lodge Oct 18, 2018, 12:05 PM

#

no problem

lilac turret Oct 18, 2018, 12:29 PM

#

has anyone here used PonyORM before? having issues with many-to-many relations

humble oyster Oct 18, 2018, 5:50 PM

#

For MongoDB (using Motor + PyMongo as the client) a way to pass a custom function in the find() call? My use case is doing fuzzy matching for a specific field and currently I'm doing the bad thing of loading everything to do the comparisons.

west lark Oct 18, 2018, 7:29 PM

#

Hey how can I sort a query by number of child rows? I have a Product that has offers, each offer has it's own table. I'm displaying those list of products on a html table , and the amount of offer available for a product. I want the user to be able to sort that table by clicking on the header. I want to be able to do something like Product.query.order_by(len(Product.offers)) , and sorting by ascending or descending. I'm also using pagination.

#

I'm using Flask + Sqlalachemy

fallow wagon Oct 18, 2018, 7:53 PM

#

anyone have an idea how to produce a counter such as A A B D E T A into 1 1 2 3 4 5 1. so that it group and do counter on each group
I am currently working with panda in spyder
i need this to run loop through the data frame for each group. please ping me if you haven an answer
I don't know how to attempt this problem because the most syntax i know that might work so far is df1[:,'counter']=df1.groupby('class').cumcount() but it's not the right answer to it

sharp jacinth Oct 19, 2018, 4:05 PM

#

I have a question and I want to see if I understand these concepts correctly in a metaphorical sense:
We have a relational database on cakes
The domain would be the cake store
The attributes Columns could be anything from cake ingredients, price of cakes, location of store
The values would be the exact ingredients of the cake, or the exact price of the cake or the exact location of the cake store.
Now the relational part would be connected through any of these domains or attributes or attributes and if we wanted to find out more info on these domains or attributes or values we can go to another table interconnected to one of these and find out the prices and/or other names of cake stores within that table?

hallow jetty Oct 19, 2018, 5:01 PM

#

Your questions is a little hard to understand, can you rephrase it? Break it up so it's easier to swallow and such

sharp jacinth Oct 19, 2018, 5:12 PM

#

Okay, will do when I’m at a computer

#

Actually I just want to make a comparison between relational databases and a metaphor:
Our relational database is what holds all of our data and can be referenced to another table right?
Our metaphor:
Our earth is equivalent to the domain of a relational database
A human on earth is equivalent to an attribute of a relational database
Height, weight, name, etc. of the human . is equivalent to a value in a relational database

That humans friends or weight values or the type of planet they live on could be related to another table that points to another humans weight, friends or planet they live on?

#

@hallow jetty

hallow jetty Oct 19, 2018, 5:30 PM

#

That sounds right tbh

#

There's probably someone that knows more than me that will pop in though

sharp jacinth Oct 19, 2018, 5:32 PM

#

Okay, phew. Thank you!

primal idol Oct 19, 2018, 7:10 PM

#

Hello. I am beginner in python and learning it since a week now. At the same time i am learning i am trying to develop at the same time.
For now, i am at the step of reading and extracting data from a json file. And i want to stock my extraction in DB for future usage (web dev).
What would be the best ? Mysql or mamgonosql (with BSON) ?
I have read the differences in theory but i wanted to know your experiences with those,

full geyser Oct 20, 2018, 12:17 AM

#

MySQL

#

Idk what bson is tho

versed coyote Oct 20, 2018, 3:40 AM

#

BSON is Binary JSON

primal idol Oct 20, 2018, 6:21 AM

#

Ok thank you

sage fable Oct 20, 2018, 10:25 AM

#

test

full geyser Oct 20, 2018, 11:33 PM

#

📎 unknown.png

#

😂 😂

marsh knot Oct 21, 2018, 3:55 AM

#

amazing

indigo goblet Oct 21, 2018, 9:42 AM

#

Is there any way to make a .mdb file with sqlite3?

glossy ermine Oct 22, 2018, 12:06 AM

#

@indigo goblet mdb is for access, what are you trying to do?

winter robin Oct 22, 2018, 9:53 AM

#

what's better for a messenger - Scylla or Postgres?

full geyser Oct 22, 2018, 12:36 PM

#

They seem to be pretty different things...

#

What u want it for specifically?

winter robin Oct 22, 2018, 2:10 PM

#

Storing huge amounts of messages (history) with optional images, videos and other embeds, servers+settings, people+settings,etc., you might've got the hint

full geyser Oct 22, 2018, 4:12 PM

#

for storing messages and stuff, id use postgres

#

include like a time with each message, and groupid and userid

#

and you can search for stuff really quickly

#

maybe like a contenttype, so if its a video, the text is the path to the image or something

#

thats how ive done mine

lavish ferry Oct 23, 2018, 12:46 AM

#

Hi, how can I host json file in heroku postgress ?

alpine fox Oct 23, 2018, 12:46 AM

#

what is the best database for storing food like if i want to make a yogurt api for school

alpine fox Oct 23, 2018, 2:30 AM

#

someone pls @ me

dull scarab Oct 23, 2018, 4:17 AM

#

Depends how big this db is to be I suppose

#

If its through django / flask they probably haave their own database systems you could use

winter robin Oct 23, 2018, 5:09 AM

#

@full geyser "you know, like discord", and they use Cassandra

winter robin Oct 23, 2018, 10:41 AM

#

aherm

eager nest Oct 23, 2018, 3:43 PM

#

Hey is this channel open?

#

I got my script to work with SQLite but switched to MySQL and am getting an error now.

in defaulterrorhandler
raise errorclass(errorvalue)
ProgrammingError: not all arguments converted during string formatting

which I get from this line:

c.execute(PutExcelInSQL, myrow)

and to make that make sense, I'm going to paste in a little more of that block so you can see how I formatted my SQL and what myrow looks like (I'm iterating through rows of excel worksheets in workbooks)

PutExcelInSQL = '''INSERT INTO Raw_Data(ID,Peptide,Peaks_Probability_Score,ppm,m_over_z,RT,Scan,Accession,PTM,Mass,pI,Length,Aliphatic_Index,Net_Charge,Hydropathy,Charge_Per_Residue,SVM_Class,SVM_AMP_Prob,RF_Class,RF_AMP_Prob,DA_Class,DA_AMP_Prob,Filename,Myrowid,UniqueID,Species,Date_Raw_Data_Acquired,Location,Author) VALUES (?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?)'''

if DBColumnCheck(myrow) is True:
    try:
        c.execute(PutExcelInSQL, myrow)

ionic pecan Oct 23, 2018, 4:57 PM

#

are you sure that you have the same amount of params as items in myrow?

eager nest Oct 23, 2018, 4:58 PM

#

I'll double check, but I did count those before

#

29 values, 29 question marks

#

tried to run it again, same issue.

ionic pecan Oct 23, 2018, 5:07 PM

#

and row is a tuple of values, right?

#

wait, does mysql even use ? for params

#

wasn't it %s in mysql?

eager nest Oct 23, 2018, 5:08 PM

#

myrow? myrow is a row in excel

#

that it, that I can't use '?' ?

#

if I change ? to %s, is that going to make it vulnerable to that injection attack thing.....

ionic pecan Oct 23, 2018, 5:09 PM

#

no, not if you do it properly

#

properly as in not by using 'INSERT INTO whatever VALUES (%s)' % values but rather 'INSERT INTO whatever VALUES (%s)', values

eager nest Oct 23, 2018, 5:10 PM

#

ok it is set up the latter way

#

I'll try using %s

ionic pecan Oct 23, 2018, 5:12 PM

#

let me know how it turns out

eager nest Oct 23, 2018, 5:13 PM

#

oh ok I get a different error now 😅

ionic pecan Oct 23, 2018, 5:16 PM

#

do show

eager nest Oct 23, 2018, 5:17 PM

#

OperationalError: (1366, "Incorrect Integer value: '>0' for column 'ID' at row 1")

#

so I look at that and think I told SQL to make the first column primary key autoincrement but I need to add a column there for that to go in because it's trying to do that with my first column of data from excel

ionic pecan Oct 23, 2018, 5:28 PM

#

im surprised mysql gives you an error for incorrect values

#

so you want to drop the first row from excel or the first column?

eager nest Oct 23, 2018, 5:29 PM

#

I'm sorry?

#

I don't want to drop anything

ionic pecan Oct 23, 2018, 6:28 PM

#

i thought what you were suggesting was that you just want to use the autogenerated id

eager nest Oct 23, 2018, 7:38 PM

#

What I want is to fix each error as they come up until this works :P

I want to keep the ID column from excel. I want to be able to insert the entire row of data, row by row. What I have is set up to do that. (just, switching from sqlite to mysql and it will work)

I think what I need to do is put a column in first, for the primary key. then make sure my data gets inserted after that. right?

#

wait a second let me try something

#

why do making changes to databases take FOREVERRRRRRRRr. I want to be coming back here saying "Hey I fixed it" but I can't do that until I can see if the change I made fixes it.

patent glen Oct 24, 2018, 3:39 AM

#

@eager nest just leave the id column out of the insert query?

#

i.e. insert into Raw_Data(Peptide,...)

tidal apex Oct 24, 2018, 12:38 PM

#

has anyone used Redis as a database ?

#

its for my web game server

#

its only gonna be storing some highscores and stuff

#

seems like key-value store like redis might be enough

#

thinkmon

quiet ermine Oct 24, 2018, 3:54 PM

#

what is the best database server for security? Does not need to be great speed wise.

#

someone said PostgreSQL but wondering if there is anything better? (never done databases)

cloud fractal Oct 24, 2018, 4:04 PM

#

https://stackoverflow.com/questions/52973196/pypyodbc-using-stored-procedures-and-storing-data-into-a-dataframe?noredirect=1#comment92852915_52973196

Stack Overflow

pypyodbc - using stored procedures and storing data into a dataframe

So i have a stored procedure that i want to connect to and execute in python so i can export the data as a fancy table.

Here is the code im using, with the comments being code that i have tried.
...

#

need help

ionic pecan Oct 24, 2018, 4:37 PM

#

@quiet ermine depends on your requirements

#

postgres has a pretty extensive security model, i think even row-level

quiet ermine Oct 24, 2018, 4:38 PM

#

basically as secure as possible without it taking forever to process, aes256/sha256 ideally (forgot what one you need for this)

#

sorry if phrasing is a bit off, new to databases & just starting out with django

ionic pecan Oct 24, 2018, 4:39 PM

#

do you want an encrypted data directory?

quiet ermine Oct 24, 2018, 4:41 PM

#

no idea

#

could always run it in a vm in the server

#

(will only ever be acessed by 1 website)

ionic pecan Oct 24, 2018, 4:47 PM

#

i still have no idea what you‘re asking

#

do you want to restrict access to the database? encrypt the data?

quiet ermine Oct 24, 2018, 4:48 PM

#

to be honest, i dont

#

😅

ionic pecan Oct 24, 2018, 4:53 PM

#

then i‘m not sure how to answer your question, the general go-to nowadays is postgres

quiet ermine Oct 24, 2018, 5:01 PM

#

👍

gleaming frost Oct 24, 2018, 5:15 PM

#

@quiet ermine there are 2 types of security volcyy mentioned

Retricting access is more about your servers security: (Who can access it, network security etc)

Data security is encrypting the data so that its useless without the correct keys

#

saying you want it to be "secure" is a little vague, as there are so many levels of "secure"

alpine fox Oct 25, 2018, 1:31 AM

#

i like to use random library to randomize every byte of my database to make sure no one can read

quiet ermine Oct 25, 2018, 10:29 AM

#

Not completely secure

#

Use sha256 with a 1 time pad ;)

fresh vale Oct 25, 2018, 12:53 PM

#

Hi

#

I want to query database using SQLalchemy

#

Someone know why print(User.query.filter_by(username=next(iter(session))).first()) dosen`t work?

gusty spindle Oct 25, 2018, 1:30 PM

#

(this has been answered in #help-coconut)

wind nexus Oct 25, 2018, 4:28 PM

#

Can someone take a look at my question at #help-grapes

west lark Oct 25, 2018, 5:00 PM

#

Does anyone knows why this is happening? I have a account model and a account info model. When I iterate over all accounts, and try to render the account info as account.info it works, but when I try to render a column it doesnt render on the template

📎 QUESTION.png

#

Using flask and sqlalchemy

upper beacon Oct 25, 2018, 10:26 PM

#

anyone here familiar with linking a python project to an existing firebase project

glossy ermine Oct 26, 2018, 12:45 AM

#

@west lark Looks like its a list rather than a single object

keen cedar Oct 26, 2018, 2:52 PM

#

So, I have a feature that adds a row of data every time someone uses it to a table in my PostreSQL database, but I want to make it so that each new added row is only kept there for 15 days and is then erased. How should I go about implementing it? Add a new "date_of_creation" column, include the day it was added to the database and have the bot loop through every single entry in that table every midnight and delete ones that have been added 15 days ago?

dull scarab Oct 26, 2018, 3:36 PM

#

You could have it loop through say every hour and looks for ones that are to be deleted in less than 1 hour and schedule a task for it

#

Unless it doesnt have to be very accurate then just check evry hour and delete

#

Dont even have to check, could just be a query deleting on creation date conditions

boreal carbon Oct 26, 2018, 4:52 PM

#

cursor.execute("SELECT username FROM discordusers WHERE username=? GROUP BY username", (e[0])) pyodbc.Error: ('07002', '[07002] [Microsoft][ODBC Microsoft Access Driver] Too few parameters. Expected 2. (-3010) (SQLExecDirectW)')

#

i've been getting this erro

#

but it wasn't there before

ionic pecan Oct 26, 2018, 5:02 PM

#

you're not passing a proper tuple

#

you're just passing e[0]

#

add a trailing ,

boreal carbon Oct 26, 2018, 5:08 PM

#

I fixed it nvm

#

apperently someone changed a database column title

#

got me all confused

faint blade Oct 27, 2018, 8:13 AM

#

So i am using asyncpg. And i am working on a punishments feature. I need to save a datetime.datetime object into the db, how would i do this? I dont know what o put in the Table etc.

#

Like, when i create my table

gleaming frost Oct 27, 2018, 11:12 AM

#

@faint blade you could try and store the object as a string of bytes

faint blade Oct 27, 2018, 11:13 AM

#

That might work. Although i found another way to save it "CURRENT_TIMESTAMP"

#

The part in the table: StartTime TIMESTAMP,

gleaming frost Oct 27, 2018, 11:14 AM

#

Ok

neon turtle Oct 27, 2018, 3:52 PM

#

TIMESTAMP datatype

quartz salmon Oct 28, 2018, 12:47 AM

#

Hey, I was told to ask my question here.

#

If I want to combine 35 CSVs that have somewhat different column names, what's the best way to do it?
I'm trying to do it with "usecols" from Pandas, but when there aren't column names that I'm specifying within a CSV, it spits an error.
Here's an example

                                                                   'COL2',
                                                                   'COL3',
                                                                   'COL4',
                                                                   'COL5',)
    for f in os.listdir(os.getcwd()) if f.endswith('csv')]

combexample = pandas.concat(example, axis=1, join='inner').sort_index()```


So, some of the files might not have COL4 (it's technically the same info within the row, but the header name is different). How can I get it to combine properly?
I'd love any help I can get.
If it helps, there are 40 total files.

patent glen Oct 28, 2018, 12:55 AM

#

@deft badge can you just get all columns, and filter after you've got the data in the dataframe?

#

or if the columns are all in the same order, you can use numbers for usecols

quartz salmon Oct 28, 2018, 12:59 AM

#

So there are two separate blocks of data - the order of the columns (and some of the names) changes between the two, but within each block they're in the same order.

patent glen Oct 28, 2018, 12:59 AM

#

ok so what i'd do is probably break your list comprehension out into a loop, for one thing

#

just do a plain read_csv (without use_cols)

quartz salmon Oct 28, 2018, 1:00 AM

#

So something like this?

#

example = [pandas.read_csv(f, index_col=[0], parse_dates=[0], engine='python')
for f in os.listdir(os.getcwd()) if f.endswith('csv')]

patent glen Oct 28, 2018, 1:01 AM

#

like i said, i would not do this as a comprehension

quartz salmon Oct 28, 2018, 1:02 AM

#

Sorry, I know I'm being dense. I'm relatively new to this - apologies if I misunderstand you.

patent glen Oct 28, 2018, 1:05 AM

#

something like

example = []
for f in os.listdir...:
    df = pandas.read_csv(f, index_col=[0], parse_dates=[0], engine='python')
    if all(x in df.columns for x in ['COL2', 'COL3', 'COL4', 'COL5']):
        df = df[['COL2', 'COL3', 'COL4', 'COL5']] # COL1 is index
    elif: #another set of columns...
        ...
    else:
        raise Exception unknown csv format
    example.append(df)```

quartz salmon Oct 28, 2018, 1:06 AM

#

So index_col is actually an empty column?

patent glen Oct 28, 2018, 1:06 AM

#

huh?

#

i'm not sure what you mean by that

#

you're specifying index_col as the first column

quartz salmon Oct 28, 2018, 1:07 AM

#

Is it literally taking index_col as the 1st column, or creating one -

#

got it

patent glen Oct 28, 2018, 1:07 AM

#

(is this always the first physical column? the situation gets hairier if it might not be)

quartz salmon Oct 28, 2018, 1:07 AM

#

The first column is the same in ALL files

#

Thank the gods

patent glen Oct 28, 2018, 1:07 AM

#

it's still workable if it's not i think, but a little more complex

#

anyway, you can't select the index column with df[[...]] because it's always included

quartz salmon Oct 28, 2018, 1:08 AM

#

Okay so let me see if I'm understanding what you've written correctly

patent glen Oct 28, 2018, 1:08 AM

#

df[[...]] makes a new dataframe with a subset (and reordering) of the columns of the first dataframe

#

I think it's a fast operation because a dataframe is fundamentally a collection of column series, not rows

quartz salmon Oct 28, 2018, 1:09 AM

#

It's going to read all of the files in the working directory, and then go through them in descending order. For each file it opens, it will do a check on whether the named columns match in the order specified - if not, it will check for some second set of columns, and if it fails both of those it'll do some third thing (in this case, shoot out an error saying it's not one of the formats you specified)

#

Once that happens, I'll have a dataframe with all the data, though the columns won't be reordered properly yet, right?

patent glen Oct 28, 2018, 1:09 AM

#

the order of columns doesn't matter

#

you're defining the order you want them in, not the order the file has them in

quartz salmon Oct 28, 2018, 1:10 AM

#

Okay. That's awesome. It'll keep the row data from the columns properly and put them into the order I want?

patent glen Oct 28, 2018, 1:10 AM

#

(the check doesn't depend on the order at all, the df[[...]] defines the order of columns in the new dataframe)

#

yeah

quartz salmon Oct 28, 2018, 1:10 AM

#

Now, do I need to explicitly name every single column, or could I use this to select only the columns I actually need?

patent glen Oct 28, 2018, 1:11 AM

#

you can select only the ones you actually need

#

but

#

you have to rename the columns that mean the same things to be the same, for the concat to work

quartz salmon Oct 28, 2018, 1:12 AM

#

Okay - so should I run this to output two separate dataframes for the two different blocks? As it's written, I understand it to say that it'll only create one dataframe (example)

patent glen Oct 28, 2018, 1:12 AM

#

It wasn't clear to me if you needed two separate dataframes

quartz salmon Oct 28, 2018, 1:13 AM

#

I ultimately want one fully combined dataframe, where the columns that aren't listed in one another just get a NaN or 0 - that said, some of the names change and are actually the same thing

patent glen Oct 28, 2018, 1:13 AM

#

ok

#

for the names that change, you need to rename them using df['new name'] = df['old name']; del df['old name']

quartz salmon Oct 28, 2018, 1:13 AM

#

So if there was already a df['col1'], and I know that df['col2'] is the same and rename that one df['col1'], will it combine the two?

patent glen Oct 28, 2018, 1:14 AM

#

(df[old name].name = new name works, but doesn't do what you need for the concat to work - it sets the series internal idea of its name, whereas you need to set the name in the dataframe)

quartz salmon Oct 28, 2018, 1:14 AM

#

OK

patent glen Oct 28, 2018, 1:14 AM

#

...no

#

the renames are in the single file data frame

#

then, you'll have all of the columns named as col1 in those, and concat will pick them up and combine them

#

i.e. in the first file's dataframe you can rename col2 to col1, in the second you can rename col3 to col1, and in the third you can leave col1 alone, and it'll combine them all in concat to the big dataframe's col1

quartz salmon Oct 28, 2018, 1:15 AM

#

Okay, got it. So for the first code block you gave me (starting with example = [] ), does that create one mega-file with both blocks in it?

#

Okay, I see on that

patent glen Oct 28, 2018, 1:16 AM

#

well, it's creating the list that you still have to pass to pandas.concat to make the mega frame - i left that out because that part doesn't change

quartz salmon Oct 28, 2018, 1:16 AM

#

Again, apologies for being dense. I'm trying to understand what you're writing rather than just copy-pasting it and having no clue what's happening

#

Got it.

patent glen Oct 28, 2018, 1:16 AM

#

you can also do more fine grained comparisons than this, it'd help if i knew what your files actually looked like

#

like, if the main difference between the blocks is that the "foo" column is named "bar" in one of them

#

you can just do py if 'foo' not in df.columns: df['foo'] = df['bar']; del df['bar']

quartz salmon Oct 28, 2018, 1:17 AM

#

Should I just message you a head(1) from each block? The format is the same through both blocks, as I said

patent glen Oct 28, 2018, 1:18 AM

#

well does my last example get the point across?

#

the all() check was only really needed for if the format is a total black box

quartz salmon Oct 28, 2018, 1:18 AM

#

Yeah, that makes sense.

patent glen Oct 28, 2018, 1:19 AM

#

and if you don't need all the columns - if you do want to use all the columns and just have NAN if it's missing, then you can do the more targeted approach

quartz salmon Oct 28, 2018, 1:20 AM

#

Okay, let me try running some code first and get back with any problems. Thank you so, so much for the help you've given so far.

patent glen Oct 28, 2018, 1:20 AM

#

yeah no problem

quartz salmon Oct 28, 2018, 1:29 AM

#

lol fml this is the error I'm getting

#

File "<ipython-input-11-e65193353112>", line 15
raise Exception unknown csv format
^
SyntaxError: invalid syntax

patent glen Oct 28, 2018, 1:30 AM

#

it was pseudocode

#

i didn't feel like typing the parentheses and quotes, i figured you'd notice it

quartz salmon Oct 28, 2018, 1:30 AM

#

I tried that

#

I've never used raise before. Bad coder

patent glen Oct 28, 2018, 1:31 AM

#

though really

quartz salmon Oct 28, 2018, 1:31 AM

#

Sorry, let me work this more. I wont come back till I've gotten something more substantively unworkable.

patent glen Oct 28, 2018, 1:31 AM

#

you don't necessarily need that section if you're going for just targeted specific column renames instead of the blanket "check if all of one set columns are present" thing

quartz salmon Oct 28, 2018, 1:32 AM

#

Okay.

#

Oh wow, it actually ran. Here's what I wrote, and I don't know if it is correct or stupid:

for f in os.listdir('C:\\Users\\baxte\\Documents\\Datasets\\house-office-expenditures-with-readme\\Detail'):
    df = pandas.read_csv(f, index_col=[0], parse_dates=[0], engine='python')
    if all(x in df.columns for x in ['ColA', 'ColB', 'ColC']):
        df = df[[''ColA', 'ColB', 'ColC'']] 
    
    elif all(x in df.columns for x in ['ColD', 'ColE', 'ColF']):
        df = df[['ColD', 'ColE', 'ColF']]

    example.append(df)```

#

No errors, either

#

So now I've got a list of all the files, right?

#

Insofar as attempting to describe the dataframe was a no go. Told me it was a list. I assumed that example got the dataframe df by the append function.

#

Okay so I guess the question is really "should I concatenate now, or change the column names first?" - I'll try both, I suppose

patent glen Oct 28, 2018, 1:49 AM

#

you hae to change the column names to concatenate

#

it matches by column names, not position

quartz salmon Oct 28, 2018, 1:50 AM

#

Okay, yeah - the fact that it was still running the concat was worrying, haha

patent glen Oct 28, 2018, 1:50 AM

#

honestly

#

you could probably just use dataframe append instead

#

final_df = DataFrame()
for f...:
    df1 = read_csv...
    # rename columns if necessary
    final_df.append(df1)```

#

that way you can print each file you ingest for some idea of progress, instead of the big expensive operation being the concat at the end that you have no idea how far it is in it

quartz salmon Oct 28, 2018, 1:56 AM

#

Okay, not understanding what you're suggesting there. Will each separate file within the directory be added to df1 with renamed columns until it's finished the loop?

#

Sorry, the pseudocode is hurting my comprehension a bit.

cerulean widget Oct 28, 2018, 1:58 AM

#

Hi, I have a data frame with len(df) == 19 and two pd.Series both with len(series) == 10, I've tried a bunch of different combinations of concat append, join, but can't get it right. I want to add the two series to the right side of the dataframe from df.index[0:11] and fill the rest( [df.index[10:]) with np.nan. Is there any way I can tackle this problem?

patent glen Oct 28, 2018, 2:07 AM

#

@cerulean widget try just adding the series to the dataframe?

#

like, this works: ```py
df = pd.DataFrame({'a':range(19), 'b':range(30, 49)})
c = pd.Series(range(50, 60))
d = pd.Series(range(60, 70))
df['c'] = c
df['d'] = d

#

if they all already have non-trivial indexes you might have to do something else

#

@quartz salmon df1 is a temp dataframe, you'd be adding them to final_df

#

but otherwise, yeah

#

sorry ... wasn't meant as pseudocode, more just "same as what you have"

quartz salmon Oct 28, 2018, 2:13 AM

#

Okay, so I created final_df1 and final_df2 for the two separate blocks, and the code ran. Once I've done that, can I concatenate the two final_df's separately and then modify the column names as needed to ultimately concatenate them together?

patent glen Oct 28, 2018, 2:13 AM

#

the point is to append to a dataframe instead of a list, to let you skip the concat step, and do the work one piece at a time instead of the whole concat thing at once

#

er

#

i was figuring just a single final_df

#

that you append both types of file into, after renaming the columns as necessary

#

since the point is to avoid the single massive monolithic concat at the end, which you said was taking a long time with no idea of progress

quartz salmon Oct 28, 2018, 2:15 AM

#

Oh, okay - so I should have a section in the elif block that renames the columns as necessary?

cerulean widget Oct 28, 2018, 2:16 AM

#

thank you for the response. so to give you guys some context, these are the two Series I am trying to append:

#

min = pd.Series([train[j].min() for j in list(info.index[0:10])])
max = pd.Series([train[j].max() for j in list(info.index[0:10])])

patent glen Oct 28, 2018, 2:16 AM

#

ok i feel like you ignored the part where i said e.g. like, if the main difference between the blocks is that the "foo" column is named "bar" in one of them py if 'foo' not in df.columns: df['foo'] = df['bar']; del df['bar']

cerulean widget Oct 28, 2018, 2:16 AM

#

i tried @patent glen 's method and I'm getting all nan

patent glen Oct 28, 2018, 2:17 AM

#

@cerulean widget when you create the series, pass info.index[0:10] as the second argument so they all have the same index

quartz salmon Oct 28, 2018, 2:17 AM

#

e.g.
example = []
for f in os.listdir('C:\Users\baxte\Documents\Datasets\house-office-expenditures-with-readme\Detail'):
df = pandas.read_csv(f, index_col=[0], parse_dates=[0], engine='python')
if all(x in df.columns for x in ['ColA', 'ColB', 'ColC']):
df = df[[''ColA', 'ColB', 'ColC'']]

elif all(x in df.columns for x in ['ColD', 'ColE', 'ColF']):
    df = df[['ColD', 'ColE', 'ColF']]
    if 'ColA' not in df.columns:
            df['ColD'] = df['ColA'];
            del df['ColD']

example.append(df)

patent glen Oct 28, 2018, 2:17 AM

#

i.e. pd.Series([train[j].min() for j in list(info.index[0:10])], info.index[0:10])

#

@deft badge er my point was to have that if/rename thing instead of the if/elif/df[[...]] thing (OOPS WRONG PING SORRY)

#

not inside the elif

cerulean widget Oct 28, 2018, 2:18 AM

#

oh wow, nice...

#

thank you @patent glen

#

what does that second argument do?

patent glen Oct 28, 2018, 2:18 AM

#

it sets the index of the series

#

by default it's just 0, 1, 2, 3, etc

cerulean widget Oct 28, 2018, 2:20 AM

#

hm..interesting

#

Thank you very much for the help! i was trying to figure this out for the past hour and a half..

#

still new to programming

#

Thank you @patent glen

📎 df.png

#

📎 df_new.png

patent glen Oct 28, 2018, 2:26 AM

#

also i maybe wouldn't call your series objects 'min' and 'max'

#

could get confused with the min and max functions

#

maybe something like min_series

cerulean widget Oct 28, 2018, 2:26 AM

#

oh right..i'll definitely be mindful of that

quartz salmon Oct 28, 2018, 2:35 AM

#

Okay, so I'm trying to run this code now - does this look right, @patent glen?

#

for f in os.listdir('directoryaddress(it's real in the code)'):
    df = pandas.read_csv(f, index_col=[0], parse_dates=[0], engine='python')
    
    if 'COLA' not in df.columns:
        df['COLA'] = '0';
    
    if 'COLB' not in df.columns:
        df['COLB'] = '0';

    if 'COLC' not in df.columns:
        df['COLC'] = df['COLD'];
    
    
    final_df.append(df)```

#

The 0's are on purpose, I just want to throw in blank/junk data where it's not there. Is it possible to do just "" for actual, blank data where the column is missing?

#

Realize that I didn't make a dataframe at the top - just a list. it should be final_df = pandas.DataFrame()

#

Got this fun error when I tried to describe it:
ValueError: Cannot describe a DataFrame without columns

#

I'm guessing that's because this new dataframe I just made has literally no column names. I'll try adding the final columns and see if that breaks it?

patent glen Oct 28, 2018, 2:46 AM

#

@quartz salmon if you just leave the columns out, the append should fill them in with NaN

quartz salmon Oct 28, 2018, 2:47 AM

#

The code I pasted up there ran and didn't spit out any errors, which suggests that it filled in, but yeah - no column names, so no ability to manipulate the data

patent glen Oct 28, 2018, 2:47 AM

#

DataFrame() itself works fine, just don't try to describe() it

quartz salmon Oct 28, 2018, 2:48 AM

#

Yeah, I just need to have the column names so I can do stuff like run basic stats on the numeric values etc.

patent glen Oct 28, 2018, 2:48 AM

#

after you add the first file it will have columns

#

like, add them before that if you want, but i don't see why you'd need to

#

or are you saying it does that at the end? idk why, append docs say "Columns not in this frame are added as new columns."

quartz salmon Oct 28, 2018, 2:50 AM

#

Once it's run the whole codeblock that I pasted, I still have no column names - I was under the assumption that this code block would take the files and their data, modify the column names where needed, and append it all to a giant dataframe with the corrected column names and all data

#

Obviously I'm misreading what you wrote above, and again I apologize for being dense. It's been ~10 hours of me trying to make this work at this point

patent glen Oct 28, 2018, 2:51 AM

#

I thought it would too, don't know why it isn't working as expected

quartz salmon Oct 28, 2018, 2:51 AM

#

So yeah, I'll try adding the column names at the top before the code runs - I'll come back with the results on that

patent glen Oct 28, 2018, 2:52 AM

#

oh wait, append doesn't mutate final_df lol

quartz salmon Oct 28, 2018, 2:52 AM

#

Wait what, so it doesn't actually add any data to it?

patent glen Oct 28, 2018, 2:53 AM

#

i'm an idiot, sorry

quartz salmon Oct 28, 2018, 2:53 AM

#

I'm obviously confused. I've definitely used append before to add data haha

patent glen Oct 28, 2018, 2:55 AM

#

it returns a new object

#

which is what i was trying to avoid by not using concat

#

try final_df = final_df.append(df)

#

but you might be better off going back to concat - measure how much time it takes both ways

#

anyway i need to go, be back later

quartz salmon Oct 28, 2018, 2:57 AM

#

Thank you so much! I'll keep trying to get this to work.

#

That actually fixed it into a real dataframe, you solved part of my issues (a huge part)

#

Thank you so, so much @patent glen - I'll keep it up and come here with any questions. You're a lifesaver.

patent glen Oct 28, 2018, 3:23 AM

#

ok i'm back btw

quartz salmon Oct 28, 2018, 3:27 AM

#

Okay, and yeah - I think I was being stupid

#

        df['COLA'] = '0';```
This doesn't actually make COLA exist for the files in which it doesn't, right? My intent was to create COLA where it didn't, and fill it with 0s.

#

@patent glen in case you're off elsewhere

patent glen Oct 28, 2018, 3:30 AM

#

i think it does, but

#

you're filling it with a string 0, not number 0

#

if you don't bother with that it'll fill with NaN

quartz salmon Oct 28, 2018, 3:32 AM

#

Okay, so I don't even need to include that - it'll just fill NaN if I don't have an if statement for a column I know doesn't exist in some files

patent glen Oct 28, 2018, 3:32 AM

#

right

#

if you want another value you can do that though

#

df[name] = scalar creates a new series consisting of the value repeated over the dataframe's index

quartz salmon Oct 28, 2018, 3:33 AM

#

It's really not important one way or the other, so the less code I write the better

quartz salmon Oct 28, 2018, 3:55 AM

#

Okay, so yeah - I've got an empty dataframe according to print

#

print(final_df) returns

Empty DataFrame
Columns: []
Index: []

#

Fuuuuuudge

patent glen Oct 28, 2018, 4:00 AM

#

ok let me see your code?

#

did you still do final_df = final_df.append(df) ?

quartz salmon Oct 28, 2018, 4:01 AM

#

Yeah, I do. I'm trying a solution by adding the ultimate column names when calling final_df initially at the start of that code block. I'll give you an update in a second once its run

patent glen Oct 28, 2018, 4:02 AM

#

i don't think that's what it is

quartz salmon Oct 28, 2018, 4:02 AM

#

No, it isn't. Still empty, but it has some shiny new columns haha

#

And yes, I was still using the append.

patent glen Oct 28, 2018, 4:02 AM

#

maybe go back to building a list and using concat

#

right but including final_df = ...append, not just the append call

#

since the problem is append returns the new dataframe

quartz salmon Oct 28, 2018, 4:03 AM

#

... yeah, I didn't have that part

patent glen Oct 28, 2018, 4:04 AM

#

you had it at one point i think - maybe got lost in editing

quartz salmon Oct 28, 2018, 4:04 AM

#

Yeah, it did. Still a problem, though, even with that being the case

#

specifically when I try to do something like final_df(['COLA'].sum() it gives me this error:

TypeError: 'DataFrame' object is not callable

patent glen Oct 28, 2018, 4:07 AM

#

you've got a stray parenthesis

#

[and probably another one elsewhere]

#

final_df['COLA'].sum(), no final_df(

quartz salmon Oct 28, 2018, 4:09 AM

#

actually that seems to work

#

I can sum it now, whoops

quartz salmon Oct 28, 2018, 4:41 AM

#

So I now realize that my code is actually only capturing the last file. God help me

#

I think that's because I put the append function outside of the for loop, so the for loop replaced itself every time

torn sphinx Oct 28, 2018, 4:52 PM

#

whats the best module to use a mysql database>?

oblique horizon Oct 28, 2018, 5:27 PM

#

Hello

#

Anyone here successfully integrated mongodb with a django project

#

???

ionic pecan Oct 28, 2018, 10:35 PM

#

@torn sphinx depends on your requirements and your definition of „best“, iirc pymysql is thebest maintained one

#

@oblique horizon dont ask to ask

#

!t ask

delicate fieldBOT Oct 28, 2018, 10:35 PM

#

ask

Asking good questions will yield a much higher chance of a quick response:

• Don't ask to ask your question, just go ahead and tell us your problem.
• Try to solve the problem on your own first, we're not going to write code for you.
• Show us the code you've tried and any errors or unexpected results it's giving
• Keep your patience while we're helping you.

You can find a much more detailed explanation on our website.

oblique horizon Oct 28, 2018, 10:36 PM

#

I'm sorry man, but I've basically scraped google for anything and I'm at a loss as to what to ask

#

I guess does anyone know of an up-to-date or well-supported/documented library that integrates mongo into django

#

because the libraries I found are either deprecated or undocumented

short fjord Oct 29, 2018, 12:42 AM

#

whats a good place to start when learning about data bases with python?

gusty spindle Oct 29, 2018, 1:57 AM

#

a nice easy one is SQLite because it's in the standard library (import sqlite3). try to think of a scenario where a database could be used. perhaps a bank application: you've got to store information about people (name, age, address, etc), who each have an account, which stores their balance. maybe allow for people to take out loans, and store those too. you could simply make a text application that allows you to work with the database. @short fjord

short fjord Oct 29, 2018, 1:58 AM

#

Ok, cool thats actually what ive been working with

gusty spindle Oct 29, 2018, 1:58 AM

#

basically, i feel like a hands-on approach is important. banking is just an example, but if there's something in particular that you're interested in, go for that instead.

short fjord Oct 29, 2018, 1:58 AM

#

Banking sounds fun actually, ill give it a shot. thanks!

gusty spindle Oct 29, 2018, 1:59 AM

#

no problem!

short fjord Oct 29, 2018, 1:59 AM

#

Does sqlite have a documentation link

#

i couldnt find it

#

for some reason

gusty spindle Oct 29, 2018, 2:02 AM

#

python documentation for the sqlite3 module specifically: https://docs.python.org/3/library/sqlite3.html
in-depth documentation for the SQL queries you can execute with that module: https://www.sqlite.org/docs.html

short fjord Oct 29, 2018, 2:02 AM

#

perfect, thanks

#

$hello

#

!hello

#

!hello

#

oops wrong channel XD

#

meant to do that in my server

gusty spindle Oct 29, 2018, 2:06 AM

#

congratulations! :D

short fjord Oct 29, 2018, 2:07 AM

#

haha

wind pelican Oct 29, 2018, 2:12 AM

#

@short fjord one hint on banking, dont use floats. use fixed point numbers

#

dont want to lose 0.0001 of a cent

short fjord Oct 29, 2018, 2:15 AM

#

Ok, thanks for the tip

upper beacon Oct 29, 2018, 7:32 AM

#

Anyone here?

#

Im pretty new and am having an issue understanding pycharm. I have a venv and my script.py which writes to my database in my project folder. If I want to have the script run do i run it straight from just the project folder or do i have to go into the venv?

#

basically when would i go into the venv

upper beacon Oct 29, 2018, 8:06 AM

#

figured it out nevermind

boreal carbon Oct 29, 2018, 5:24 PM

#

cursor.execute("SELECT discord_id FROM discordusers WHERE discord_id=? GROUP BY discord_id", (ctx.message.author.id)) pyodbc.Error: ('HYC00', '[HYC00] [Microsoft][ODBC Microsoft Access Driver]Optional feature not implemented (106) (SQLBindParameter)')

#

im getting this error

#

but i have no idea what it means

ionic pecan Oct 29, 2018, 6:06 PM

#

sounds like some form of paywall based on the microsoft

#

wait no

#

you need a trailing comma

#

to make the parameter actually a tuple

#

cursor.execute("SELECT discord_id FROM discordusers WHERE discord_id=? GROUP BY discord_id", (ctx.message.author.id,))

cunning ginkgo Oct 29, 2018, 7:22 PM

#

Continuing what you were saying @ionic pecan

#

The thing is, I was looking for a db for my bot, and everyone suggest using postgres

#

I just want a database that's easy to use/understand, while still working on a larger scale

ionic pecan Oct 29, 2018, 7:23 PM

#

postgres is the correct choice then

#

once you get past the initial setup stuff it's very straightforward

#

and once you understood that once it sticks

cunning ginkgo Oct 29, 2018, 7:24 PM

#

That's the problem

#

Understanding it

#

A large part of it makes 0 sense to me

ionic pecan Oct 29, 2018, 7:26 PM

#

which resources are you using?

cunning ginkgo Oct 29, 2018, 7:26 PM

#

Wdym?

ionic pecan Oct 29, 2018, 7:28 PM

#

for learning postgres

cunning ginkgo Oct 29, 2018, 7:28 PM

#

The wiki and http://www.postgresqltutorial.com/

PostgreSQL Tutorial

admin

PostgreSQL Tutorial - Learn PostgreSQL from Scratch

Learn PostgreSQL quickly through a practical PostgreSQL tutorial designed for database administrators and application developers.

ionic pecan Oct 29, 2018, 7:31 PM

#

http://www.postgresqltutorial.com/postgresql-administration/ seems to be what you want

PostgreSQL Tutorial

admin

PostgreSQL Administration

PostgreSQL administration covers the most important activities for PostgreSQL database server including roles and databases management, backup and restore.

cunning ginkgo Oct 29, 2018, 7:33 PM

#

Think on, I have 0 experience outside of using an sqlite db a few times

#

This is gonna be fun..

#

I got this far, now nothing makes sense

📎 unknown.png

ionic pecan Oct 29, 2018, 7:51 PM

#

i wouldn't recommend using a gui for just figuring the base structure out

cunning ginkgo Oct 29, 2018, 8:03 PM

#

Then what would you suggest? I have 0 clue how to do anything with postgres

ionic pecan Oct 29, 2018, 8:16 PM

#

the CLI

cunning ginkgo Oct 29, 2018, 8:17 PM

#

Well, this is gonna be -very- fun

#

And it's just the beta build for my pc

#

I still gotta do it all again for my server

cunning ginkgo Oct 29, 2018, 8:38 PM

#

Is there not a way to view the columns, rows and tables using a gui? @ionic pecan

ionic pecan Oct 29, 2018, 8:38 PM

#

prett ysure there is

cunning ginkgo Oct 29, 2018, 8:39 PM

#

Welp, time to find that thing

ionic pecan Oct 29, 2018, 8:39 PM

#

why not just use the cli

#

\d+ table

cunning ginkgo Oct 29, 2018, 8:39 PM

#

📎 unknown.png

ionic pecan Oct 29, 2018, 8:39 PM

#

so you don't have the table

cunning ginkgo Oct 29, 2018, 8:40 PM

#

doubt this is how you make a new table

📎 unknown.png

#

but I tried

#

still gives the no relation

ionic pecan Oct 29, 2018, 8:40 PM

#

where did you pick that up?

cunning ginkgo Oct 29, 2018, 8:41 PM

#

absolutely no clue

#

📎 unknown.png

#

once again, no clue

ionic pecan Oct 29, 2018, 8:45 PM

#

you're missing semicolons

cunning ginkgo Oct 29, 2018, 8:45 PM

#

ughh

#

It's my js days all over again

#

At least I've got this far

📎 unknown.png

hoary forge Oct 29, 2018, 9:00 PM

#

wow 😄

cunning ginkgo Oct 29, 2018, 9:00 PM

#

yeah ikr

hoary forge Oct 29, 2018, 9:01 PM

#

i realy need to start learning more python 😄

cunning ginkgo Oct 29, 2018, 9:05 PM

#

yay

📎 unknown.png

#

:D

📎 unknown.png

hushed forum Oct 29, 2018, 9:27 PM

#

Can someone please tell me a quick solution to run a .sql file without installing apache and phpmyadmin etc

#

Is a connection absolutely necessary to run the file? Why can't we just execute the statements?

torn sphinx Oct 29, 2018, 10:23 PM

#

you can ?

#

create a database from the mysql REPL (using CREATE DATABASE your_db, and then you can use ```
mysql -u your_username -p your_db < your_script.sql

median snow Oct 30, 2018, 6:32 PM

#

Hi, is someone familiar with sqlite? If so can you help me understand why I get a syntax error while running this command?
c.execute("INSERT INTO "+table+"(date,price) VALUES(?,?) ON CONFLICT(date) DO UPDATE SET price=excluded.price",(date,price))
(both table, date and price are variable defined in the python code and date is a primary int key for the table)

patent glen Oct 30, 2018, 6:41 PM

#

does it give you any more detail on the syntax error

median snow Oct 30, 2018, 6:42 PM

#

sqlite3.OperationalError: near "ON": syntax error it's all it says

patent glen Oct 30, 2018, 6:43 PM

#

The phrase "ON CONFLICT" is also part of UPSERT, which is an extension to INSERT added in version 3.24.0 (2018-06-04).

#

i get the error too, i'm on version 3.22

#

what version of sqlite are you on

median snow Oct 30, 2018, 6:43 PM

#

I copied the command from the documentation on upsert and just modified what was needed for my case

#

ehm, how do I check? for sure 3.xx but not sure which one

patent glen Oct 30, 2018, 6:44 PM

#

sqlite3.sqlite_version

#

anyway your syntax looks fine to me, it's probably just not supported on your version

median snow Oct 30, 2018, 6:47 PM

#

is sqlite the best way to handle databases in python? I'm pretty new to them and that's the first that I found. I don't need to handle a very large database

#

I'm on version 3.21 of sqlite apparently, wouldn't come up earlier because I was using the version included in python and hadn't installed it separately

patent glen Oct 30, 2018, 6:54 PM

#

you can get the version in python with sqlite3.sqlite_version

#

that may or may not be the same as your standalone sqlite

#

anyway i wouldn't say there's a best way necessarily, sqlite has the advantage of not requiring a server setup

median snow Oct 30, 2018, 6:58 PM

#

yeah was asking just in case there was an option that was "mainstream" and largely preferred over the others, maybe will try to look for some other ways

teal rampart Oct 30, 2018, 8:34 PM

#

I have a python file that runs correctly. When I create a exe with pyinstaller and I try to run my program, it works only the first time (second times I get a database error). The code without compiling works fine
This is all the database loading: https://paste.pydis.com/ujiriwuxob.py
All the code: https://paste.pydis.com/bezufokawa.py

ionic pecan Oct 30, 2018, 8:39 PM

#

you need to show us the error you're getting

cerulean widget Oct 31, 2018, 2:54 AM

#

I have a DataFrame with column names range(0, 28). I have a variable column: column =[RANK, TEAM, GP, W, L, WIN_PERC, MIN, PTS, FGM, FGA, FG_PERC, THREE_PM, THREE_PA, THREE_P_PERC, FTM, FTA, FT_PERC, OREB, DREB, REB, AST, TOV, STL, BLK, BLKA, PF, PFD, PLUS_MINUS]. How can I replace the numbered column names with the values in the columns?

carmine heart Oct 31, 2018, 7:04 AM

#

Pandas dataframe?

df.columns = column

should do that.

torn sphinx Oct 31, 2018, 10:05 AM

#

i had a quick question, is MongoDB blocking?

merry garden Oct 31, 2018, 11:50 AM

#

Question regarding aiomysql

This is how I open a connection and a cursor:

async with self.pool.acquire() as conn:
    async with conn.cursor() as cur:
        await cur.execute(query, values)
        await conn.commit()

But, should I close the cursor or connection or both or none after using it? 🤔

await cur.close()
conn.close()

dull scarab Oct 31, 2018, 2:00 PM

#

Opening them as context managers should handle the teardown and closing automatically when you exist the scope of the context manager (the async with part)

torn sphinx Oct 31, 2018, 3:22 PM

#

if i want to download mysql, whichone do i download?

#

does community edition work or i need to pay for it

indigo mason Oct 31, 2018, 3:24 PM

#

@torn sphinx Don't

torn sphinx Oct 31, 2018, 3:24 PM

#

hi

indigo mason Oct 31, 2018, 3:24 PM

#

MySQL is the worst SQL

torn sphinx Oct 31, 2018, 3:24 PM

#

rly?

indigo mason Oct 31, 2018, 3:25 PM

#

Yeah

torn sphinx Oct 31, 2018, 3:25 PM

#

im trying to get this provably fair system thingy

#

and the guide wants me to use mySQL

#

📎 unknown.png

indigo mason Oct 31, 2018, 3:26 PM

#

And is the guide god of all SQL flavours

#

I suggest you use and learn PostgreSQL and if this is for your bot, use asyncpg as your library

torn sphinx Oct 31, 2018, 3:27 PM

#

o

dull scarab Oct 31, 2018, 3:30 PM

#

Saying Mysql is the worst sql doesn't really help anyone here

#

What are you using the database for?

indigo mason Oct 31, 2018, 3:30 PM

#

I understand a usecase

#

And

#

Unless there's no other option

#

MySQL isn't the way to go

torn sphinx Oct 31, 2018, 4:25 PM

#

is @indigo mason here

indigo mason Oct 31, 2018, 4:27 PM

#

Yes

torn sphinx Oct 31, 2018, 4:27 PM

#

hey is postgresql good

indigo mason Oct 31, 2018, 4:27 PM

#

There ya go

torn sphinx Oct 31, 2018, 4:27 PM

#

it is?

indigo mason Oct 31, 2018, 4:28 PM

#

Yes

#

It is

torn sphinx Oct 31, 2018, 4:29 PM

#

tyty

#

wait which one do i dwonload

#

📎 unknown.png

cunning ginkgo Oct 31, 2018, 4:52 PM

#

Well

#

which os do you use

novel vigil Oct 31, 2018, 5:32 PM

#

can someone help me understand what's wrong with how im inserting things into this db?

+-------+---------+------+-----+---------+----------------+
| Field | Type    | Null | Key | Default | Extra          |
+-------+---------+------+-----+---------+----------------+
| id    | int(11) | NO   | PRI | NULL    | auto_increment |
| title | text    | NO   |     | NULL    |                |
| body  | text    | NO   |     | NULL    |                |
+-------+---------+------+-----+---------+----------------+

and im inserting using
INSERT INTO post ("test post", "my first test post");
which gives a syntax error

carmine heart Oct 31, 2018, 6:44 PM

#

SQL? Where your values are now the column names should be, the values go after "VALUES". See this for instance: https://www.w3schools.com/sql/sql_insert.asp

#

INSERT INTO table_name (column1, column2, column3, ...)
VALUES (value1, value2, value3, ...);

novel vigil Oct 31, 2018, 8:37 PM

#

ooooh ok

#

im dumb, thank you

cunning ginkgo Oct 31, 2018, 8:46 PM

#

I'm using

        query = "SELECT warnings FROM warnings WHERE userid = $1;"
        row = await self.bot.db.fetchrow(query, ctx.author.id)

and it returns <Record warnings=0>, which is the value set. But how can I just get it to show 0?

ionic pecan Oct 31, 2018, 8:53 PM

#

row.warnings?

cunning ginkgo Oct 31, 2018, 8:55 PM

#

  File "C:\Users\Paws\AppData\Local\Programs\Python\Python36-32\lib\site-packages\discord\ext\commands\core.py", line 61, in wrapped
    ret = await coro(*args, **kwargs)
  File "C:\Users\Paws\Desktop\buibot\cogs\mod.py", line 56, in warns
    await ctx.send(f"You currently have **{row.warnings}** warnings.")
AttributeError: 'asyncpg.Record' object has no attribute 'warnings'

#

weird

ionic pecan Oct 31, 2018, 8:55 PM

#

row['warnings']

#

https://magicstack.github.io/asyncpg/current/api/index.html#record-objects

cunning ginkgo Oct 31, 2018, 8:56 PM

#

:o

#

That worked, thanks

turbid crypt Nov 1, 2018, 3:11 AM

#

Looking to chat with someone about imdbpy. In accessing a s3 database file. spicifically lines 54 - 105 of https://paste.pound-python.org/show/35r0Yg0KRK215bXMpHlK/ this is me just playing around learning.

turbid crypt Nov 1, 2018, 5:47 AM

#

nvm, i got some help. thank you anyway.

hallow drum Nov 4, 2018, 12:45 PM

#

does anyone game here

#

and knows python?

#

and is around 15 - 18

ionic pecan Nov 4, 2018, 1:36 PM

#

doesnt sound like something related to databases to me

#

also we don‘t exclude people by age here

#

!t ask

delicate fieldBOT Nov 4, 2018, 1:36 PM

#

ask

Asking good questions will yield a much higher chance of a quick response:

• Don't ask to ask your question, just go ahead and tell us your problem.
• Try to solve the problem on your own first, we're not going to write code for you.
• Show us the code you've tried and any errors or unexpected results it's giving
• Keep your patience while we're helping you.

You can find a much more detailed explanation on our website.

foggy shore Nov 5, 2018, 5:22 PM

#

Oh so if i ask a good question it will add a much higher chance to an iterable?

#

Cool x3

steel slate Nov 6, 2018, 6:45 PM

#

!rules

delicate fieldBOT Nov 6, 2018, 6:45 PM

#

Rules

The rules and guidelines that apply to this community can be found on our rules page. We expect all members of the community to have read and understood these.

ionic pecan Nov 6, 2018, 6:55 PM

#

bot stuff --> #bot-commands

tall kraken Nov 6, 2018, 8:22 PM

#

I have no clue where this question belongs but:
Heroku keeps resetting my database to the state it was uploaded in, help?

#

I have an sqlite database implemented through python for my discord bot

#

Heroku keeps restarting the database to how it was when it was uploaded (value wise)

#

Why, and how do I fix this?

dull scarab Nov 6, 2018, 8:27 PM

#

One of the reasons heroku is not suited for a discord bot. Heroku essentially clones the environment and destroy it at each request (or something in that manner). SQL files should be stored else where or use a heroku database. They have strict rules that you cannot expect files saved to not be frequently wiped
( Source: https://stackoverflow.com/a/13552761 & https://www.reddit.com/r/Heroku/comments/82c8sr/prevent_heroku_from_overwriting_sqlite_db_on/ )

tall kraken Nov 6, 2018, 8:29 PM

#

Oh

#

I guess I'm gonna host it somewhere else then

#

any suggestions

dull scarab Nov 6, 2018, 8:31 PM

#

I haven't used anything other than a raspberry pi, but digital ocean has been recommended

ionic pecan Nov 6, 2018, 8:31 PM

#

i use digitalocean, it's pretty good

tall kraken Nov 6, 2018, 8:32 PM

#

Can you give me an idea of the price range?

#

$5 per month 🤔

#

alright thanks 😄

cunning ginkgo Nov 6, 2018, 9:18 PM

#

I'm using postgresql and asyncpg, and can't work out how to get information from a row and put it into a message. I can get a single random row, but not the info from the columns.

ionic pecan Nov 6, 2018, 9:27 PM

#

you're gonna need to give us some more information

#

!t ask

delicate fieldBOT Nov 6, 2018, 9:27 PM

#

ask

Asking good questions will yield a much higher chance of a quick response:

• Don't ask to ask your question, just go ahead and tell us your problem.
• Try to solve the problem on your own first, we're not going to write code for you.
• Show us the code you've tried and any errors or unexpected results it's giving
• Keep your patience while we're helping you.

You can find a much more detailed explanation on our website.

cunning ginkgo Nov 6, 2018, 9:29 PM

#

            query = await db.execute("SELECT * FROM sketchdaily ORDER BY RANDOM() LIMIT 1;")
            row = await db.fetch(query)
            print('True, sending webhook message')
            webhook = DiscordWebhook(url=f'{config.webhookurl}', content=f"{row['idea']}")
            webhook.execute()

With the error

Traceback (most recent call last):
  File "index.py", line 33, in <module>
    loop.run_until_complete(run())
  File "C:\Users\Paws\AppData\Local\Programs\Python\Python37-32\lib\asyncio\base_events.py", line 573, in run_until_complete
    return future.result()
  File "index.py", line 25, in run
    webhook = DiscordWebhook(url=f'{config.webhookurl}', content=f"{row['idea']}")
TypeError: list indices must be integers or slices, not str

ionic pecan Nov 6, 2018, 9:30 PM

#

fetch probably returns a list of rows, not a single row

#

you're probably looking for something like fetchone

cunning ginkgo Nov 6, 2018, 9:31 PM

#

Traceback (most recent call last):
  File "index.py", line 33, in <module>
    loop.run_until_complete(run())
  File "C:\Users\Paws\AppData\Local\Programs\Python\Python37-32\lib\asyncio\base_events.py", line 573, in run_until_complete
    return future.result()
  File "index.py", line 23, in run
    row = await db.fetchone(query)
AttributeError: 'Pool' object has no attribute 'fetchone'

ionic pecan Nov 6, 2018, 10:10 PM

#

probably this then https://magicstack.github.io/asyncpg/current/api/index.html#asyncpg.cursor.Cursor.fetchrow

brave grotto Nov 7, 2018, 2:12 PM

#

Guys. I'm facing a bug that is beyond my comprehension/knowledge

#

I have a database

#

linked to my discord bot

#

I use a table as a cache for the bot

#

every 15min

#

I want to reset the tables to write in it again

#

Thus

#

I do this query

#

TRUNCATE TABLE cmc_coins;

#

then I write again in the table

#

but it looks like

#

there is some bug between the moment I truncate

#

and the moment I Insert Into

#

and the try except gives me nothing

#

moreover. The bot silently crash and disconnect. Event though it's still running in the IDE

#

some Side Infos : Postgres 9.6 DB hosted on Google Cloud. Python 3.7 using pipenv. Psycopg2 as lib for DB queries

#

Please help me 😦

pure scroll Nov 7, 2018, 2:19 PM

#

how do you use your db? through the ORM?

brave grotto Nov 7, 2018, 2:19 PM

#

ORM ?

pure scroll Nov 7, 2018, 2:19 PM

#

https://en.wikipedia.org/wiki/Object-relational_mapping

Object-relational mapping

Object-relational mapping (ORM, O/RM, and O/R mapping tool) in computer science is a programming technique for converting data between incompatible type systems using object-oriented programming languages. This creates, in effect, a "virtual object database" that can be used ...

#

e.g SQLAlchemy

#

how do you execute TRUNCATE and INSERT using the python, which library or whatsoever

brave grotto Nov 7, 2018, 2:21 PM

#

self.cursor.execute("TRUNCATE TABLE cmc_coins;")

#

📎 unknown.png

pure scroll Nov 7, 2018, 2:21 PM

#

ok

brave grotto Nov 7, 2018, 2:21 PM

#

and I defined the connection and cursor in the attributes of the class

pure scroll Nov 7, 2018, 2:22 PM

#

and what exactly your bug is?

brave grotto Nov 7, 2018, 2:22 PM

#

nothing happens

#

it's stuck somewhere

#

and I can't figure out

pure scroll Nov 7, 2018, 2:22 PM

#

TRUNCATE is not executed?

brave grotto Nov 7, 2018, 2:22 PM

#

what is going on

#

Truncate is executed

#

well I don't know

pure scroll Nov 7, 2018, 2:22 PM

#

and the table is wiped or not?

brave grotto Nov 7, 2018, 2:22 PM

#

I don't know... When I do a SELECT * FROM cmc_coins;

#

the query never ends

#

Here is the Truncating code part

📎 unknown.png

pure scroll Nov 7, 2018, 2:25 PM

#

but you could manually check the DB

brave grotto Nov 7, 2018, 2:25 PM

#

I want to DB to be wiped every 15min a few seconds before refilling them (thus the 895sec timer)

#

I have Datagrip open to check the DB manually

dull scarab Nov 7, 2018, 2:32 PM

#

Why use a database for that? Is the cache huge?

brave grotto Nov 7, 2018, 2:32 PM

#

1500+ rows

#

24 columns

dull scarab Nov 7, 2018, 2:32 PM

#

Why wipe it then?

brave grotto Nov 7, 2018, 2:33 PM

#

to renew it

#

if there is another way

#

teach me X)

dull scarab Nov 7, 2018, 2:33 PM

#

Cant you update the values?

brave grotto Nov 7, 2018, 2:33 PM

#

....

#

would you believe me

#

if I say

dull scarab Nov 7, 2018, 2:33 PM

#

I fail to see the need tp wrote 1500 rows every 15 min.

brave grotto Nov 7, 2018, 2:33 PM

#

that I didn't think of this ...

dull scarab Nov 7, 2018, 2:34 PM

#

To write *

brave grotto Nov 7, 2018, 2:34 PM

#

my bot needs it

#

my bot needs the fresh data every 15 min

dull scarab Nov 7, 2018, 2:34 PM

#

Im sure it might currently

pure scroll Nov 7, 2018, 2:34 PM

#

we don't you keep in memory

dull scarab Nov 7, 2018, 2:34 PM

#

Im just trying to understand what it's doing.

brave grotto Nov 7, 2018, 2:35 PM

#

Here is he kind of data i'm storing

📎 unknown.png

#

the api is rate limited

#

thus the need to cache

pure scroll Nov 7, 2018, 2:35 PM

#

and how do you use it then?

brave grotto Nov 7, 2018, 2:35 PM

#

I select things in there when I need them

pure scroll Nov 7, 2018, 2:36 PM

#

well I would keep that in memory

#

if it's 1500 rows only

#

but still your question is why nothing is happening

brave grotto Nov 7, 2018, 2:37 PM

#

yep

#

it's like a fucking magikarp

#

nothing happens

dull scarab Nov 7, 2018, 2:38 PM

#

None of the prints trigger?

brave grotto Nov 7, 2018, 2:38 PM

#

nope

#

none of the try execpt too

dull scarab Nov 7, 2018, 2:38 PM

#

How are you calling the method then

brave grotto Nov 7, 2018, 2:38 PM

#

📎 unknown.png

#

background task

dull scarab Nov 7, 2018, 2:39 PM

#

And are your other tasks running?

brave grotto Nov 7, 2018, 2:39 PM

#

yessir

#

there are 3 tasks

#

1 tasks to wipe 2 tables (the one we are talking about)

#

another task to fill the table cmc_coins

#

and the last task to fill the table cmc_global

#

the task to wipe runs every 895 seconds

dull scarab Nov 7, 2018, 2:41 PM

#

Is the bot responsive?

brave grotto Nov 7, 2018, 2:41 PM

#

no

#

when the bot 'bug'

dull scarab Nov 7, 2018, 2:41 PM

#

What does the other tasks look like

brave grotto Nov 7, 2018, 2:41 PM

#

it disconnect from discord

#

even though the bot still runs in the terminal

dull scarab Nov 7, 2018, 2:42 PM

#

Sounds to me that you're blocking the bot then.

brave grotto Nov 7, 2018, 2:42 PM

#

the other task are basically a big Insert Into

pure scroll Nov 7, 2018, 2:42 PM

#

sounds like you have a blocking issues there.

dull scarab Nov 7, 2018, 2:42 PM

#

Also are you using asyncpg

brave grotto Nov 7, 2018, 2:42 PM

#

no

#

psycopg2

#

and I do indeed havea blocking issue

#

the problem is that I can't find it precisely

dull scarab Nov 7, 2018, 2:43 PM

#

I'd try asyncpg then and see if that fixes some of them

brave grotto Nov 7, 2018, 2:43 PM

#

it works like psycopg2 ?

#

the methods are the same ?

dull scarab Nov 7, 2018, 2:43 PM

#

Pycopg2 is blocking though