#databases
1 messages · Page 100 of 1
you only give it a tuple if you're using execute many or bulk insertions
btw you might also get the issue with the WHERE message in ($1, $2, $3, $4) as iirc that method got depreciated so now it checks an Array rather than a listed set
yep try await conn.execute(query, *message_ids), for the first issue
what do you mean by that @brazen charm ?
the deprecated thing
Its something i remember when ive been building my asyncpg orm
i had an issue doing if x in {set of stuff here}
after some googling it got deprecated in postgre (apparently) to now be x in array
Ok so that is working now when i unpack for running query
should i worry about what you say CF8?
if it works it works ig
Yes it works 🙂
what version of postgre r u on?
oh, this isn't an array literal though so it's fine right? @brazen charm
oh yeah it might be tbh
i'd be very shocked if they deprecated IN (a, b)
PostgreSQL 12.3, on windows but on the VPS it is 10.12
yeah dw about it
ok thanks, and thanks for all help
i was thinking of inserting a array with a place holder 🙂
ah thats actually annoying. asyncpg won't translate a list or tuple to an array?
that would be a killer feature for a sql library tbh. auto generating all those damn placeholders
either generating an array literal or just a (a, b, ...) sequence
tbf ive actually made a converter for that
tho i use it for my orm
you mean something less ugly than query = query_template.format(', '.join(f'${m}' for m in range(len(message_ids))))?
does asyncpg or postgres handle like they call sql injection or we must still do?
yes that's the point of using the query placeholders
it prevents sql injection
well... helps prevent
unless your psycopg2
you can still cause it with bad database design
i just wish that asyncpg would spin off their binary postgres protocol parser
so someone can write a non-async version
i wonder if magicstack ever actually released that software product they were developing all this code to support
they were doing it for a end software?
maybe that was the uvloop event loop replacement
hmm idk
idk how they're even a company, i guess they do consulting
and just build these tools on the side
EdgeDB
that's the one
looks like it's out https://edgedb.com/
i mean tbf the stuff they've done has made async python considerably better
yeah they've made a huuge contribution
oh that looks fucking coool
im pretty grateful that their management supports FOSS software like this
its one thing coming from google and facebook, another coming from a small shop that has every incentive to keep this stuff private
but it's made magicstack an almost household name among python devs
here i am literally giving them free advertising
tbf i have no issue supporting companies like that
if we didnt have asyncpg or uvloop async would be no ware near as supported overall i think
precisely
idk if you've looked at that edge db
but damnnnnnnnn
its like a freaking OOP based SQL db
interesting
if i have 3 columns like
ID, date_a, date_b
how can i select all the rows containing only the values where datea and date_b both have values less than 2019-01-01 or have null values
WHERE
(date_a IS NULL OR date_a < '2019-01-01') AND
(date_a IS NULL OR date_b < '2019-01-01')
something like that?
how do i use an sqlite database on a vps
oh
is it possible to import an existing database to a vps, and if so how would i go about doing it
Seems like you do it the same way you would move any other file to the VPS. FTP?
I have an sqlite3 databse that isn't updating the values. I am making a currency system for a discord bot for anyone wondering
@torn sphinx did u do db.commit()?
@commands.command()
async def buy(self, ctx, *, item):
old_bal = self.bal._get_bal()
if item == "Burger":
new_bal = old_bal - 2
self.bal.update_value(column='bal', value=new_bal)
await ctx.send("$2 has been subtracted from your account.")
@commands.command()
async def work(self, ctx):
bal = self.bal._get_bal()
new_bal = bal + 50
self.bal.update_value(column="bal", value = new_bal)
await ctx.send("You gained $50!")
print(bal)
print(new_bal)
@commands.command()
async def bal(self, ctx):
bal = self.bal._get_bal()
await ctx.send(f"You have ${bal} in your account")
print(bal)
query = f"SELECT * FROM {TABLE} WHERE id = ? AND uid = ?"
self.cursor.execute(query, (self.guild.id, self.user))
info = self.cursor.fetchall()
if info:
self.bal = info[0][2]
return self.bal
else:
self._create_user()
self._get_bal()
def _create_user(self):
try:
query = f"""INSERT INTO {TABLE} VALUES (?, ?, ?)"""
self.cursor.execute(query, (self.guild.id, self.user, 0))
self.conn.commit()
except sqlite3.Error:
pass
def update_value(self, column, value):
query = f"UPDATE {TABLE} SET {column} = ? WHERE id = ?"
self.cursor.execute(query, (f"{value}", self.user))
self.conn.commit()
self._get_user_info()```
@torn sphinx did u do db.commit()?
@opaque pumice yes
i had the same problem yesterday but it somehow works today
def _create_table(self):
query = f"""CREATE TABLE IF NOT EXISTS {TABLE} (id BIGINT, uid INT, bal INT)"""
self.cursor.execute(query)
self.conn.commit()
It works on my other databases
like the other files I have
Yeah I can't seem to understand whats going on
more rather what the issue is
hi
does any one know how i can find the master user and password of a database in AWS?
you mean an RDS database?
yes they were
I think you can get the master username, but the master password is always hidden after creation
but i d like to know how to find them in case i forget
i found that i can change the master password
but i dont know about the user
Can someone tell me how to make a connection from MySql database to vscode
you want to connect a text editor to a database?
let's back up a step- what are you trying to do?
hey guys i am trying to migrate my db from sqlite3 to postgres, and am using docker pgloader, im using this command docker run --rm --name pgloader -v /home/ubuntu/Apex-Website/ecommerce/sqlite3.db:/etc/ dimitri/pgloader:latest pgloader ./etc/sqlite3.db postgresql:///newdb
and i get this error ```2020-07-24T01:49:10.028000Z LOG pgloader version "3.6.3~devel"
KABOOM!
FATAL error: Failed to connect to pgsql at :UNIX (port 5432): The value of CL-POSTGRES::USER is NIL, which is not of type STRING.
An unhandled error condition has been signalled:
Failed to connect to pgsql at :UNIX (port 5432): The value of CL-POSTGRES::USER is NIL, which is not of type STRING.
What I am doing here?
Failed to connect to pgsql at :UNIX (port 5432): The value of CL-POSTGRES::USER is NIL, which is not of type STRING.```
what to do
Can someone tell me how to make a connection from MySql database to vscode
@wooden wraith you can't connect mysql to vs code. You need to use something like mysql workbench to access the database
await db.execute("Update ProtectedTags set Uses = Uses + 1 where GuildID = ? and Tag = ?", (ctx.guild.id, tag))
await db.commit()```
two errors happen randomly sometimes:
sqlite3.DatabaseError: cannot start a transaction within a transaction
cannot commit - no transaction is active
weird because I have this exact code in my other cog only with the column name being different and i've never seen the error appear once there...
it still works sometimes so idk what's going on
<@&267628507062992896> He's going through all the channels. What is up tonight...
sigh
!tempban 726017978130497556 14d You've only joined here to spam your YouTube channel across all our channels. You should've read our rules when joining; we do not allow this here.
:incoming_envelope: :ok_hand: applied ban to @shell citrus until 2020-08-07 04:31 (13 days and 23 hours).
cleanup is always no fun
Once again, thanks for all you folks do.
await db.execute("Update ProtectedTags set Uses = Uses + 1 where GuildID = ? and Tag = ?", (ctx.guild.id, tag))
await db.commit()```
two errors happen randomly sometimes:
sqlite3.DatabaseError: cannot start a transaction within a transaction
cannot commit - no transaction is active
weird because I have this exact code in my other cog only with the column name being different and i've never seen the error appear once there...
hello
can someone help me setup postgres db on the host instance
and connect it with pgadmin4
does anyone know where i can find the gigaword database?
I've got this huge query
SELECT * FROM (
SELECT
emotes.emote as emote,
sum(emotes.count) as message_count,
sum(reactions.count) as reaction_count,
sum(emotes.count) + sum(reactions.count) as total_count
FROM emotes
INNER JOIN reactions
ON (emotes.emote = reactions.emote
AND emotes.period = reactions.period
AND emotes.guild_id = reactions.guild_id)
WHERE (emotes.guild_id=%s OR reactions.guild_id=%s)
UNION ALL
SELECT
emotes.emote as emote,
sum(emotes.count) as message_count,
0 as reaction_count,
sum(emotes.count) as total_count
FROM emotes
LEFT JOIN reactions
WHERE (reactions.emote IS NULL AND reactions.guild_id=%s)
UNION ALL
SELECT
reactions.emote as emote,
0 as message_count,
sum(reactions.count) as reaction_count,
sum(reactions.count) as total_count
FROM reactions
LEFT JOIN emotes
WHERE (emotes.emote IS NULL AND emotes.guild_id=%s))
GROUP BY emote
ORDER BY total_count DESC```
And I get this error `ERROR: syntax error at or near "WHERE" at character 994`
994 is on the line `WHERE (reactions.emote IS NULL AND reactions.guild_id=%s)`
What am I missing?
Tbh at this point I just think about creating another table and duplicating the data, but it will at least allow for easier querying
Ping me if you can help, I might forget I asked here
@torn sphinx you are using query parameters for this?
as in, you are not using % interpolation in python, right?
Yes, sure
ok good
i guess now it's a matter of just debugging
consider using sqlfiddle to test and debug
The first part before union worked separately btw
Nice tool, I'll try it a bit later
your 2nd and 3rd queries in the union
you forgot to use ON
you typically can't use JOIN without ON from what i remember
Ah, right, that may be the problem
you can write FROM reactions, emotes WHERE ... instead
but that's an inner join
left and right joins, i definitely recommend ON
Can't check right now, but I'll try when I get home, thanks
also using table aliases is typically recommended when possible
oh nvm you're using tbl.col
that's fine
yeah you just need ONs
i am using mysql
how can i connect python code from other computers to the database in localhost?
or should i make a database which is not stored in localhost
plz ping me when reply thx a lot
sqlite3 database not updating for some reason, no error given. ```py
@commands.command()
async def buy(self, ctx, *, item):
old_bal = self.bal._get_bal()
if item == "Burger":
new_bal = old_bal - 2
self.bal.update_value(column='bal', value=new_bal)
await ctx.send("$2 has been subtracted from your account.")
@commands.command()
async def work(self, ctx):
bal = self.bal._get_bal()
new_bal = bal + 50
self.bal.update_value(column="bal", value = new_bal)
await ctx.send("You gained $50!")
print(bal)
print(new_bal)
@commands.command()
async def bal(self, ctx):
bal = self.bal._get_bal()
await ctx.send(f"You have ${bal} in your account")
print(bal)
Database file:py
def _get_bal(self):
query = f"SELECT * FROM {TABLE} WHERE id = ? AND uid = ?"
self.cursor.execute(query, (self.guild.id, self.user))
info = self.cursor.fetchall()
if info:
self.bal = info[0][2]
return self.bal
else:
self._create_user()
self._get_bal()
def update_value(self, column, value):
query = f"UPDATE {TABLE} SET {column} = ? WHERE id = ?"
self.cursor.execute(query, (f"{value}", self.user))
self.conn.commit()
self._get_user_info()
self._get_bal()```
@torn sphinx i was going to say that maybe you didn't commit... but you did
if it helps, f"{value}" should just be value or str(value)
Everything that is needed is defined
yeah
I should do that
ohhhh
I think that actually might fix it
cause the number is an int, and that was a str
ah
well
so it's possible the error is happening but you're suppressing it somehow
oh
uh
don't re-use the cursor
re-use the connection
1 cursor per query
banish any thought of self.cursor
yes
replace*
at least for inserts/updates
yes
for select, conn.execute will return a cursor
well, conn.execute always returns a cursor
oh ok
but you only need to actually use that cursor for select queries
note that this is sqlite3 behavior and some other database libs require you to manually create a cursor
but even then, the same rule applies: 1 cursor per query
do not share cursors
alright
i am using mysql
how can i connect python code from other computers to the database in localhost?
@harsh pulsar it is returning a recursion error, meaning that the maximum recursion depth was reached
@torn sphinx I always us the ON clause in all my joins and the results ar different depending on the order of the joins. Here are some of my typical join clauses that are in some of my simpler queries. Not knowing your DB structure I can't make a solution from your data but you can get the idea from these I hope.
LEFT JOIN sheep_table AS sire_table ON sheep_table.sire_id = sire_table.sheep_id
LEFT JOIN sheep_table AS dam_table ON sheep_table.dam_id = dam_table.sheep_id
LEFT OUTER JOIN sheep_cluster_table ON sheep_table.sheep_id = sheep_cluster_table.sheep_id
LEFT JOIN cluster_table ON cluster_table.id_clusternameid = sheep_cluster_table.id_clusterid
I've got to it and this is my query now:
https://pastebin.com/ecc4bH4p
BUT for some reason it returns numbers that are way more that they actually are. What's even worse, it only gives me the inner joined part
Pastebin.com is the number one paste tool since 2002. Pastebin is a website where you can store text online for a set period of time.
Shoulda used pastebin
@torn sphinx There's missing ) somewhere in the SELECT sections. Not sure where as I don't quite see what you are trying here's a query of mine that has lots of SELECT clauses
Pastebin.com is the number one paste tool since 2002. Pastebin is a website where you can store text online for a set period of time.
@torn sphinx show your code
Like there's no way that kekw was used 300+ times for today
https://i.imgur.com/RLL7FfL.png
def update_value(self, column, value):
query = f"UPDATE {TABLE} SET {column} = ? WHERE id = ?"
self.conn.execute(query, (value, self.user))
self.conn.commit()
self._get_bal()
def _get_bal(self):
query = f"SELECT * FROM {TABLE} WHERE id = ? AND u_id = ?"
self.conn.execute(query, (self.guild.id, self.user))
info = self.cursor.fetchall()
if info:
self.bal = info[0][2]
return self.bal
else:
self._create_user()
self._get_bal()
shoot, wrong order
its flipped around
@torn sphinx get rid of self.cursor entirely
never write self.cursor ever
it is cursed
an angry snake will visit you at night and hiss at you until you stop it
cursor = self.conn.execute(query, (self.guild.id, self.user))
info = cursor.fetchall()
do this
await db.execute("Update Blacklist values set Channels = ? where GuildID = ?",
(json.dumps([]), ctx.guild.id))
await db.commit()```
discord.ext.commands.errors.CommandInvokeError: Command raised an exception: SystemError: <sqlite3.Connection object at 0x0000021599A88C60> returned NULL without setting an error
anyone know what that error means?
I want to convert a Date column to string inside a dataframe. the date column is dd-mm-yyyy. I want to convert it to ddmmyyy
maybe
is the date column it's own object
or is it a string
cause it its a string, you can just do string.replace('-', '')
is a datetime64 object
just get the str() value of it
that may work
im not very familiar with datetime64
and then use the str functions to modify it
just dir('')
to see everything you can do with it
i specify that the table is called Users, not users, in every line
where did i made a mistake?
what's the full stacktrace say?
just before you raise try doing this import traceback; traceback.print_exc()
to get more info
yeah
ok
you should see 2 stack traces separated by The above exception was the direct cause of the following exception:
do you see that?
All identifiers (including column names) that are not double-quoted are folded to lower case in PostgreSQL
try "Users" and not Users
ok
Hello, I store a datetime.datetime object in my database (as BLOB) but when I retrieve it, it becomes a str instead of a datetime.datetime object.
How to solve this?
alright so now it works, but other problem showed up
it doesn't save
rdbaker are you there?
where is the code of you saving it? @urban smelt
You should be using your cursor there, not your bot instance.
oh
@proven arrow do you know how to store a datetime object in a database?
Which database?
@lime echo self.client.pg_con.execute?
i...haven't defined it
@urban smelt normally, you should connect your database, did you do that?
yeah i did
await conn.executescript("""
CREATE TABLE IF NOT EXISTS guilds (
guild_id BIGINT NOT NULL UNIQUE,
auto_role BIGINT NULL,
mute_role BIGINT NULL,
logs JSON DEFAULT '{}'
);
CREATE TABLE IF NOT EXISTS giveaways (
description TEXT,
winners_count INT,
creator BIGINT,
message_id BIGINT,
channel_id BIGINT,
emoji TEXT,
ending_time TIMESTAMP
);
CREATE TABLE IF NOT EXISTS giveaways_ended (
description TEXT,
ending_time INT,
winners_count INT,
creator BIGINT,
message_id BIGINT,
channel_id BIGINT,
emoji TEXT,
winners BIGINT[] DEFAULT '{}'
);
""")``` For some reasons im getting `sqlite3.OperationalError: near ")": syntax error` i dont know if thats a problem with sqlite because syntax looks correct for me
@urban smelt send me the line of that code.
@copper echo ); should never be inside quotation.
Yeah postgreSQL that's the answer @urban smelt
so what should i do if i want to create multiple tables at once
@lime echo Yes you can store the datetime, be sure to pick the right data type
@urban smelt I don't know much about PostgreSQL, but normally, after you open your database, you need to create a cursor c=db.cursor() for instance, then use that cursor when you execute your database.
oh
in asyncpg you dont have to
@proven arrow I picked BOB but it stores it as str.
im using it
Oh well then you can still convert the string into datetime
@urban smeltyou were right here https://canary.discordapp.com/channels/267624335836053506/342318764227821568/736323794943213679
You probably want to use datetime.strptime as well to convert it
@lime echo so what should i do to create multiple tables with one script
its something like beta discord
@copper echo try ditching all the ;
show me error from terminal lol
i dont get one
@proven arrow that converts it from datetime to str. I want the opposite.
so dont load it
but i added
else:
raise error
What? no, its str to datetime. https://docs.python.org/3/library/datetime.html#strftime-strptime-behavior @lime echo
still disable it for a second
ok
You are using pools correct? @urban smelt
yes
Did you get a connection from pool first?
async with pool.acquire() as con:
await con.execute(...)
con = await pool.acquire()
try:
await con.execute(...)
finally:
await pool.release(con)
async with pool.acquire() as con:
async with con.transaction():
await con.execute(...)
From the docs you can use like this.
When I run this code, I don't get any errors for around 5-10 seconds until nothing sends and i get the following errors. I'm using mongoDB and all other commands owrk
@client.command()
async def points(ctx):
memberid = ctx.author.id
result = collection.find_one({"id":memberid})
await ctx.send(f'Points: {result["points"]}')
discord.ext.commands.errors.CommandInvokeError: Command raised an exception: ServerSelectionTimeoutError: 713373939472269322:27017: [Errno 11001] getaddrinfo failed
@tasks.loop(seconds=30)
async def check_reminders(self):
cursor = await db.execute("Select MemberID, Reminder from Reminders where Time <= ?", (time.time(),))
result = await cursor.fetchall()
for tup in result:
try:
member = self.bot.get_user(tup[0]) or await self.bot.fetch_user(tup[0])
await member.send(embed=to_embed(tup[1], info=True))
except: # Member deleted account, member doesn't share server with bot etc.
pass
await db.execute("Delete from Reminders where Time <= ?", (time.time(),))
await db.commit()```
sqlite3.OperationalError: cannot start a transaction within a transaction
last 2 lines, although i thought including the task was helpful as well
how to connect a localhost database from another computer?
i am using mysql
can someone help me with getting sqlite into my project
@whole mica another computer in the same network?
uhh no
im making a game
and it will be run in different computers
i want them to connect to the database and get data
or save
this is sort of a database thing as well as a discord.py but I think it is more a database thing I am using this code to store stuff in sqlite py @setup.command() async def autorole(ctx, arole:discord.Role): if ctx.message.author.guild_permissions.manage_messages: db = sqlite3.connect('main.sqlite') cursor = db.cursor() cursor.execute(f"SELECT autorole FROM main WHERE guild_id = {ctx.guild.id}") result = cursor.fetchone() if result is None: sql = ("INSERT INTO main(guild_id, autorole) VALUES(?,?)") val = (ctx.guild.id, arole) await ctx.send(f"Channel has been set to {arole.mention}") elif result is not None: sql = ("UPDATE main SET autorole = ? WHERE guild_id = ?") val = (arole, ctx.guild.id) await ctx.send(f"Welcome Channel has been updated to {arole.mention}") cursor.execute(sql, val) db.commit() cursor.close() db.close() all of the other stuff works right just not this one but I am getting this error ``` File "bot.py", line 359, in autorole
cursor.execute(sql, val)
sqlite3.InterfaceError: Error binding parameter 1 - probably unsupported type.
@whole mica generally SQL is never public facing. You want a server that talks to the SQL server for you
With public facing SQL any random person can get all the info in the database
ooh ic thx
so how can i make a server that talks to my sql server?
Well I assume you have the SQL bit down already from your current program
Here is an intro to networking in python
ooooh ok thx a lot
It is a bit of a big subject, so definitely don't be afraid to ask questions in #networks
kk thx
x_value_temperature = df_temperature.resample('M', on='DATE').mean()
after i did this
x_value_temperature becomes a weird dataframe
i cant get rid of the DATE column
temperature_2019 = pd.read_csv('C:/Users/dotha/PythonNotebook/File/temperature (2019) NYC.csv')
temperature_2020 = pd.read_csv('C:/Users/dotha/PythonNotebook/File/temperature (2020) NYC.csv')
list_of_temperature_2019 = temperature_2019['Temperature'].tolist()
list_of_temperature_2020 = temperature_2020['Temperature'].tolist()
temperature =[]
for i in range (len(list_of_temperature_2019)):
variable = ((list_of_temperature_2020[i]-list_of_temperature_2019[i])/list_of_temperature_2019[i])*100
temperature.append(variable)
temperature = pd.Series(temperature, name = "Average Temperature")
df_temperature = pd.concat([temperature_2019['DATE'],temperature], axis=1)
df_temperature['DATE'] = df_temperature['DATE'].astype('datetime64[ns]')
df_temperature = df_temperature[df_temperature['DATE'].dt.month != 6]
x_value_temperature = df_temperature.resample('M', on='DATE').mean()
#x_value_temperature = x_value_temperature.drop(columns="DATE")
x_value_temperature['Average Temperature']
You can vectorize your for loop. It's cleaner and much faster.
like this:
list_of_temperature_2019 = temperature_2019['Temperature'].values
list_of_temperature_2020 = temperature_2020['Temperature'].values
temperature = ((list_of_temperature_2020-list_of_temperature_2019)/list_of_temperature_2019)*100
.values basically converts it to Numpy array, which supports vectorized (element-wise) calculations like these.
yes, it will be a list. The result will be identical to your for loop. You should try it out, it's much more efficient than a for loop and you would want to learn vectorized operations like these to make your code run faster
Now back to your original question, what is the problem you're facing?
the 'DATE' column wont drop
I think if you just convert it to a list, it will go away. The date in the output is not a column, it's just the index for the series. The output is a series, not a dataframe
hmm if that is the case i can convert it to a df then
Do you want a dataframe?
because then all you have to do is write x_value_temperature[['Average Temperature']] instead of x_value_temperature['Average Temperature']
Double box brackets
to be honest, i am very lost right now
i am trying to follow
"The more 'scientific method' would be to do a regression where the y-variable is PM2.5 (and try the daily %diff of PM2.5 as well just to see what happens) and x-variable is something independent of covid but a good predictor of PM2.5 levels, like weather (temperature/rainfall). You may need to add a dummy variable (a binary variable) for weekend when less industrial activity means less pollution usually. You wouldn't need to go all the way back to 2010 for this but maybe 2018 would be good (try it out with the data you have so far first if you want). Note that when you do a regression you need to train the regression model with data before quarantine starts, then use that model to predict what things would have looked like had quarantine not happened against our actual observations (where quarantine did happen)."
btw do you know how to drop 59th element in the list_of_temperature_2020?
your way
Claim a help channel and ping me from there. This is giong quite off-topic for this room
Just copy paste your question there
lmao i didnt know there is help channel. how do i claim a channel hmm
Go to the help available sections, there will be two channels there. Take any one of them. Refer #❓|how-to-get-help
Did you get a connection from pool first?
@proven arrow I used this to connect to it
Should i learn sql or nosql
You'll eventually have to learn both, most likely. But I'd say start with SQL.
Mostly because there's no single nosql language. Each nosql db has its own query language. Whereas most parts of SQL are fairly standardised for the commonly used databases like MySQL, PostgreSQL, MS SQL, SQLite, etc. Only when you reach intermediate to advanced stuff, will these start to vary somewhat. Basically when you're dealing with Db specific implementation of a feature.
SQL is useful and easy to learn.
So i am good to go with mysql as a beginner?
yeah, MySQL is good for starting out with. You can install the database on your local machine and use mysql workbench or php mysql or something like that
Linux or Windows?
Windows
You can install MariaDB (an open source implementation of MySQL) and Dbeaver community edition. It's a bit of an advanced Db management tool, but you should be able to use it
can someone help me possibly in dms about how to convert my discord level system from saving stuff in json to sqlite, or should I just ask here?
@tepid cradle can u recommend mysql tutorials
Let me look up
Give this a try https://www.tutorialrepublic.com/sql-tutorial/
You can also ask for help here if you are stuck with anything specific
can someone help me possibly in dms about how to convert my discord level system from saving stuff in json to sqlite, or should I just ask here?
@sinful condor Is your JSON nested?
nested?
@tepid cradle
heres what my json looks like {"449756487510654988/721149838573699154": {"xp": 112, "level": 4, "warnings": 0}, "469228384321536011/732567342525186111": {"xp": 30, "level": 3, "warnings": 1}, "603611557162647552/721149838573699154": {"xp": 5, "level": 1, "warnings": 0}, "663117661139632148/721149838573699154": {"xp": 61, "level": 3, "warnings": 0}, "449756487510654988/732567342525186111": {"xp": 53, "level": 3, "warnings": 0}, "530818947227058186/732567342525186111": {"xp": 0, "level": 1, "warnings": 0}, "599632471038099496/732567342525186111": {"xp": 1, "level": 1, "warnings": 0}, "482154322084233217/732567342525186111": {"xp": 0, "level": 1, "warnings": 0}}
does that help
Yeah, this is nested. As in, the values of the top level keys are objects, not just strings or ints
I asked because if it was not nested, it's very easy to convert it to tables. I don't really know how to convert this. Or at least, not well enough to explain it to someone else
so do i basically have to rewrite all of the code to get it into sqlite?
No, only the parts which interact with your JSON file
which in this is basically everything
I se maybe 10 lines out of 100 that dont interact with a json file
What does your program do?
its a discord bot
its a level system for a discord bot so everything needs to be stored
every message it gets activated
and this is my code ```@client.event
async def on_message(message):
db = sqlite3.connect('main.sqlite')
cursor = db.cursor()
cursor.execute(f"SELECT prefix FROM main WHERE guild_id = {ctx.guild.id}")
result = cursor.fetchone()
await update_data(users, message.author)
await add_xp(users, message.author, 1)
await level_up(users, message.author, message.channel)
cursor.execute(sql, val)
db.commit()
cursor.close()
db.close()
async def update_data(users, user):
userguild = f'{user.id}/{user.guild.id}'
if user == client.user or str(userguild) in users:
return
else:
users[str(userguild)] = {}
users[str(userguild)]["xp"] = 0
users[str(userguild)]["level"] = 1
users[str(userguild)]["warnings"] = 0
async def add_xp(users, user, exp):
if user == client.user:
return
else:
userguild = f'{user.id}/{user.guild.id}'
users[str(userguild)]["xp"] += exp
async def level_up(users, user, channel):
if user == client.user:
return
else:
userguild = f'{user.id}/{user.guild.id}'
xp = users[str(userguild)]["xp"]
lvl_start = users[str(userguild)]["level"]
lvl_end = int(xp ** (1/3))
if lvl_start < lvl_end:
userguild = f'{user.id}/{user.guild.id}'
await channel.send(f"{user.mention} has leveled up to level {lvl_end}")
users[str(userguild)]["level"] = lvl_end
``` basically everything interacts with a json file in some way or another
the top section is the one I am working on
OK. If you're not familiar with SQL, you can explore other options. You have nosql databases like MongoDb where you can directly store the JSON. Or you can go for SQLAlchemy ORM where you can build models and write SQLAlchemy commands instead of writing SQL queries directly. But all of them will require you to change pretty much all the lines
If you already know SQL, then IMO writing plain old SQL queries is the easiest
ok
Because of its age and maturity, there's also plethora of help content available for SQL, which doesn't seem to be the case for MongoDb.
How would I make this a parametrized query?
curs.execute(f"SELECT value FROM prices WHERE name = "{asset}" AND stamp >= {fromstamp} AND stamp <= {tostamp}'")
which database? The syntax varies slightly depending on that
curs.execute("SELECT value FROM prices WHERE name = %s AND stamp >= %s AND stamp <= %s", (asset, fromstamp, tostamp))
This would be the general syntax
you also open a ' but then dont close it
👍
In general, don't use quotes inside your query, parameterization takes care of it wherever needed
yes, i just read this before but it didnt really make sense
now it does
I'm experiencing a problem inserting values into a SQLite database. The data I download from the Norwegian Parliament site data.stortinget.no. The error I get is: sqlite3.OperationalError: unrecogn...
hey @tepid cradle I am almost done at least trying to switch it over to sqlite here is one section but I have a question ```py
async def add_xp(users, user, exp):
db = sqlite3.connect('main.sqlite')
cursor = db.cursor()
cursor.execute(f"SELECT xp FROM users WHERE userguild = {userguild}")
result = cursor.fetchone()
expe = result += 1
if user == client.user:
return
else:
userguild = f'{user.id}/{user.guild.id}'
sql = ("UPDATE users SET xp = ? WHERE userguild = ?")
val = (expe, userguild)
yes. That's not the cause of the error though. When you do cursor.fetchone(), it returns a tuple, no matter what your query. So you might need to do result[0] += 1 or even result[0][0] += 1
Just print out result and check what it is, I'm not quite sure if it's a tuple or a list of tuples.
I cant print result because I cant get the bot to load because of the errors
Also, don't use f-strings for SQL queries, it is bad practice as it leaves your code vulnerable to SQL injection attacks.
I switched it to result[0] += 1
Then run the query manually on the terminal or a separate file and see the response
It returns a tuple, result[0] should be fine
fetchall returns a list of tuples, fetchone returns a tuple
yeah it still has the error though
Are there any other errors? I can see one in the last block
add something after return
right now it only shows this one but I know there are alot more because when I fix one like 5 more pop up
heres all of the code in a pastebin https://pastebin.com/raw/CYqkr98A
what should I add after return
You should add something after return. It's not an error, but right now it's not returning anything, so you won't know what happened if the function goes into that block
Whatever you suppose should happen in the else condition
If nothing should happen, then there's no need of that if condition at all
yes but I dont want the bot to be able to level up and get xp and stuff
Then return None or False or something
ok
Then this whole block is problematic:
else:
userguild = f'{user.id}/{user.guild.id}'
sql = ("UPDATE users SET xp = ? WHERE userguild = ?")
val = (expe, userguild)
you create the variable sql (which you should ideally name query), but don't use it.
Nor do you use the variable val anywhere. The function just ends after that line, effectively rendering the else block meaningless
I am meant to put cursor.execute(sql, val) below it I think
OK
Then it would make sense. Then don't put brackets around the query. Again, not an error, but unnecessary
where?
expe = result[0] += 1
show your updated code
And the error message
its just invalid syntax invalid syntax (<unknown>, line 944)pylint(syntax-error)
its not letting me start the bot because of it
The code itself looks fine
Can you try running that function separately with some dummy values
I dont get the error when I put it in a dummy thing
Then it's just the linter showing the error
Oh, sorry
It is incorrect syntax
remove the second =
so expe = result[0] + 1
expe = result[0] + 1
ok
+= is used when you want to increment the value of that variable itself, not when you are assigning it to a new variable
That's because there are no values in the Db. If you fetch from a blank table, you'll get this error
why didnt the updata_data not kick in to make a section for them
You need to commit after insert or update queries. Just add cursor.commit() to commit
would it be cursor.commit or db.commit
cursor
cursor got error but db worked I have been using db for every commit in my bot. but I have a problem
its making 2 different things
and it is happening whenever I put a message
how do I put it in one
you're inserting in the same table, why do you want two queries?
no I want to make it one but I dont know how
you're already inserting values in 2 columns, just add a third as well
what do you want to do?
INSERT INTO inserts a new row
use UPDATE if you want to update existing row
that first one is only supposed to happen if there is nothing in there with that userguild(userid/guildid)
But then you'll have to check if it already exists, otherwise update will throw an error
that is what the if result is not None or user == client.user: return is supposed to do
First correct this part
else:
if result is None:
sql = ("INSERT INTO users(userguild, xp) VALUES(?,?)")
val = (userguild, startxp)
sql1 = ("INSERT INTO users(userguild, level) VALUES(?,?)")
val1 = (userguild, startlvl)
cursor.execute(sql, val)
cursor.execute(sql1, val1)
First, use elif for second condition, not another if inside your else>
Second, if you've already checked that it is not none, and that was not satisfied, then it's obviously none. So you don't need to check again at all
ok
Here is the code async def update_data(users, user): userguild = f'{user.id}/{user.guild.id}' startxp = '0' startlvl = '1' db = sqlite3.connect('main.sqlite') cursor = db.cursor() cursor.execute(f"SELECT userguild FROM users WHERE userguild = {userguild}") result = cursor.fetchone() if result is not None or user == client.user: return elif result is None: sql = ("INSERT INTO users(userguild, xp, level) VALUES(?,?,?)") val = (userguild, startxp, startlvl) cursor.execute(sql, val) db.commit() it still does the seperate thing
or like this if result is not None or user == client.user: return else: sql = ("INSERT INTO users(userguild, xp, level) VALUES(?,?,?)") val = (userguild, startxp, startlvl) cursor.execute(sql, val) db.commit()
I am also getting this error ``` File "bot.py", line 941, in add_xp
expe = result[0] + 1
TypeError: 'NoneType' object is not subscriptable
@tepid cradle
cursor
@tepid cradle
where
So, a question about the asyncpg library to access a postgresql database. I need to dynamically select the table name, but that librayr doesn't provide a SQL object like pyscopg2 does. What should I do?
I can reword the question if it's unclear
Yeah but what about safe parsing?
They say to NEVER EVER EVER EVER concatenate or .format() SQL code
I wish there was just a safe_format() function or something
def listPriceHistory(asset, fromstamp, tostamp):
conn = sqlite3.connect('testing.db')
curs = conn.cursor()
try:
curs.execute('SELECT value, staml WHERE name = ? AND stamp >= ? AND stamp <= ?', (asset, fromstamp, tostamp))
except sqlite3.Error as e:
print("An error occurred:", e.args[0])
return curs.fetchall()```
you need to specify which table you're selecting from
SELECT value, staml FROM [db name] WHERE ...
oof
yeah fixed
thanks
and i also somehow managed to misspell stamp
Hi, if there's anyone here who knows about MongoDB I have a collection that stores points for each user in a discord server. I want to create a leaderboard function and find the top 10 users with the most points (which is the value of each post) but not sure how i can index each line in the DB. I could probably figure out the python part of it but for now i'm not sure how to grab the highest 10 {"points"} values from the collection
Hey guys, I think this is the right place to ask this, if not, please point me in the right direction.
Hey so, I'm trying to get some data from a redis website. (It has a .php ending) and then cache that data and then put it into either google sheets or some kind of analysis tool. I'm really new to this so I'm flying blind.
Which fit this situation? I'm guessing there's something out there that will already listen to a redis site and then cache your data but I'm not sure what that would be called. There's so many options and I'm pretty sure that there's probably a good way to do this so if you guys have any suggestions I would really appreciate it!
I need to revise my DB in python, what would be a good starting point?
where should i start learning / resources available
i have a really dumb question. In mongodb, what data type should my dict be stored as?
right now ihave a dict inside an array, and it defaults it to string, is that normal?
useritems[ #array
0: "{ some info}" #string
1: "{more info}" #string
]```
how do i connect to a rds with mysql workbench cuz i cant seem to make a successful connection
it's been a long time since I used AWS, but...have you added inbound connection rules that would allow your computer to connect?
if I recall correctly, AWS default security group settings don't have any inbound rules set, and so ignore incoming connections from outside by default
sqlite3's cursor closes automatically after finishing running the file right?
I'm not sure on whether it's necessary, but it should be closed when it's garbage collected (not guaranteed to happen); if it doesn't support a context manager directly you could use closing from contextlib to make sure it's closed
k thanks
i just realized i made my thing with public accessiblity as no
now i gotta figure out how to change it to yes
you'll need to add an inbound rule that lets you connect. you could add 0.0.0.0/0, but ideally you don't do that
maybe best add just your IP address for now
in future you could proxy through a bastion. workbench should support connecting with an SSH port forward
Yo yo yo I need help with adding a dB to my Discord bot - I already a dB for my prefixes but I need one for blacklisting members so banned members can’t use the bot
What I have so far:
@commands.Cog.listener()
async def on_message(self, message: discord.Message):
embed = discord.Embed(
colour=discord.Colour.red()
)
embed.add_field(name=“U were banned”, value=“reason”
if not message.author.id in Blacklist:
await command.process(message)
else:
message.send(embed=embed)
Don’t mind the indents I’m on mobile typing this
Preferably sqlite3
i'm not sure if this is the right place to ask this
but what's the best way to do feature selection
measuring how close a feature is to a specific feature, like hospital death
with connection:
cursor.execute("""
SELECT coins FROM users WHERE ? = userid
""")
if cursor.fetchone() # <--- Here ```
Trying to make this check if it can't find coins e.g. meaning not in the database
cursor's remember what you did with them
so, that .fetchone() will grab the first result, and then the cursor will forget what it just did
calling .fetchone() again will return an error
you should store the result of .fetchone() to do stuff with it
So store it as a variable?
Then do something like
if x:
#add the coins
else:
#inset a value into the database?
X being the cursor.fetchone btw
have you committed the changes? Usually, if there is no syntax error, then forgetting to commit is the reason for SQL statements not working
async def register(self, ctx):
db = sqlite3.connect('economy.sqlite')
cursor = db.cursor()
cursor.execute(f"INSERT INTO economy (user_id) \ VALUES ({ctx.author.id})")
result = cursor.fetchone()
if result is None:
await ctx.send("k")
db.commit()
cursor.close()
db.close()```
This code shows this error when i run it:
Command raised an exception: OperationalError: unrecognized token: ""
Ping me Thanks
Okay so back when I was very new to discord.py I had a broken json prefix thing and now I have 45 guild IDs and prefixes and I am wondering if it’s possible to remove all of them and when a person runs a command and the guild ID isn’t in the SQLite3 DB it would add it.
TL;DR
If remove all guild_ID and prefixes is it possible when a user runs a command it adds them back? So I am not seeing guild IDs and prefixes that don’t use my bot anymore?
i want to replace all of the values if "guild" is the same instead of adding a new row with the same guild value. how can i do that?
Just check if the guild already exists in the database. If yes, then run an update query, if no, then run an insert query
Use a select query to count the number of existing rows with that particular guild, then check if it's zero, which means that it doesn't exist
i am working on a bot where admins can enter few questions to the user , and stores it in db , but the problem is i want the maximum number of questions to be 15 and the min can be any if the user wants 5 he can Type done when he completes 5 questions and only 5 must be saved in db py @commands.command() @commands.guild_only() @commands.has_permissions(administrator = True) async def applyquestions(self,ctx): conn = sqlite3.connect('infobot.sqlite') c = conn.cursor() c.execute('''CREATE TABLE IF NOT EXISTS application_qustions (guildid TEXT , application_id TEXT , qus1 TEXT ,qus2 TEXT ,qus3 TEXT ,qus4 TEXT ,qus5 TEXT,qus6 TEXT,qus7 TEXT,qus8 TEXT,qus9 TEXT , qus10 TEXT) ''') question = [] for i in range(1,11): a = await ctx.send(f"enter your question ") q = await self.bot.wait_for("message" , check = lambda message : message.channel == a.channel and message.author == ctx.author) qustionf = q.content if (qustionf in ["done" , "Done"]): break else : question.append(qustionf) await ctx.send(question) conn = sqlite3.connect('infobot.sqlite') c = conn.cursor() c.execute('''CREATE TABLE IF NOT EXISTS application_qustions (guildid TEXT , qus1 TEXT ,qus2 TEXT ,qus3 TEXT ,qus4 TEXT ,qus5 TEXT,qus6 TEXT,qus7 TEXT,qus8 TEXT,qus9 TEXT , qus10 TEXT) ''') c.execute(f"INSERT INTO application_qustions(guildid , qus1, qus2, qus3, qus4, qus5, qus6, qus7, qus8, qus9, qus10 ) VALUES('{ctx.guild.id}','{question[0]}','{question[1]}','{question[2]}','{question[3]}','{question[4]}','{question[5]}','{question[6]}','{question[7]}','{question[8]}','{question[9]}')") conn.commit()
Problem: when user stops at 5 questions it wont be saved in db , and i get a error ``discord.ext.commands.errors.CommandInvokeError: Command raised an exception: IndexError: list index out of range`
Lol.. @wary abyss , you posted this question earlier, and I worked out an answer, but when I came back to post it, the question had disappeared 😆
@tepid cradle sooo sorry , i thought no one would answer and deleted it sorry
Here you go
base_query = "insert into table_name(col1, col2, col3) values {}"
questions = ['q1', 'q2', 'q3']
query = base_query.format("(%s, %s, %s), "*len(questions))[:-2]
print(query)
cur.execute(query, questions)
You are not allowed to use that command here. Please use the #bot-commands channel instead.
OK Mr. bot 😛
sure, that's why I wanted to do it with eval, would have been easier to explain. But anyway.
So I'm creating the query without placeholders, with just {}.
Then I'm getting the questions in a list.
Then I take a list of placeholders, i.e. (%s, %s, %s), and multiply it with the length of the above list. This gives me a list of list of placeholders just as long as the messages list requires. Then I add it to the query.
In the end, your query will look like this
This has nothing to do with Sqlite though, it's all just python
the [:-2] only removes the last two characters, i.e., , which would otherwise cause an SQL syntax error
ooh! okayyy
but where do i insert it? , wait let me edit the code and send it
aaaaa
This is your query. You have to replace your query with this. Since you don't know the number of inputs, you can't use a static query. You have to prepare your query this way
Let me know if this works for you
yeah sure i am trying it out
There's also cursor.mogrify(), which I don't fully understand. You can check out this answer for that method, if you want:
https://stackoverflow.com/questions/8134602/psycopg2-insert-multiple-rows-with-one-query
okayy i will tysm bro
@tepid cradle py @commands.command() @commands.guild_only() @commands.has_permissions(administrator = True) async def applyquestions(self,ctx): question = [] for i in range(1,11): a = await ctx.send(f"enter your question ") q = await self.bot.wait_for("message" , check = lambda message : message.channel == a.channel and message.author == ctx.author) qustionf = q.content if (qustionf in ["done" , "Done"]): break else : question.append(qustionf) await ctx.send(question) conn = sqlite3.connect('infobot.sqlite') c = conn.cursor() c.execute('''CREATE TABLE IF NOT EXISTS application_qustions (guildid TEXT , qus1 TEXT ,qus2 TEXT ,qus3 TEXT ,qus4 TEXT ,qus5 TEXT,qus6 TEXT,qus7 TEXT,qus8 TEXT,qus9 TEXT , qus10 TEXT) ''') base_query = "insert into application_qustions(col1, col2, col3) values {}" questions = ['q1', 'q2', 'q3'] query = base_query.format("(%s, %s, %s), "*len(questions))[:-2] print(query) c.execute(query, questions)
is this correct? i have a error
wait, why are you redefining the questions list. In my example it was obviously just an example. You already have that list above. You'll have to adapt my code to suit the parameters that you have. Like, you have some 10-11 parameters, I had used only 3 to show how it would work.
you can't just copy paste my code, work on it to suit your application.
okayyy lol , got it
I'll explain it further if it's not clear what my code is doing, you can ask
If you have questions about any specific part of my code
okayyy , ig i have a idea how its working , i will try according to it if its not working i will ask
I am quite new to sql, I am using postgresql if a have a table my_table
id | name
----+----------
How could I get a random row that contains a certain text.
I have this so far, to choose a random row:
select * from my_table offset random() * (select count(*) from my_table) limit 1;
How would I modify this to only choose from rows with name like whatever?
using mongodb (pymongo), can i update a document to have the values of a dictionary?
like if i have a dict a = {'key':'value', 'key':'value'} , how can i update a field to this dictionary?
@bot.command()
async def warn(ctx, member: discord.Member, *, reason=None):
await ctx.send(f"{member.mention} has been warned for {reason}.")
await member.send(f"You've been warned in Rocket Legion for the following reason: {reason}")
with open('warns.json', 'w') as f:
data = json.load(f)
if not member.id in data:
warns[member.id] = 1
else:
warns[member.id] += 1
with open('warns.json', 'w'):
json.dump(data, f, indent=4)
``` `UnsupportedOperation: not readable` idk what that error means (ik this is dpy but i was told to ask here since its a python issue not a dpy issue)
When joining tables A and B on indexed columns of B, a full scan in A will be needed?
@torn sphinx hey so i got everything set up but i may need your help actually translating if you're down. i spent about 2-3 hours last night trying to understand the basics of SQLite and i do now, but majority of those 2-3 hours was me having issues with "translating" the JSON command into an SQLite command
@pliant spoke you meed to read it first them write on it
@tepid crow k thx. now im getting this ```py
@bot.command()
async def warn(ctx, member: discord.Member, *, reason=None):
await ctx.send(f"{member.mention} has been warned for {reason}.")
await member.send(f"You've been warned in Rocket Legion for the following reason: {reason}")
with open('warns.json', 'r') as f:
data = json.load(f)
if not member.id in data:
data[member.id] = 1
else:
data[member.id] += 1
with open('warns.json', 'w'):
json.dump(data, f, indent=4)``` `ValueError: I/O operation on closed file.` in the json.dump line
@hazy mango do I use asyncpg?
Yea
That's the module you use inside Python
Then you use PgAdmin4 on your PC to view the database contents
@orchid yacht
Alrighty
try closing the file (im not sure if thats gonna help) @pliant spoke
Hey @hazy mango, so I am currently in PgAdmin4 and it's asking me to add a server where do I go from here?
Oh I have got this now https://i.imgur.com/18xc5Kv.png
Yeah im here
Okay
So
Right-click on servers and press Create
Then call it like my_local_server or something
Call it what you want but that's the common name
I'm trying to remember how tf to do this lmao
Do I need to open the port in my firewall or smth?
Probably
aren't you already connected? go into schemas
if its the default settings:
host: localhost
port: 5432
username: postgres
password: none/postgres
also yeah you are already connected lol

your screenshot already shows you're looking at the default postgres db, I dunno if this is the database you meant to connect to, but it's a database
well, your tables are under schema folder
Still appreciate you helping
I think the "schemas" thing catches people out because it's not there on MySQL or sqlite. but that's a postgres feature, not a pgadmin one, you'll see it if you use any database browser
as it appears on dbeaver for example
@hazy mango what's the next step? Creating the tables?
Yep
To hold the userid and jump_url
Yea, so say ban_messages
Then the columns would be like jump_url of type text and user_id of type bigint kinda thing
All ids are classified as bigint btw
Pretty much I think
Make sure you create the field correctly though
E.g. making so they can't be None
https://i.imgur.com/GQFicWk.png type also none?
Okay but Not NULL? should be yes correct?
Yes
Alright I installed and imported asyncpg but I can't really find a setup guide or anything 
Nice
https://magicstack.github.io/asyncpg/current/api/index.html#connection-pools you want to make a pool and make it part of your bot
I can give example in a sec
class MyBot(commands.Bot):
"""Custom class for bot."""
def __init__(self):
...
# Create postgreSQL database pool
self.database_pool = self.loop.run_until_complete(asyncpg.create_pool(host="localhost", user="user",password="password", database="name"))
hey so i have a "database" in JSON and i'm trying to rework it into SQLite, could any of you help me? i've never worked with it before
I have a load of other stuff in that class but that's the main database bit
And then I actually have a function for running statements and stuff
https://paste.pythondiscord.com/osevafazut.py this is all the database stuff in my class @orchid yacht
Then I do likepy await self.bot.execute_sql('SELECT thing FROM table WHERE otherthing=$1', (val_for_$1), index=0)
i have some code right now that looks like this:
c.execute("UPDATE commtable SET " + arg1 + " = " + arg2 + " WHERE guild = " + guild_id)
that prints this:
UPDATE commtable SET rsample = y WHERE guild = 716030141234544681
but i get the error "OperationalError: no such column: y"
(commtable is the name of my table)
You need 'y' not y I'm assuming
@gaunt pasture
This isn't how you should do concatenation though in sql
I don't need to add those other database functions that you have do I?
so, like, str(arg2)?
I found I needed them to fix a bug but idk if you'll get it or not @orchid yacht
@gaunt pasture no
Just use the proper way
idk the proper way haha
What database are you using?
Okay will try with out first then
What database language
Sqlite, PostgreSQL, MySQL etc.
Then do```python
"UPDATE commtable SET ?=? WHERE guild=?", (arg1, arg2, arg3)
Something like that
File "C:/Users/Nik/PycharmProjects/beatsbot/beats.py", line 1957, in db
c.execute("UPDATE commtable SET ?=? WHERE guild=?, (arg1, arg2, guild)")
sqlite3.OperationalError: near "?": syntax error
i think maybe the question marks need parentheses but i thought i tried that already
@hazy mango so with the pool setup now I would want to make an on_message event have it check the ban log channel and look for the user_id and then send the userid and jump_url to the table right?
Wdym? @orchid yacht
Well I want to grab the userids and jump_urls now right?
async with self.bot.database_pool.acquire() as connection:
jump_url = await connection.fetchval("SELECT jumpurl FROM table WHERE userid=$1", (user_id,), column=0)
```iirc @orchid yacht
so how do i use a variable to call for a column
That grabs the info right? How does it get the info?
How does the table get the userids and jump_urls?
I don't get what you're saying
How would I add from now on?
Each time the bot sends a message about person being banned, insert into the database the message.jump_url and the user id of person that was banned
Doesn't matter if it's missing old messages
https://www.w3schools.com/sql/sql_insert.asp this is how to insert into a database
Sweet will have a look at that
@ me if you need any more help @orchid yacht, afk for a bit
Wil do! ty
so right now i have
"UPDATE commtable SET {col}={val} WHERE guild={gld}"
and it's telling me
discord.ext.commands.errors.CommandInvokeError: Command raised an exception: OperationalError: no such column: ***this is what val is***
so whatever {val} is, let's say it's "hello", it would say
OperationalError: no such column: hello
this is the whole line:
c.execute("UPDATE commtable SET {col}={val} WHERE guild={gld}".format(col=arg1, val=str(arg2), gld=guild_id))
I think it's converting the column name to string
wait, let me check something
Oh, the opposite is happening actually
You string doesn't have quotes around them.
So your query becomes update commtable set arg1 = hello whereas it should be update commtable set arg1 = 'hello'
@gaunt pasture
You can resolve this by using parameterized query. You should anyway not use string formatting for SQL queries, it's a security risk. change your query like this:
c.execute("UPDATE commtable SET {col} = %s WHERE guild = %s".format(col=arg1), ("Hello", 12345))
Your db library will automatically take care of the required quotes
col needs to be a variable too
so %s = %s?
@tepid cradle
File "C:/Users/Nik/PycharmProjects/beatsbot/beats.py", line 1956, in db
c.execute("UPDATE commtable SET {col} = %s WHERE guild = %s".format(col=arg1), (arg2, guild_id))
sqlite3.OperationalError: near "%": syntax error
No, you can't parameterize column names, that's why I left it like that.
Oh, and if you're using sqlite, replace %s with ?
i had that before and it said syntax error for ?
File "C:/Users/Nik/PycharmProjects/beatsbot/beats.py", line 1957, in db c.execute("UPDATE commtable SET ?=? WHERE guild=?, (arg1, arg2, guild)") sqlite3.OperationalError: near "?": syntax error
No, no. As I said, you can't parameterize column name
For column name, use string formatting. Use query parameters only for the values
Also, close quotes after your query
c.execute("UPDATE commtable SET {}=? WHERE guild=?".format(arg1), (arg2, guild)")
c.execute("UPDATE commtable SET {col} = ? WHERE guild = ?".format(col=arg1), (arg2, guild_id))
here's my line
putting more quotes gives it a red line
Hi,
Why do i need to upload pictures to a file in a model like in the example below?
isnt the purpose of a database to store data? or it gives the picture an id in the db and it is stored somewhere else?
picture= models.ImageField(upload_to='pictures/', default='pictures/none.png', null=True, blank=True)
c.execute("UPDATE commtable SET {col} = ? WHERE guild = ?".format(col=arg1), (arg2, guild_id))
@gaunt pasture This is right. Earlier you had closed the quotes at the end of the line, I think. Does this run fine?
Hi,
Why do i need to upload pictures to a file in a model like in the example below?
isnt the purpose of a database to store data? or it gives the picture an id in the db and it is stored somewhere else?picture= models.ImageField(upload_to='pictures/', default='pictures/none.png', null=True, blank=True)
@torn sphinx Databases store text data. If you want to store images in the database, you can convert them to base64 strings. But it's not advisable as it will increase the size of the database which 1. makes it slow, and 2. increases your bills because usually database storage space is more expensive than file storage
ok ,thanks, it is very clear now
is it possible to do something like UPDATE commtable SET {col} = ? IF EXISTS
what do you want to happen if it doesn't exist? @gaunt pasture
#IF THIS GUILD DOES NOT HAVE DATA
c.execute("INSERT INTO commtable (guild, " + arg1 + ") VALUES (?, ?)", (guild_id, arg2))
#IF THIS GUILD HAS DATA ALREADY
c.execute("UPDATE commtable SET {col} = ? WHERE guild = ?".format(col=arg1), (arg2, guild_id))
i tried
c.execute("IF NOT EXISTS (SELECT 1 FROM commtable WHERE guild = ?)", (guild_id))
but got this error:
File "C:/Users/Nik/PycharmProjects/beatsbot/beats.py", line 1952, in db
c.execute("IF NOT EXISTS (SELECT 1 FROM commtable WHERE guild = ?)", (guild_id))
sqlite3.OperationalError: near "IF": syntax error
@tepid cradle
I think the simplest way is to do a count. In pseudo code, this would be:
if count of guild > 0
then run update query
else
run insert query
what is "if count of guild > 0" in not pseudo code 😅
cuz that's the issue i'm having rn
But if the column is a primary key, you can also use on conflict with replace
https://www.sqlite.org/lang_conflict.html#:~:text=The ON CONFLICT Clause&text=The ON CONFLICT clause is,therefore might not be familiar.
i don't think it's a primary key
maybe it should be :p
again, first time working with this
cursor.execute("select count(*) from table_name where column = ?", (arg1))
value = cursor.fetchall()
if value[0][0] > 0:
#update query
else:
#insert query
@tepid cradle
c.execute("SELECT COUNT(*) FROM commtable WHERE guild = ?", (guild_id))
sqlite3.ProgrammingError: Incorrect number of bindings supplied. The current statement uses 1, and there are 18 supplied.
I think your guild_id is a list
?
The parameter that you're passing, guild_id. Is it int or list?
string
it's an int at first
just like 28572380509
and then i make it a string
Not sure what's happening in that case
Try running the queries in an interactive console
will help you figure out the error
no clue what that means
Your IDE has a console, or you can run it in the command line interactively by entering the python console
You can also use Jupyter Notebook
I have to go, getting late here. Hope you figure it out, all the best
alr
@hazy mango how do I change the port that it looks at? I am not using the default one
class MyBot(commands.Bot):
"""Custom class for bot."""
def __init__(self):
...
# Create postgreSQL database pool
self.database_pool = self.loop.run_until_complete(asyncpg.create_pool(host="localhost", user="user",password="password", database="name"))
I currently have this but it's not adding anything to the tables
class MyBot(commands.Bot):
"""Custom class for bot."""
def __init__(self):
...
# Create postgreSQL database pool
self.database_pool = self.loop.run_until_complete(asyncpg.create_pool(host="localhost", user="postgres", password="password", database="postgres"))
@bot.event
async def on_message(msg):
punishment_logs = bot.get_channel(736628279267557396)
embed = msg
if msg.channel == punishment_logs and msg.author.bot == False:
await MyBot.execute(f'''
INSERT INTO postgres (userid, jump_url) VALUES ({embed.footer.text}), ({msg.author.id});
''')
Please ping me when you respond
I have this code async def add_xp(users, user, exp): userguild = f'{user.id}/{user.guild.id}' db = sqlite3.connect('main.sqlite') cursor = db.cursor() cursor.execute(f"SELECT xp FROM users WHERE userguild = {userguild}") result = cursor.fetchone() expe = result[0] + exp sql = ("UPDATE users SET xp = ? WHERE userguild = ?") val = (expe, userguild) cursor.execute(sql, val) db.commit() and on the expe = line I am getting an error TypeError: 'NoneType' object is not subscriptable when I do expe = '2' it works with no errors is there something wrong with that line or with result or is there another problem please ping if you respond
Hello, I am having a bit of trouble with aiosqlite for python. The guys over at the discord.py server said you all could help.
@sinful condor you're trying to use the fetchone() function, which means that it will only find one. if you use fetchall() or fetchmany(size) it should work
@worldly lotus now it says list index out of range
@sinful condor oh i know, it's that it didn't find any results
try maybe checking your execute line
can we go into dms?
I have a few problems that I have with my database
I have been trying to convert this from json to sqlite
hello, i'm getting a strange error when attempting to do a migration in my django app. in my settings.py i have py DATABASES = { 'default': { 'ENGINE': 'django.db.backends.postgresql', 'NAME': os.environ['POSTGRES_DATABASE'], 'USER': os.environ['POSTGRES_USER'], 'PASSWORD': os.environ['POSTGRES_PASSWORD'], 'HOST': os.environ['POSTGRES_HOST'], 'PORT': os.environ.get('POSTGRES_PORT', '5432'), } }
my POSTGRES_HOST env var is set to ruby.db.elephantsql.com. when i run python manage.py migrate, i get: Traceback (most recent call last): File "/mnt/d/Python/formal/venv/lib/python3.7/site-packages/django/db/backends/base/base.py", line 220, in ensure_connection self.connect() File "/mnt/d/Python/formal/venv/lib/python3.7/site-packages/django/utils/asyncio.py", line 26, in inner return func(*args, **kwargs) File "/mnt/d/Python/formal/venv/lib/python3.7/site-packages/django/db/backends/base/base.py", line 197, in connect self.connection = self.get_new_connection(conn_params) File "/mnt/d/Python/formal/venv/lib/python3.7/site-packages/django/utils/asyncio.py", line 26, in inner return func(*args, **kwargs) File "/mnt/d/Python/formal/venv/lib/python3.7/site-packages/django/db/backends/postgresql/base.py", line 185, in get_new_connection connection = Database.connect(**conn_params) File "/mnt/d/Python/formal/venv/lib/python3.7/site-packages/psycopg2/__init__.py", line 127, in connect conn = _connect(dsn, connection_factory=connection_factory, **kwasync) " to address: Name or service not knownanslate host name "ruby.db.elephantsql.com
i'm not sure what happened to the error message on the last line, but that's what it gave me. here's the full error message, i left out most of it because it didn't fit: https://hasteb.in/puxefefe.sql
oh yeah... I forgot to ask my original question when I first made my comment. Sorry about that
ok i believe it's just a weird formatting error reading from the env vars although it appears perfectly fine whenever i print them
yeah just an extra newline lol it's working now
ok, so... for python aiosqlite does anyone know why would cursor.fetchone() not be returning an single tuple when I run an query?
Everything else would start doing this behavior as well. This would only happen with a few queries. Anyone know how I can fix up this behavior?
in sqlite3 i need a help to make my bot do something if the user is already existed in my data and if the user does not exist in my data the bot should do something else but i can't find a way to do it only this and it does not work
for record in records:
if record[0] == user_id:
print("true")
elif record[0] != user_id:
print("false")
Am in the process of switching from sqlite3 to aiosqlite, but I can't seem to ```py
import aiosqlite
NameError: name 'aiosqlite' is not defined
are you using an virtual envoiment? Double check to see if it is installed. You may of installed it to the wrong directory
If it wasnt installed, it would probably give modulenotfound or import error
import aiosqlite
ModuleNotFoundError: No module named 'aiosqlite'
aiosqlite is installed, but could be the wrong intepreter. Anyone think they could help?
with connection:
cursor.execute("""
SELECT coins FROM users WHERE ? = userid
""")
x = cursor.fetchone()
How could I add a value to them coins?
^ Using sqlite3 btw
do you mean change the value of the row coins or?
change the value of coins per user
@tough needleChange the users coins
I'm trying to just make a simple "add coins" command atm
UPDATE users SET coins = ? WHERE userid = ?
So i'm trying to select there coins to check if they are in the database if they are then im trying to figure out how to use the selected coins + the added coins (this will be an admin command) and then update the database
def add_coins(self, connection, user_id, added_coins)
with connection:
cursor.execute("""
SELECT coins FROM users WHERE ? = userid
""")
x = cursor.fetchone()
if x:
cursor.execute("""
UPDATE users SET coins = ?
""", ((x + added_coins),))```
Does that look about right?
I was planning on using "return" as a way to make my discord bot inform the user on the outputted coins and stuff.
(Sqlite3)
c.execute("SELECT * from cards WHERE collectionid = {}".format(cardid))
this is throwing an error due to cardid containing a hyphen, since it's trying to find the column of the stuff before the hyphen. Is there any way I can fix this so that sqlite reads the string as a string and not reading the hyphen as a minus symbol?
def add_coins(self, connection, user_id, added_coins)
with connection:
cursor.execute("""
SELECT coins FROM users WHERE ? = userid
""")
x = cursor.fetchone()
if x:
new_coins = x + added_coins
cursor.execute("""
UPDATE users SET coins = ? WHERE user_id = ?
""", (new_coins, user_id))
@tough needle That looking about right?
Try it out
Well I could if I currently had a value in the database
.-.
I havnt figured out how to add the data into it so
eh
INSERT OR IGNORE INTO users VALUES (row1, row2...)
wdym "ignore into"
if there already is data we don't want to overwrite it
That's why I was checking "if x:"
So "else:" would mean it hasnt selected coins
meaning they are not in the database
right>?
eh
nvm
^ I messed something up above it so
(Sqlite3, apologies for repeating but I don't want this to be lost and I need help still)
c.execute("SELECT * from cards WHERE collectionid = {}".format(cardid))
this is throwing an error due to cardid containing a hyphen, since it's trying to find the column of the stuff before the hyphen. Is there any way I can fix this so that sqlite reads the string as a string and not reading the hyphen as a minus symbol?
@tough needle Any ideas?
I had to change all this
Because discord.py cant use global variables so
how many tables r required to represent a class with a list of tuples that have a length 2? one for the class and another one with columns for key, tuple[0] and tuple[1]?
add_coins(user_id, added_coins)
RuntimeWarning: Enable tracemalloc to get the object allocation traceback```
^ Anyone any idea?
await
Just mention me if you have an idea how to fix it
await Command.__call__
What?
Read the error
@bot.command()
async def add_coins(ctx, coins:int):
await ctx.send('Testing.')
user_id = ctx.author.id
added_coins = coins
add_coins(user_id, added_coins)
#This is a seperate py file Below
def add_coins(user_id, added_coins):
connection = sqlite3.connect('maindb.db')
cursor = connection.cursor()
with connection:
cursor.execute("""
SELECT coins FROM users WHERE ? = userid
""")
x = cursor.fetchone()
print (x)
if x:
new_coins = x + added_coins
cursor.execute("""
UPDATE users SET coins = ? WHERE user_id = ?
""", (new_coins, user_id))
if connection:
connection.close()
I did
I read the error and I dont understand
That's the 2 peices of code
I know the top parts discord.py stuff but I'm not sure where the problems being caused
read the error to find out
connection = await sqlite3.connect('maindb.db')
Bro what
😂
def add_coins(user_id, added_coins):
connection = await
sqlite3.connect('maindb.db')
cursor = connection.cursor()
with connection:
cursor.execute("""
SELECT coins FROM users WHERE ? = userid
""")
x = cursor.fetchone()
print (x)
if x:
new_coins = x + added_coins
cursor.execute("""
UPDATE users SET coins = ? WHERE user_id = ?
""", (new_coins, user_id))
if connection:
connection.close()```
weird
discord.ext.commands.errors.ExtensionFailed: Extension 'cogs.database' raised an error: SyntaxError: 'await' outside async function (database.py, line 14)
:190: RuntimeWarning: coroutine 'Command.call' was never awaited
add_coins(user_id, added_coins)
RuntimeWarning: Enable tracemalloc to get the object allocation traceback
^ I added "async to it
async def add_coins(user_id, added_coins):
connection = await sqlite3.connect('maindb.db')
cursor = connection.cursor()
with connection:
cursor.execute("""
SELECT coins FROM users WHERE ? = userid
""")
x = cursor.fetchone()
print (x)
if x:
new_coins = x + added_coins
cursor.execute("""
UPDATE users SET coins = ? WHERE user_id = ?
""", (new_coins, user_id))
if connection:
connection.close()```
That's the code currently
then add await to add_coins in your command
discord.ext.commands.errors.CommandInvokeError: Command raised an exception: AttributeError: 'int' object has no attribute 'send' what does it even mean by that
I havn't tried to give it the attribute send
eh
@bot.command()
async def add_coins(ctx, coins:int):
await ctx.send('Testing.')
user_id = ctx.author.id
added_coins = coins
await add_coins(user_id, added_coins)```
I know that's discord.py stuff but
show errormessage
discord.ext.commands.errors.CommandInvokeError: Command raised an exception: AttributeError: 'int' object has no attribute 'send' what does it even mean by that
with mariadb (sql) I tried this and this doesn't work
I saw things similar working on websites and idk how to do it :(
is this deprecated ?
Traceback (most recent call last):
File AppData\Local\Programs\Python\Python38-32\lib\site-packages\discord\ext\commands\core.py", line 83, in wrapped
ret = await coro(*args, **kwargs)
File "", line 190, in add_coins
await add_coins(user_id, added_coins)
File \AppData\Local\Programs\Python\Python38-32\lib\site-packages\discord\ext\commands\core.py", line 354, in __call__
return await self.callback(*args, **kwargs)
File , line 187, in add_coins
await ctx.send('Testing.')
AttributeError: 'int' object has no attribute 'send'
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "C:\UsersAppData\Local\Programs\Python\Python38-32\lib\site-packages\discord\ext\commands\bot.py", line 892, in invoke
await ctx.command.invoke(ctx)
File "C:\Users\AppData\Local\Programs\Python\Python38-32\lib\site-packages\discord\ext\commands\core.py", line 797, in invoke
await injected(*ctx.args, **ctx.kwargs)
File "C:\Users\AppData\Local\Programs\Python\Python38-32\lib\site-packages\discord\ext\commands\core.py", line 92, in wrapped
raise CommandInvokeError(exc) from exc
discord.ext.commands.errors.CommandInvokeError: Command raised an exception: AttributeError: 'int' object has no attribute 'send'```
eh why's that went bold?
@bot.command()
async def add_coins(ctx, coins:int):
await ctx.send('Testing.')
user_id = ctx.author.id
added_coins = coins
await add_coins(user_id, added_coins)``` It's probably caused by that 2nd arg
remove :int
It needs to be an int
shouldn't hurt
yeah well
if you as an admin know that
the bot doesn't
so if you type in integers
then do int(coins) farther down
added_coins = int(coins) <= that should do
y
Same error without the "int"
That should have worked anyways?
I just re checked the docs
and it's definitely a thing
You pass user_id into the first parameter of add_coins
Which is ctx in that coroutine
That is your problem
In main I have:
@bot.command()
async def add_coins(ctx, coins):
await ctx.send('Testing.')
print(ctx)
user_id = ctx.author.id
added_coins = int(coins)
await add_coins(user_id, added_coins)```
In database.py in a cog I have
```python
async def add_coins(user_id, added_coins):
connection = await sqlite3.connect('maindb.db')
cursor = connection.cursor()
with connection:
cursor.execute("""
SELECT coins FROM users WHERE ? = userid
""")
x = cursor.fetchone()
print (x)
if x:
new_coins = x + added_coins
cursor.execute("""
UPDATE users SET coins = ? WHERE user_id = ?
""", (new_coins, user_id))
if connection:
connection.close()```
No of add_coins
heh?
oh
I can't seem to be able to convert from sqlite to aiosqlite
Command raised an exception: NameError: name 'add_coins_db' is not defined
I can install aiosqlite, but once I try to import it it does not work, module aiosqlite does not exist
define it then @sharp magnet
async def add_coins_db(user_id, added_coins):
connection = await sqlite3.connect('maindb.db')
cursor = connection.cursor()
with connection:
cursor.execute("""
SELECT coins FROM users WHERE ? = userid
""")
x = cursor.fetchone()
print (x)
if x:
new_coins = x + added_coins
cursor.execute("""
UPDATE users SET coins = ? WHERE user_id = ?
""", (new_coins, user_id))
if connection:
connection.close()
I have
That's defining it no?
heh
.-.
¯_(ツ)_/¯
@bot.command()
async def add_coins(ctx, coins):
await ctx.send('Testing.')
print(ctx)
user_id = ctx.author.id
added_coins = int(coins)
await add_coins_db(user_id, added_coins)```
discord.ext.commands.errors.ExtensionFailed: Extension 'cogs.database' raised an error: SyntaxError: 'await' outside async function (database.py, line 14)
It needs to be inside an async
no?
remove await
Command raised an exception: NameError: name 'add_coins_db' is not defined
Im so confused
¯_(ツ)_/¯
remove await
I did
@bot.command()
async def add_coins(ctx, coins):
await ctx.send('Testing.')
print(ctx)
user_id = ctx.author.id
added_coins = int(coins)
add_coins_db(user_id, added_coins)
``` ^ That's from main
```python
def add_coins_db(user_id, added_coins):
connection = sqlite3.connect('maindb.db')
cursor = connection.cursor()
with connection:
cursor.execute("""
SELECT coins FROM users WHERE ? = userid
""")
x = cursor.fetchone()
print (x)
if x:
new_coins = x + added_coins
cursor.execute("""
UPDATE users SET coins = ? WHERE user_id = ?
""", (new_coins, user_id))
if connection:
connection.close()
``` that's in a cog
show error
Traceback (most recent call last):
File "C:\UsersAppData\Local\Programs\Python\Python38-32\lib\site-packages\discord\ext\commands\core.py", line 83, in wrapped
ret = await coro(*args, **kwargs)
File line 191, in add_coins
add_coins_db(user_id, added_coins)
NameError: name 'add_coins_db' is not defined
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "C:\UsersAppData\Local\Programs\Python\Python38-32\lib\site-packages\discord\ext\commands\bot.py", line 892, in invoke
await ctx.command.invoke(ctx)
File "C:\Users\AppData\Local\Programs\Python\Python38-32\lib\site-packages\discord\ext\commands\core.py", line 797, in invoke
await injected(*ctx.args, **ctx.kwargs)
File "C:\UsersAppData\Local\Programs\Python\Python38-32\lib\site-packages\discord\ext\commands\core.py", line 92, in wrapped
raise CommandInvokeError(exc) from exc
discord.ext.commands.errors.CommandInvokeError: Command raised an exception: NameError: name 'add_coins_db' is not defined```
what is your cog file called
initial_extensions = ['cogs.example','cogs.database']
if __name__ == '__main__':
for extension in initial_extensions:
try:
bot.load_extension(extension)
except Exception as e:
print(f'Failed to load extension {extension}', file=sys.stderr)
traceback.print_exc()```
is it a class?
yep
then typ database before add_coins_db
heh
database.add_coins_db
