#databases
1 messages · Page 140 of 1
id do that either way
cause with the other thing ud basically do that aswell 😂
only difference is
i've been told
the less you request from mongodb
the faster it goes
user_name = member.user_name member.display_name, or member.name here
i don't think there's going to be a reasonable performance drop because of this
Will it work with MySQL?
sure the idea is the same
make a pool, assign it to a bot variable
well not exactly the same, you wouldn't do the loop.run_until_complete stuff because with mysql connector it's not async functions
but will asyncpg work with mysql?
no, asyncpg is for use with postgresql
what I'm telling you to infer from that message is not the module, but the idea being used
Do I have to do that but use mysql.connector?
Please don't ping helpers. I've never used postgres, and don't know what the best option for you is.
yeah that's the idea
Yeah, but with a question I was more familiar with. 🙂
instead of making a connection each time, on startup you call the function which creates the pool, and assigns it to a bot variable (which you can see being done in the message i linked) @worldly plaza
Would something like this work?
client.db = client.loop.mysql.connector.connect(pool_name = "mypool", pool_size = 5,host='localhost', user='root', password='open1234', database='discordbot', use_unicode=True, charset='utf8', port=3306)
no need of the client.loop stuff
Now do I refer to my connection as self.client.db?
And I don't need to define any connection since it connects on startup?
yes
Is there a way to get it to autocommit automatically?
dunno, refer mysql connector docs to see if that exists
Eh, I can just add self.client.db.autocommit = True to every command lol
i'm screwing up something obvious i think. I implemented a db class to use as a singleton, initialized before application start in fastapi: ```
class Database:
async def create_pool(self):
self.pool = await asyncpg.create_pool(
database=settings.db_name,
host=settings.pg_host,
port=settings.pg_port,
user=settings.pg_user,
password=settings.pg_password
)
async def execute_query(self, sql, *args, **kwargs):
async with self.pool.acquire() as connection:
try:
return await connection.execute(sql, *args, **kwargs)
finally:
await self.pool.release(connection)```
but whenever i do db.execute_query or try to implement something that does self.execute_query, i.e. ```
cursor = await self.execute_query(my_query)
print(cursor)
it prints the first word in my query, where i expect it to give me the results. am i crazy?
this is with asyncpg, and using psycopg went fine. relevant docs: https://magicstack.github.io/asyncpg/current/api/index.html#asyncpg.pool.Pool.execute
if you're trying to do some select query, you need to use fetch, instead of execute to be able to get the result set (execute just returns the response you get for the query from the database)
It is returning no error but it is also not doing what it is suppose to
@commands.command()
@me_check()
async def logging(self, ctx, channel: discord.TextChannel=None):
self.client.db.autocommit = True
cursor = self.client.db.cursor(buffered=True)
if channel == None:
cursor.execute(f"DELETE FROM logging WHERE guild = {ctx.guild.id};")
embed = discord.Embed(description=f'**Moderation logs are now off**', colour=main.colour)
embed.set_footer(text=f'Set by {ctx.author}')
return await ctx.send(embed=embed)
cursor.execute(f"DELETE FROM logging WHERE guild = {ctx.guild.id};")
cursor.execute(f"INSERT INTO logging VALUES('{ctx.guild.id}', '{channel.id}');")
embed = discord.Embed(description = f'**Moderation logs will be sent to {channel.mention}**', colour=main.colour)
embed.set_footer(text=f'Set by {ctx.author}')
await ctx.send(embed=embed)
thanks!
anand wanna do some more mongo magic? 🤪
it looks like you're trying to copy data from tables which are in different databases, and your driver doesn't support that (FeatureNotSupportedError)
ah sorry I gotta get going now, I've been in this channel for the past 2 hours now
i've made one myself, it runs in a couple of private servers
what does it do
oh not much, it's got the usual economy, tags stuff
ahhh
reminders, and also some functions to help us coordinate games
Hi if I have encountered problems with a pyspark program would this be the place?
yeah im making a more fun version of yui lmao
with a different more fun kind of economy I hope
aswell
and different themes and stuff
i've spent enough time in #discord-bots to know a lot about the library though :)
nice 👍
maybe run the query in the shell (psql)?
and well just learning how to use the libary since i literalyl started
like
last week lmao
ah right
all the best 👍
tyyy
now lets see how many users the bot can handle lmao
I can get it acces to 150k members in a second but im scared it's gonna be all buggy or slow 😂
honestly i've got no idea how this copying stuff worked, i answered that based on the error
if you're running this on your own computer, you can just search "psql" in the search bar
ah right
motor can be kinda slow at times :/
didnt you say it was better than mongo since its async
it's better than pymongo for use in async environments yeah
pymongo would block your bot
alright i gotta go now, good luck 👍
I am trying to create a pool however when I try and edit the database it doesn't edit it and then proceeds to not accept any commands and times out. Here is my pool code and my command code
client.db = mysql.connector.connect(pool_name = "mypool", pool_size = 5,host='localhost', user='root', password='open1234', database='discordbot', use_unicode=True, charset='utf8', port=3306)
@commands.command()
@me_check()
async def logging(self, ctx, channel: discord.TextChannel=None):
self.client.db.autocommit = True
cursor = self.client.db.cursor(buffered=True)
if channel == None:
cursor.execute(f"DELETE FROM logging WHERE guild = {ctx.guild.id};")
embed = discord.Embed(description=f'**Moderation logs are now off**', colour=main.colour)
embed.set_footer(text=f'Set by {ctx.author}')
return await ctx.send(embed=embed)
cursor.execute(f"DELETE FROM logging WHERE guild = {ctx.guild.id};")
cursor.execute(f"INSERT INTO logging VALUES('{ctx.guild.id}', '{channel.id}');")
embed = discord.Embed(description = f'**Moderation logs will be sent to {channel.mention}**', colour=main.colour)
embed.set_footer(text=f'Set by {ctx.author}')
await ctx.send(embed=embed)
Hello, I am using mongodb py def add_effect(self, effect, value): l = self.effects l[effect] = value print(l) teams.update_one({'_id': self.id}, {'$set': {'effects': l, 'rs': self.rs_cards, 'fs': self.fs_cards}})This is my code. print(l) prints {'hunting': datetime.datetime(2021, 3, 12, 22, 28, 7, 984281)} but when I get my data from mongo the effects dictionary is empty
Please ping me when responding
oo
Sorry, seems like I saved the id under 'id' and not '_id', damn my code is inconsistent
Well that didn't fix it unfortunately
with sqlite3 how could I check multiple columns?
For example I want to do this but idk if it will work:
sql = "DELETE FROM users WHERE id=? AND end_date=?"
the "AND"
could it be used?
@terse stump do you know how?
because I need to do some many other things, and I want to be sure it will work
It takes like 5 seconds to test a query with soem test db
Test before you move on rather than assuming someone's statement about whether it will or will not work
So my task.loop isn't removing the user from the db or their role
@tasks.loop(seconds=30.0)
async def checker(self):
conn = await aiomysql.connect(host='localhost', port=3306,
user='root', password='open1234', db='discordbot')
cur = await conn.cursor()
await conn.autocommit(True)
guild = await self.client.fetch_guild(814695010603368488)
time = datetime.now()
await cur.execute(f"SELECT user_id FROM muted WHERE clock <= {time};")
members = await cur.fetchall()
for member in members:
member = int(member[0])
usr = await guild.fetch_member(member)
muted = guild.get_role(815735188045692969)
await usr.remove_roles(muted)
await cur.execute(f"DELETE FROM muted WHERE clock <= {time};")
await cur.close()
conn.close()
Did you commit the changes?
autocommit seems to be a keyword argument for .connect, why are you doing await conn.autocommit(True)?
moreover, why are you still making a new connection object everytime?
I am still trying to figure out creating a pool, I think I got it tho. In regards to the problem the solution was simply...starting the loop.
This is python/postgres related:
I feel like I am very close on this. results is an object coming from tableA in a fetchall() as a tuple or 1000 sql rows. I am now trying to bulk insert those rows into tableB
#results = [(row) for row in results]
cursor.executemany('INSERT INTO master.mlb_events (cola, colb) VALUES (%s, %s);', (tuple(results,)))
conn.commit()
print(cursor.rowcount, "***** Record inserted successfully into table *****")```
The main changing factor here is the results object and I how I can insert it into the table with `executemany()`
In the above method I get back `TypeError: not all arguments converted during string formatting`
I've also tried `cursor.executemany('INSERT INTO master.mlb_events (cola, colb) VALUES (%s, %s);', (results,))`
but this returns another error `TypeError: not all arguments converted during string formatting`
I've also tried cursor.executemany('INSERT INTO master.mlb_events (cola, colb) VALUES (%s, %s);', [(row) for row in results])`
but that also gave me another error `TypeError: not all arguments converted during string formatting`
Anyone have experience with this?
How can I use databases in python? (Im new to databases I learnt SQL today only)
you'd use a database driver module which you can install with pip to connect to the database from your code
@torn sphinx which database are you using?
idk which to use
Im gonna use it for my discord bot
any suggestions?
I'd suggest postgresql
It is what is mostly recommended to be used with discord bots
the driver module you'd use is asyncpg
!pypi asyncpg
feel free to ask your questions, anyone who can answer will
the SQL looks fine to me
aight I installed this
so how would I use it?
huh, i don't know why that would be happening, can't guess without more context
is there any site I could learn how to use it
Well, the asyncpg docs is probably the best place https://magicstack.github.io/asyncpg/current/usage.html
tq
there's some pins in this channel which answers some common questions related to asyncpg as well, maybe check that out as well
i'm kinda busy right now i can't go through the entire thing 😅
tysm
Why am I getting this error:
psycopg2.errors.SyntaxError: syntax error at or near "12"
LINE 1: UPDATE users SET discordbotlist = 2021-03-14 12:50:04.849481
I am trying to add the current time to the timestamp
This is the code:
cur.execute(f"UPDATE users SET discordbotlist = {datetime.datetime.utcnow()} WHERE userid = {userid}")
conn.commit()
please don't ghost ping
the error is because that datetime needs to be in quotes
but a better idea is to just not use f-strings for SQL queries, and instead pass in the parameters separately
cur.execute("UPDATE users SET discordbotlist = %s WHERE userid = %s", (datetime.datetime.utcnow(), userid))
Ok
Hello everyone!
I'm running into this issue when trying to add the values to my newly added status field for the database.
I defined the column as such in Flask-sqlalchemy
status = db.Column(db.Enum('Unapproved', 'Approved', name='approval_status', server_default='Unapproved', nullable=True))
I didn't find a fitting explanation for my case as for what this "DETAIL: Missing left parenthesis." error means.
Can someone help?
I'm using a Postgres database and the plan is to add values to all fields so I can do a migration to turn the column into nullable=False
So I got that error to work but now I am getting this error:
TypeError: 'int' object does not support indexing
I am trying to recreate a Command which I have seen from another Bot. How the Command works? When running the Command the Client will show all the Logs of the Mentioned User. Bans, Unbans, Kicks, Warns, Mutes, Unmutes with their Reasons, Datetime, Moderator's Name. Everything has to be Per-Server and not Global. I have not tried anything yet because I am still trying to find out how it will work. I am using also a SQLite Database and not a .JSON File. Now I already have something in my mind which is:
First the Client will have to check if the Mentioned User ID exists in the Guild's ID inside of the DB. If not, it will send an error saying This User has no Logs. Otherwise it will send an Embed with the User's Logs. Seems easy but then I get confused. How would I show every stored Log of the User? Another problem is that I do not know how many Tables would I need for this, and also What will all these Tables do? Or maybe I will have to use 1 Table with many fields? That's where I get so confused. I would love if anyone could help me get any ideas of how this will work!
can't really answer without seeing any code, it just looks like you tried to index an integer (indexing is done on sequences, like strings, list etc)
One table would work fine here in my opinion - with these columns
guild_id
user_id
action
reason
infraction_time
mod_id
``` (you can also optionally add a case ID column which will autoincrement with something like a SERIAL datatype, to uniquely identify cases)
the guild ID column will help you make sure that the cases are per-server.
the type of the infraction (ban, unban, kick etc) can be stored in the `action` column.
whenever you run this command, you'd query something like
```sql
SELECT * FROM cases WHERE user_id = ? AND guild_id = ?
``` This will retrieve all the cases that user has in that guild. Check if the returned result set is an empty list (has length 0) - if this is true, it means that there were no matching rows in the table, meaning the user has had no logs.
The reason I say to store Moderator ID and not name is because names on discord can change but IDs will not
I see alright alright. Thank you a lot for the Idea!
Hi
I am using postgresgql + sqlalachemy and trying to make a SELECT query that checks if ARRAY is equal to any of given lists
heres how the array is defined as a model
path = db.Column(db.ARRAY(db.String), default=[])
so im trying to use in_
instrument_paths = [['FUTURE'],['FUTURE','CME']]
active_instrument_perms = DeliveriesPermissions.query.filter(
DeliveriesPermissions.path.in_(instrument_paths).all()
and getting
sqlalchemy.exc.ProgrammingError: (psycopg2.errors.UndefinedFunction) operator does not exist: character varying[] = text[]
LINE 3: WHERE deliveries_permissions.path IN (ARRAY['FUTURE'], ARRAY...
^
HINT: No operator matches the given name and argument types. You might need to add explicit type casts.
which probably means ARRAY doesnt support in_ clause
how'd request should be done instead?
i mean i need the same statement like
['FUTURE'] in [['FUTURE','CME'],['FUTURE']]
but in sqlalchemy style
Guys does anyone know how can I delete a row,using asyncpg?
DELETE FROM table WHERE ...;
Anyone can answer?
it sends the message 2 times even if there is no loop or smth like that. can someone identify the reason?
do you mean username column?
in that case you'd specify that
SELECT username FROM warnings WHERE Username = ?
this means that you're only gonna retrieve the username column where that condition is met (this query is kinda pointless
, but you get the idea - * means "give me all the columns", otherwise you can just specify the columns that you want)
@burnt turret can you also take a look at this
Working
I'd say you should check if you've got two instances of the bot running
like running token in the different scripts
check if you've got any extra python processes in the process manager
I stopped current one and the bot went offline...
🤔
you're in the wrong channel right now, maybe ask #discord-bots
there's only two reasons i can think of - running the bot process multiple times, and bot.process_commands(message) in a place where it wasn't supposed to be there
ups sorry
for that you'd remove the condition
(the condition is what you're specifying after WHERE)
will this work to create my user for asyncpg?
🤔 the = isn't needed
https://www.postgresql.org/docs/8.0/sql-createuser.html
then..you'd add a condition again
okay
after I change that is it fine?
should be i think
I saw that link
Postgres docs has examples at the very end, if the syntax at the top is confusing
Sure
you'll need to await close too
!d asyncpg.connection.Connection.close
coroutine close(*, timeout=None)```
Close the connection gracefully.
Parameters:**timeout** ([*float*](https://docs.python.org/3/library/functions.html#float "(in Python v3.9)")) – Optional timeout value in seconds. Changed in version 0.14.0: Added the *timeout* parameter.
so after that, is it ok?
Should be I think
you aren't passing any argument to it
Also, you're using execute with a SELECT query
RuntimeError Traceback (most recent call last)
<ipython-input-5-2b855f98e0cc> in <module>
6 await conn.close()
7
----> 8 asyncio.get_event_loop().run_until_complete(main())
C:\Program Files\Python38\lib\asyncio\base_events.py in run_until_complete(self, future)
590 """
591 self._check_closed()
--> 592 self._check_running()
593
594 new_task = not futures.isfuture(future)
C:\Program Files\Python38\lib\asyncio\base_events.py in _check_running(self)
550 def _check_running(self):
551 if self.is_running():
--> 552 raise RuntimeError('This event loop is already running')
553 if events._get_running_loop() is not None:
554 raise RuntimeError(
RuntimeError: This event loop is already running
``` with `asyncio.get_event_loop().run_until_complete(main())` and ```
RuntimeError Traceback (most recent call last)
<ipython-input-6-0645829bfbff> in <module>
6 await conn.close()
7
----> 8 asyncio.run(main())
C:\Program Files\Python38\lib\asyncio\runners.py in run(main, debug)
31 """
32 if events._get_running_loop() is not None:
---> 33 raise RuntimeError(
34 "asyncio.run() cannot be called from a running event loop")
35
RuntimeError: asyncio.run() cannot be called from a running event loop
``` with `asyncio.run(main())` ;-;
you will need to fetch as well
how are you running this file
in an ipynb
right I was thinking of something else, you can use an execute here
but you don't need to commit. you only commit after making some changes
Im running a jupyter nb cell
to retrieve values from it you'd do it the way you've been doing all along, nothing different
Looks like an asyncio error, maybe ask #async-and-concurrency
ok i used it in a py file and it gave a different error, https://mystb.in/AcquireNvidiaCriminal.sql. it has something to do with conn = await asyncpg.connect('postgresql://postgres@localhost/test')
Huh, are you sure the postgres server is running on your computer?
uhhh no?
check if it is
its not
if you don't have any variables to check, of course, you don't need the ? part
wait I have to download it?
SELECT * FROM table
``` - no use of any variables here
yes, you need to install postgresql to use it
oh tysm
If you know how to use docker, pulling a postgres image can often be easier
Why? userisn't being used anywhere in that query
that looks like an incomplete query
a) you aren't specifying what columns to retrieve
b) you haven't completed that query.
...WHERE
where what?
WHERE some_column = some_value
you aren't specifying any condition right now, you've just stopped at WHERE
still not quite yet there
you had the query right here lmao, you just weren't passing the user in
just complete it now
await db.execute("SELECT * FROM warnings WHERE usernames = ?", (user,))
Note the difference between what I've done and what you did
What you did -
(user)
What I've done -
(user,)
This is because if you simply put()around something, Python will not evaluate it as a tuple. To make a single element tuple you need a trailing comma (the comma afteruser)
The * means "give me all the columns"
you remove the *, and write the name of the column you want to retrieve
but you need to specify some column
So there's multiple stuff happening there
Did you understand that the ? is just a placeholder? and that the arguments we are passing in after the query (in your example (user,)) will replace that question mark?
right, so now what that does is
... WHERE usernames = user
``` It will look through the table, and only give you those rows, where the username column has a value that is `user`
Yeah
Well, not really
one second I need to refer something from the docs
Well it isn't hard either
Just be patient
So this is an example from the doc, quite similar to what we've got there
The query returns a Cursor object - this object lets you handle the data that the database returned
See how they are retrieving the data from the Cursor object - fetchone retrieves a single row
fetchall retrieves all the rows that matched the query
A row will be basically a python list once you do fetch
so fetchone will give you a plain list, and fetchall will give you a nested list; a list containing lists inside it, because fetchall retrieves all the rows
yeah
once you do this it's just playing around with indexing and such to manipulate the data
Do you know what indexing is?
no that's fine, i don't mind
How to write query which does pagination in Sql?
that'll get the last term
Negative indexing starts from the other side
Positive indexing starts from 0
a[0] gets you the first element, yes
so uh what should I put in my user in the SQL Shell (psql)? Im not able to create a user cuz server isn't running ;-;
simply pressing enter will use the default value
yes, if you used fetchall
then it defaults to postgres and asks for password ;-;
you'd have set one while installing it
that'll also work
Following the example from the docs would be
cursor = await db.execute(...query)
rows = await cursor.fetchall()
for row in rows:
# do stuff with the rows
That doesn't really seem like a database problem anymore
lmao quite a long time by the looks of it
I think I often end up answering your questions though because we're online at similar times 🤷♂️
glad I could help 😄
How would case_id be per server? I've set it as an INT and AUTOINCREMENT. But for some reason the Case number just goes up without checking if the guild's ID has changed
cursor = self.db.cursor()
cursor.execute("""
SELECT person_id
FROM ticket
WHERE guild_id = (?) AND message_id = (?)
""", (payload.guild_id, payload.message_id))
result = cursor.fetchone()
person = self.bot.get_user(int(result[0]))
can someone help me with this?
person = self.bot.get_user(int(result[0]))
TypeError: 'NoneType' object is not subscriptable
Auto increment increases the values for each row you insert.
And it doesn’t make sense what you are trying to do. Why should case Id go up if only the guild id has changed?
The result is returned as none. No match was found in the db for your query.
Basically what I'm trying to do is that every guild's Cases have to start from 0. So I thought of Auto Increment. That would just get the number Higher every time a new row is created. Well now I'm trying to set it so any time a new guild is saved it will just start from 0
That's basically the point
Well auto increment is not for that
await conn.execute('INSERT INTO config (bot_channel, muted_role, guild_id) VALUES (67890, 123, 123);')
row = await conn.fetchrow('SELECT * FROM config;')
for x in row.items():
print(x)
``` this gives ```py
('bot_channel', 12345)
('muted_role', None)
('guild_id', None)
``` what am I doing wrong (Im a big noob and I learnt SQL basics just today 😅)
(`12345`, `None` and `None` are the values from before)
What’s the issue?
its not inserting ig
Because you did fetchrow
I see, so I would have to code it so any time the guild changes it will have to add +1 number and remove the autoincrement?
Which only gets a single row
Is it really necessary for cases to be incremental like that? What value does it give?
It is not necessary but it is important at least for me to have that. Just so it looks a bit nicer when someone wants to get the Data of the user.
I don’t see how it looks nicer. Users don’t care what their case number is. Also many times your users shouldn’t have knowledge of how many cases are stored in your db.
If you really want you can use a window function, and partition by guild id to get the ordered cases.
Why shouldn't they?
I see, I may try that.
Then they have knowledge of how many cases are in a server. Should this be known?
Well yeah, also this will be a staff only command
Not anyone will have access to it but still it won't change anything if they know how many cases are in the servsr
what can I use for inserting a timestamp value in postgres?
So this is more a visual thing then, and should be handled on the front end/application side or with the window function as I mentioned previously. The incrementing case value shouldn’t be stored in the db because even if a case was deleted, then it would mean you have to recalculate for update all the case ids again.
Postgres has a timestamp datatype
This tutorial helps you understand the PostgreSQL timestamp data types: timestamp and timestamptz and shows you how to handle timestamp values.
i know
could I use datetime.datetime.utcnow?
asyncpg
async def on_member_ban(self, guild, user):
self.client.db = await aiosqlite.connect('main.db')
cursor = await self.client.db.execute()
async for b in guild.audit_logs(limit=1, action=discord.AuditLogAction.ban):
banner = b.user
await cursor.execute("SELECT COUNT(*) FROM whitelist WHERE guild_id=? AND user_id=?", (guild.id, banner.id))
count, = cursor.fetchone()
if count > 0:
await self.client.db.close()
return
ban_limit = 1
ban_secs = 20
after = datetime.datetime.now() - datetime.timedelta(seconds=ban_secs)
items = await guild.audit_logs(limit=ban_limit + 1, after=after, user=banner,action=discord.AuditLogAction.ban).flatten()
if len(items) > ban_limit:
embed = discord.Embed(title="test", description="\u200b", color=0x2f3136)
await banner.send(embed=embed)
await guild.ban(banner, reason="Anti-Nuke: [~] Banning Members")
await self.client.db.close()
nvm got it working
That's not actually needed. I think you have seen a ticket bot before correct? Well if yes, when a ticket is made and then Closed and deleted the next ticket starts from the next number. That's basically what is going to happen with the cases. If a case gets delete it will just continue as it would. There is no need to update it. As you mentioned in the beginning, it's just a visual thing I want to have.
The only thing that I was looking for was just to find a way to get cases per-server and not just Global
Here are the types of python/asyncpg https://magicstack.github.io/asyncpg/current/usage.html#type-conversion
Your execute function is empty.
Oh ok
How does a database work? Do I have to host it?
It depends on what database you wish to use, There is a built-in module for Python called SQLite3 that allows you to interact with a database as a file or in memory. This will be done using the SQL language and I'd recommend starting with this. For other databases like PostgresSQL, MongoDB, etc. you will need to run them somewhere, this can be locally or online.
I see
What is the difference between a database as a file or in memory and a databse that you could run online?
For in memory it will give you quicker read/write speeds but the amount of data you can store will be much more limited than as a file. Another good thing for using memory is that it is never saved, making it good for testing.
The file gives the benefit of persistent storage.
For online/offline one of the main things is network transmission, if you can have the database on the same device. Meaning it won't be accessed outside of that device. It is usually the better way to go. As you won't have to use the internet. Especially if its large amounts of data.
The reasons they exist are for different use cases.
its a good and easy way to do it. especially if you don't require to store massive quantities of data.
for example settings files usually/can be done is JSON/YAML or similar
Using a Database can help you store larger amounts of data, for SQL databases the best part is the langauge. You tell it how to query the database, then it does it all for you. This usually will optimise the way it actually retrieves the data.
The reason why I wanted to use databases is because I host a Python Discord bot on Heroku but Heroku follows the GitHub repository which means any changes done to the json files when the server was running will be reset and it will use the GitHub repository files again if I restart, that's why I wanted a database so I can store the data in a different way and to retain it.
It looks like Heroku has some built in support for database connections. https://elements.heroku.com/addons/heroku-postgresql
I've never looked into Heroku too far but if you can find anyways to make persistent files, then you could use a SQLite3 database.
^ seems to be free for >1gb storage. So if it is just small data. you should be alright
@torn sphinx But why would you use Google Spreadsheets when there are easier-to-use alternatives like SQLite?
@brave bridge If your working on a project that doesn't require too many users, and you don't want it to be local, spreadsheets would be good.
It could also be good for a Feedback form.
But why would you want it to be not local on a small project?
It's just increasing the latency by a hundred times.
if it integrates well with google stuff -- yes, sure
@brave bridge Local is only on one machine.
You wouldn't use it for a small global project.
small global project?
Was I speaking russian?
It can be used to let users around the world to send request.
You can't do that with a local database.
I think we're talking about different things.
If you want Google Spreadsheets primarily as a human-readable and human-editable platform with occasional automated access, then yes, it will work.
If you want to use it as a data store for an automated application, it's not a good choice:
- If only one machine is going to access it, it's extra overhead (100x increase in latency for no reason)
- If several machines access the database, it's not suitable either -- for example, spreadsheets don't support transactions as far as I can tell
How about a code, which has its own administrator account, the user can request the code, to input some values?
Would you use a local database for e-commerce use?
Account system, anybody from around the world?
Why not?
I thought local is just on one machine.
Yes, you'll run the database on the same machine as the server.
So it's a local server, for the user?
That's what @delicate field and our website do, for instance.
Not the end user. The end user makes requests to a server.
What if the user has a new machine, wanted to sign in to their account?
The "server" is not the software the end user runs.
The server is what responds to the user's requests.
What is the end user?
Hi. Someone have idea what's wrong with my mongodb? I have free tier from mongodb.com. When I try insert data to collection using python code I got this error and I do not have idea what's wrong...
No primary available for writes, Timeout: 30s, Topology Description: <TopologyDescription id: 604e86a0ba4670272e789ad8, topology_type: ReplicaSetNoPrimary, servers: [<ServerDescription ('mydb-shard-00-00.bxtvj.mongodb.net', 27017) server_type: RSSecondary, rtt: 0.048760000000707814>, <ServerDescription ('mydb-shard-00-01.bxtvj.mongodb.net', 27017) server_type: Unknown, rtt: None, error=AutoReconnect('connection closed')>, <ServerDescription ('mydb-shard-00-02.bxtvj.mongodb.net', 27017) server_type: Unknown, rtt: None, error=AutoReconnect('connection closed')>]>
@brave bridge How would it work if the server is local?
Like any client/server system
Your code is on the server. The user is the client. The client makes requests to the server. Server responds.
A server runs both a web application and a database. Users make requests to the web server.
Wouldn't the local server be on their machine?
No, it's on the same machine as the web server.
@brave bridge Is there a article where I can read this?
So the machine is running a web server, therefor, people around the world can send request to your machine?
So you would have your own server?
For example, @delicate field runs a Python program that interacts with the Discord API. On the same machine there's a Postgres database that stores infractions and other data.
who's "you"?
Yes. That's how all web applications work. You send a request to some machine over a network.
I was just saying it in a perspective of a person who would be using it.
@brave bridge Wouldn't that cost from bills?
You can delete it yes
When you type a message in here, your browser will send an HTTP request to some server belonging to Discord . When you open up a web page like https://google.com, you send an HTTP request to some server belonging Google.
DELETE FROM table_name WHERE some_condition
Well, both running an application and accessing the internet costs money, of course.
The end user pays their ISP, and the server owner pays some company that maintains some infrastructure for them (or they do it themselves).
@brave bridge Would it be best to hire a technician to review the machine every week? Similar to how google does it?
If you're running an application, well, yes, you need to maintain the infrastructure and fix issues if they come up.
There are other models like serverless computing, where you don't manage the infrastructure at all, but it's a different thing altogether.
What do you mean by 'hire a technician'?
You won't own or rent the physical machine (unless you do exactly that).
yo with SQLite how do i read whats in the DB?
@brave bridge Which database would I use if I wanted a fast query, with large amounts of users?
@brave bridge Account system.
you could try this:
SELECT id, COUNT(1) numrecords FROM messages GROUP BY id ORDER BY numrecords DESC LIMIT 3
not sure about the structure of your database or what problem you're trying to solve, but if messages.id is unique, then while this query will technically work, it'll probably not get you the answer you're looking for
LIMIT needs to come after order by, so at the end of the query
what is the best thing to use for mongodb?
hey guys
ughm, can you give me some advice on how i can design a train route database?
mainly i need someway or somehow to store individual routes that have stations in order, timings in which they arrive to each respective station and timings of departure of each station
so far i have used json to do this, i just write it to a file as an object, and each key points to an array of station, timings, and so on
but i really want to migrate to database style storage, i just don't know how exactly to approach this. Please @me if you can help
Is there a way to connect to database like postgresql on mobile like mongodb://localhost:27017/?
Any good reads on starting with sql
i use json rn but i think its bad
yo im getting started with asyncpg, and idk how to set up a db, someone plz help
is anyone able to help me with a case statement in mysql?
i know there is a better way to do this with just an if statement but im wondering why when i try to do the same thing with case this does not detected the null properly
How can I get the ping or response time of my database?
yo im learning asyncpg, how do i set up a database?
learn it first
how is one supposed to learn a tool without setting it up?
If you want zero setup and want to learn Postgres's SQL flavour, you can use a database-as-a-service like ElephantSQL. But keep in mind that all queries you send will have a much higher latency that if it were on your machine.
bro ty thats what im asying
okay bet
that was a response to Dylster 🙂
oh
What's your OS?
mine?
yes
You can download and install postgres here:
https://www.postgresql.org/download/windows/
But keep in mind that windows server 2008 (which is pretty ancient) is only supported up to version 10, as you can see from the table
nono not 2008
i didnt mean 2008
that was for smth else
ah
i have the latest version i think
you might also want pgadmin https://www.pgadmin.org/download/pgadmin-4-windows/, which is a very powerful GUI for postgres
but the question is: why do you need postgres?
how many users will your bot have, approximately?
i donttt knowwww
i want to get it to 250 servers by next month
so a few thousand
users
What are you using for storage now?
oh
i had a bot with 1k lines of code
but i scrapped it cause
it was using json
as a janky db
"db"
If you want very little setup, you can use SQLite: https://pypi.org/project/aiosqlite/
It should be good enough for thousands of users.
huh?
okok look
i watched a code academy course on it
and it taught me nothjing on ints
and im making a economy bot
wdym by ints?
it prints [<Record>, <Record>]
try:
search = await bot.db.fetch("SELECT * FROM config") # bot.db is a pool
except Exception as error:
print(f'There was an error ;-; \n`{error}`')
return
print(search)
``` shouldnt it print nothing? its an empty table Im using asyncpg
what language is that
for ur db
asyncpg
docs ig
okok
i got a questionnn
in the docs
theres this:
conn = await asyncpg.connect('postgresql://postgres@localhost/test')
wtf is this postgresql://postgres@localhost/test
@vestal shoal postgres is quite a bit more complex than SQLite. If you're already familiar with the basics of it, why do you want something else?
there are good tutorials like https://www.sqlitetutorial.net/ and https://sqlbolt.com/
because people say SQLite is trash
u have to start a postgres server (idk what its called) from ur pc
download postgres
also the big boys like syck and vco know asyncpg
I dont know asyncpg I just started yesterday ;-;
SQLite wouldn't be appropriate if you have hundreds of thousands of users. You shouldn't look away from an option just because someone vaguely referred to it as 'trash'
hmmm m m m mmmm
no but I do know French
hmmm
i got a questionnn for u fix error
do u know SQLite
I'm fairly familiar
but I don't know it in-depth
is there a way i could pay u to teach me
No, we don't allow paid work here.
You can take a look at these tutorials ^
If you want to pay, there are books like "Using SQLite" (Oreilly)
yeahhhhh
i got a uqestion
SO
why are different database languages good and bad
what makes them good orbad
like what makes JSON so bad
What kind of data do you need to store?
(sometimes with json the va;ies would randomly change)
okok
so
i wanna store usercash
and i wanna store items
and minerals
like 30 items
20 minerals
and one value for usercash
any help?
@torn sphinx @vestal shoal Perhaps you should each open a help channel, we have a concurrent write issue
ok
Re
I'm using pymongo and I want to append a object to another object like this:
{
"name": {
"data1": "test1",
"data2": "test2",
"data3": "test3"
}
}
If I have only the first object {} how I can append "name" to it?
That's not postgres tho
Alter table?
I just wanna use any kind of database on mobile
@bot.command()
@commands.cooldown(3, 30)
async def work(ctx):
workcash = ''.join(random.choice(string.digits)for i in range(3))
await ctx.send(f"You earned ${workcash} at work today!")
cursor.execute("UPDATE usercash SET usercash = usercash + ?", (workcash,))
connection.commit()
@work.error
async def clear_error(error, ctx):
if isinstance(error, commands.CommandOnCooldown):
await ctx.send("Whoa, you're on cooldown pal, don't overwork yourself!")```
why does no code below line `cursor.execute("UPDATE usercash SET usercash = usercash + ?", (workcash,))` work?
What happens?
ok
but i got a diff quiestion
plz
@rain plank ok so
cursor.execute("SELECT * FROM usercashdata WHERE memberid = ?", (author))
why doesnt that work ?
you missed a comma
so it isn't a tuple
cursor.execute("SELECT * FROM usercashdata WHERE memberid = ?", (author,))
np
Sqlite doesn’t support lists. However, can store a serialised version of it.
And depending on what the list is for you may want to convert the list into tabular form, which is generally better than storing a list.
how do i check if a record is already in my DB sqlite?
You can do:
SELECT EXISTS (SELECT * FROM table_name WHERE some_condition);
The query will return 1/0 depending on if row exists or not
However there may be better/efficient ways of doing this depending on what you want to do exactly if a record exists.
@vestal shoal
bro i was reading abt exists and it said SUBQUERY so i was confused as a bitch
tysm
Yeah so in the example above the subquery is everything in the brackets
Just ask your question, if someone is able to help they will reply. Saves time for everyone.
can i make a join of this with another table?
FROM
(SELECT position.position_id, NTILE(100) OVER (ORDER BY position.position_id) FROM position) AS buckets
GROUP BY 2 ORDER BY 2; ```
psql
Yes you can
how do i check if its true or false ?
You mean from python?
So what library are you using
SQLite 3
let me try, gimme a min
SELECT MAX(buckets.position_id), NTILE AS percentile
FROM
(SELECT position.position_id, NTILE(100) OVER (ORDER BY position.position_id) FROM position) AS buckets
JOIN position ON position.position_id = bucket.position_id
GROUP BY 2 ORDER BY 2; ```
ERROR: missing FROM-clause entry for table "bucket"
LINE 4: JOIN position ON position.position_id = bucket.position_id
db_conn = sqlite3.connect(...)
cursor = db_conn.execute(“query...”, (parameters,))
result = cursor.fetchone()
Hello! Need some help. I have my api https://my-ariana-grande-api.herokuapp.com/facts/2 and i want to just print out the fact in phyton, how do i do???
Because the column bucket.position_id doesn’t exist
Your window function will give a different name to that column
So give it an alias with AS
And then use that in the join
Use the requests module, or aiohttp
You can make a GET request, to that url and get the data as json.
someone already helpt me
but i already did give it the alias "buckets"
so where should i place that alias
Because you used the window function the default column name will change
SELECT MAX(buckets.position_id), NTILE AS percentile
FROM
(SELECT position.position_id, NTILE(100) OVER (ORDER BY position.position_id) AS posid FROM position ) AS buckets
INNER JOIN position ON position.position_id = posid
GROUP BY 2 ORDER BY 2;
like that?
ERROR: column "ntile" does not exist
LINE 1: SELECT MAX(buckets.position_id), NTILE AS percentile
Then you also have to update the column name in your select to use the alias
And it’s buckets.posid
Can you send a db fiddle, with your tables, and some test data. I will take a look when I get chance
An online SQL database playground for testing, debugging and sharing SQL snippets.
Hello is it possible to filter a document in mongodb with two fields like so:
document = collection.find_one({ 'field1': 'value', 'field2': 'value'})
Try it and see
It doesn't work
You want the document where both the conditions are met?
You'd have to join them with a $and most likely then
I'm on mobile so I'll try my best to type it out:
I switched to document = collection.find({ 'field1': 'value', 'field2': 'value'}) and it works but I need a for loop to iterate over the cursor in order to retrieve each document
Yes!
Thank you I really appreciate it
{"$and": [{"field1": "value1"}, {"field2": "value2"}]} something like this
Actually if it works for find I don't see why it doesn't work for find_one though🤔 are you sure the cursor returned by find actually contains documents?
@bot.command()
async def give(ctx, member: discord.Member ,amounttogive: int):
author = str(ctx.author.id)
cursor.execute("SELECT * FROM usercashdata WHERE memberid = ?", (author,))
print(cursor.fetchone()[1])
userbal = cursor.fetchone()[1]```
why doesnt this work ?
discord.ext.commands.errors.CommandInvokeError: Command raised an exception: TypeError: 'NoneType' object is not subscriptable
userbal doesnt work 💔
Yep, by using projection operator $and I got the document
Because it's None, so you can not access index of None
Does cursor.fetchone() return None?
hello can anyone help me?
time_now = datetime.now()
timenow = time_now.strftime("%Y-%m-%d") # because the enddate values are set in this format ^
query = """DELETE FROM inactive WHERE enddate =?"""
cursor.execute(query, (timenow,))```
basically my problem is, it deletes the whole enddate row
instead of a similar column (for example the date rn is march 15, so 2021-03-16, it would delete every data with similar date)
im using sqlite3
It returns one row right? I think that when you call fetchone inside print statement it returns the info properly but then you call it again the new value will be None because the pointer has already advanced past the only row in the result set
Need help resolving the error shown in the bottom - Im using SQLITE
table1 = """CREATE TABLE IF NOT EXISTS tblbusiness(BusinessName STRING PRIMARY KEY,
BusinessType STRING,
StartingRevenue FLOAT,
TotalRevenue FLOAT,
StartingCountry STRING)"""
cursor.execute(table1)
table2 = """CREATE TABLE IF NOT EXISTS tblcountry(CountryName STRING PRIMARY KEY,
BusinessName STRING,
FOREIGN KEY(BusinessName) REFERENCES tblbusiness(BusinessName),
StaffUpgrade BOOLEAN,
SizeUpgrade BOOLEAN,
TransportUpgrade BOOLEAN,
Xcoordinate FLOAT,
Ycoordinate FLOAT,
StoreIncome FLOAT,
DailySales INTEGER)"""
cursor.execute(table2)```
ERROR :
```py
Traceback (most recent call last):
File "C:/Users/N/PycharmProjects/tensorenv/test.py", line 26, in <module>
cursor.execute(table2)
sqlite3.OperationalError: near "StaffUpgrade": syntax error```
The foreign key constraints in the second table should be specified at the end of the statement.
dude i progressed a little bit
SELECT MAX(buckets.position_id), NTILE, buckets.course, buckets.speed, buckets.activity_id, buckets.lat, buckets.long FROM
(SELECT position.position_id, position.speed, position.course, position.lat, position.long, position.position_created_at, position.activity_id, NTILE(100) OVER (ORDER BY position.position_id) FROM position where activity_id = 8) AS buckets
GROUP BY NTILE, buckets.course, buckets.speed, buckets.activity_id, buckets.lat, buckets.long ORDER BY 2;
i'm kind of starting to understand how this works
BUT
if i just left the columns
MAX and NTILE
it just works
it gets the max of each tier of ntile
when i add every column of my table
it tells me i need to add it to the group by
when i do
it starts showing me more NTILEs
instead of just the max one
it shoes me 2
shows*
@bio.command()
async def set(ctx, *, message):
getBio = bios.find_one({"id": ctx.author.id})
if getBio is None:
newUser = {"id": ctx.author.id, "bio": message}
bios.insert_one(newUser)
await ctx.message.add_reaction('')
else:
bios.update_one({"id": ctx.author.id, "bio": message})
await ctx.message.add_reaction('')
why isn't it inserting in the database
bios = cluster['force']['bios']
i've figured out for bio.update_one() but .insert_one() isn't workin
Okay, basically
I got this:
def registerUser():
user_name = eName.get()
user_email = eEmail.get()
connection = connectDB('database\\db_users.db')
insertUser(connection, user_name, user_email)
connection.close()
# Elements
Label(window, text='Name', anchor=W, bg='#333', fg="#ccc").place(x=10, y=10, width=100, height=20)
eName = Entry(window, bg='#ccc', fg='#333')
eName.place(x=10, y=30, width=250, height=20)
Label(window, text='Email', anchor=W, bg='#333', fg="#ccc").place(x=10, y=60, width=100, height=20)
eEmail = Entry(window, bg='#ccc', fg='#333')
eEmail.place(x=10, y=80, width=250, height=20)
Button(window, text='Register', bg='#ccc', fg='#333', command=registerUser).place(x=10, y=110, width=100, height=30)
When i run THIS python file and execute the function, it works fine.
But when i use this on my MAIN program, it doesnt work
I call this file using exec(open(file).read())
It says that eName is not defined
idk what is happening
I might be missing something here, but is there a reason to execute the code like that vs just importing the code from a local file?
@toxic vector
from tkinter import *
import os
# Window Setup
app = Tk()
app.title('Main Window')
app.geometry('500x300')
app.configure(bg='#333')
window_folder = os.path.dirname(__file__)
# Functions
def newUser():
exec(open('insert_user.py').read())
def deleteUser():
exec(open('delete_user.py').read())
# Elements
menuBar = Menu(app)
menuUser = Menu(menuBar, tearoff=0)
menuUser.add_command(label='New', command=newUser)
menuUser.add_command(label='Delete', command=deleteUser)
menuBar.add_cascade(label='Users', menu=menuUser)
app.config(menu=menuBar)
# Window Main Loop
app.mainloop()
Basically i want to execute the file when i click on the cascade menu option
The way i do it doesn't matter, to be honest. If you know a better practice pls tell
Why is it that this works (writes to the database)
@bot.group(aliases=['w', 'white', 'whitelist'])
@commands.is_owner()
async def wl(ctx, member:discord.Member):
whitelistedUsers = whitelist.find_one({"id": member.id})
if whitelistedUsers is None:
addUser = {"id": member.id, "added by": ctx.author.id}
whitelist.insert_one(addUser)
But this doesn't write to the database?
@bio.command()
async def set(ctx, *, message):
getBio = userBio.find_one({"id": ctx.author.id})
if getBio is None:
newUser = {"id": ctx.author.id, "bio": message}
userBio.insert_one(newUser)
await ctx.message.add_reaction(':check:')
else:
userBio.update_one({"id": ctx.author.id}, {"$set": {"bio": message}})
await ctx.message.add_reaction(':check:')
I have also defined what userBio is:
userBio = cluster['force']['bios']
@frank fern
Hmm okay so how big is the code in insert_user and delete_user? Because I would recommend just importing those functions. assuming the files are in the same folder beside each other
from .insert_user import insert_user
and then on your menubar commands use an expression like
command=lambda: insert_user() to call them
the lambda is just there so the code doesn't get called when tkinter draws the GUI
alternatively, you could write your own class that inherits from tk.Toplevel or tk.Frame, and just include your insert/delete code as methods on that class.
here's the general approach I use
`"""The entry point for the program."""
import tkinter as tk
from .myGUI import myGUI
def main() -> None:
"""The Tkinter entry point of the program; enters mainloop."""
root = tk.Tk()
myGUI(root).grid()
root.mainloop()
if name == "main":
main()`
and then in myGUI.py you could do something like
`class myGUI(tk.Frame):
def init(self, parent):
tk.Frame.init(self, parent)
self.build()
def build(self):
# UI declarations here`
Hmmm okay
Until now i was not using classes
I'll be better if i use OOP for this type of project
yeah, most likely. tkinter is a little awkward but very flexible once you get the hang of it
All right, thank you for the help
no problem 🙂 good luck and have fun
bro can you help me with postgres?
I need to capture 100 percentiles from a table
i do it this way
(SELECT position.position_id,NTILE(100) OVER (ORDER BY position.position_id) FROM position where activity_id = 8) AS buckets
GROUP BY NTILE ORDER BY 2;```
this gets me the 100 ids of the 100 percentiles
but i also need to show the other data from the table
but if i add it to the first select
the MAX stops working cuz all of the data i have in the table is different
so do you know a way of doing this :X
like i said, this just doesn't work
SELECT MAX(buckets.position_id), buckets, buckets.ntile FROM
(SELECT position.position_id, position.speed, position.course, position.lat, position.long, position.activity_id, position.position_created_at , NTILE(100) OVER (ORDER BY position.position_id) FROM position where activity_id = 8) AS buckets
GROUP BY 3 ORDER BY 3;
i'm too stupid to make a working join XD
@bot.command()
async def give(ctx, member: discord.Member ,amounttogive: int):
author = str(ctx.author.id)
cursor.execute("SELECT * FROM usercashdata WHERE memberid = ?", (author,))
print(cursor.fetchone()[1])
userbal = cursor.fetchone()[1]```
why doesnt this work ?
discord.ext.commands.errors.CommandInvokeError: Command raised an exception: TypeError: 'NoneType' object is not subscriptable
userbal doesnt work 💔
it doesn't work
what i fixed it
Need some help with regex.
I need a regex query that will select all text starting with -----BEGIN PGP SIGNED MESSAGE----- and ending with the -----END PGP SIGNATURE-----
I need it to group each instance starting and ending with those lines as it's own group.
I got this far... Never got it working right. Any help here is appreciated...
(?<=[\-\-\-\-\-BEGIN PGP SIGNED MESSAGE\-\-\-\-\-])[.*](?=[\-\-\-\-\-END PGP SIGNATURE\-\-\-\-\-])
How could I get different Data from same user and get the result in a message which will look like this:
- Type: Warning - Reason: Bad Words
- Type: Warning - Reason: NSFW Emojis
What part do you need help with? Sql or python
mostly SQL, I can not find any way to select the rows and add them in different results
What I want to achieve is that, whenever someones runs the command it will shows the Data in the DB. only problem is that it will only get the result of 1 row of data. What I want it to do tho, is to get any row that has his user_id and get the result somehow. Only thing I have achieved at the moment is to get the Data of 1 row only
This for example
SELECT type, reason FROM warnings where user_id = ? AND guild_id = ?
Then from python you would do fetchall() or similar depending on your library to get all rows
Then it’s just a matter of looping through the result
oh so fetchall() will get any data that has the user_id and guild_id, correct?
Yes all rows
Ohhh, alright alright thank you a lot!
Is there a way to use databases on mobile?
there might be a few apps which would let you experiment with databases. You could also use Termux if you're on Android
im doing db's now looking into them a lot so i was wondering if im using the free version of mongodb does the db only hold a certain amount of storage
cause im doing a economy bot so like will it run out of storage to hold my digital currency
when I try to connect to an elephantsql postgre db with asyncpg, it throws me the error-
Postgresql at <urlstring of my db> rejected SSL upgrade```
this particularly started when I first made a pool connection to the db (maybe conincidentally), and now it's still erroring when I try to connect normally
I tried creating a new instance in the elephantsql, it's still throwing me the same error
is this something wrong with my host? because this started happening suddenly
Can I store a custom object in a SQLite dtatabase?
Hey @mental sleet!
It looks like you tried to attach file type(s) that we do not allow (.csv). We currently allow the following file types: .gif, .jpg, .jpeg, .mov, .mp4, .mpg, .png, .mp3, .wav, .ogg, .webm, .webp, .flac, .m4a.
Feel free to ask in #community-meta if you think this is a mistake.
Nvm I got it
SQL or MongoDB for lowest RAM usage possible?
If im using SQLite and I put my .db file in a folder, do i need to do something different to connect with it?
Because its in a folder
!d sqlite3.connect
sqlite3.connect(database[, timeout, detect_types, isolation_level, check_same_thread, factory, cached_statements, uri])```
Opens a connection to the SQLite database file *database*. By default returns a [`Connection`](#sqlite3.Connection "sqlite3.Connection") object, unless a custom *factory* is given.
*database* is a [path-like object](../glossary.html#term-path-like-object) giving the pathname (absolute or relative to the current working directory) of the database file to be opened. You can use `":memory:"` to open a database connection to a database that resides in RAM instead of on disk.
When a database is accessed by multiple connections, and one of the processes modifies the database, the SQLite database is locked until that transaction is committed. The *timeout* parameter specifies how long the connection should wait for the lock to go away until raising an exception. The default for the timeout parameter is 5.0 (five seconds).... [read more](https://docs.python.org/3/library/sqlite3.html#sqlite3.connect)
in a different folder*
i.e. you would switch from sqlite3.connect('test.db') to sqlite3.connect('folder/test.db') iirc
ok thx
any flask-sqlalchemy experts here? i need to execute an INSERT statement (to log progress) while running a long job that commits in the end. any idea how to solve it?
Has anyone managed to figure out how to seperate all model classes into a seperate file when using flask-sqlalchemy? I can't seem to create the database no matter what I do
Hi guys, I am using sqlite and was wondering why when I select an integer column in a database it returns it in the following form:
Note: This function was run 3 times
Doesn't matter
I didn't format the string oops
@dim dock yes, make a dir called models and in init_.py you put db = SQLAlchemy() and then import your models from separate files... something like that
it seems the issue was separate and not related to models being in a separate file, it seems flask_sqlalchemy just hates me
on that note, anyone wanna help me figure out why my database won't be created
i have had many problems with import loops, that's quite fun to mess with... well it's a problem with all models in the same file too
this is the code if anyone wants to take a look
doesn't work if I try to do it through the code or by importing the db and using create_all()
@dim dock no error?
File "<stdin>", line 1, in <module>
File "/opt/virtualenvs/python3/lib/python3.8/site-packages/flask_sqlalchemy/__init__.py", line 1039, in create_all
self._execute_for_all_tables(app, bind, 'create_all')
File "/opt/virtualenvs/python3/lib/python3.8/site-packages/flask_sqlalchemy/__init__.py", line 1031, in _execute_for_all_tables
op(bind=self.get_engine(app, bind), **extra)
File "/opt/virtualenvs/python3/lib/python3.8/site-packages/flask_sqlalchemy/__init__.py", line 962, in get_engine
return connector.get_engine()
File "/opt/virtualenvs/python3/lib/python3.8/site-packages/flask_sqlalchemy/__init__.py", line 555, in get_engine
options = self.get_options(sa_url, echo)
File "/opt/virtualenvs/python3/lib/python3.8/site-packages/flask_sqlalchemy/__init__.py", line 570, in get_options
self._sa.apply_driver_hacks(self._app, sa_url, options)
File "/opt/virtualenvs/python3/lib/python3.8/site-packages/flask_sqlalchemy/__init__.py", line 914, in apply_driver_hacks
sa_url.database = os.path.join(app.root_path, sa_url.database)
AttributeError: can't set attribute```
I even tried cutting down my User class to just
class User(db.Model):
__tablename__ = "Users"
id = db.Column('id', db.Integer, primary_key=True)
def __repr__(self):
return '<User: {}>'.format(self.id)```
still doesn't work
what if you just try the minimal example here https://flask-sqlalchemy.palletsprojects.com/en/2.x/quickstart/
nope
doesn't wanna work
just really really dislikes me I guess
from flask_sqlalchemy import SQLAlchemy
from flask import Flask
app = Flask(__name__)
app.config['SQLALCHEMY_DATABASE_URI'] = 'sqlite:////tmp/test.db'
db = SQLAlchemy(app)
class User(db.Model):
id = db.Column(db.Integer, primary_key=True)
username = db.Column(db.String(80), unique=True, nullable=False)
email = db.Column(db.String(120), unique=True, nullable=False)
def __repr__(self):
return '<User %r>' % self.username
db.create_all()```
this crashes
File "main.py", line 17, in <module>
db.create_all()
File "/opt/virtualenvs/python3/lib/python3.8/site-packages/flask_sqlalchemy/__init__.py", line 1039, in create_all
self._execute_for_all_tables(app, bind, 'create_all')
File "/opt/virtualenvs/python3/lib/python3.8/site-packages/flask_sqlalchemy/__init__.py", line 1031, in _execute_for_all_tables
op(bind=self.get_engine(app, bind), **extra)
File "/opt/virtualenvs/python3/lib/python3.8/site-packages/flask_sqlalchemy/__init__.py", line 962, in get_engine
return connector.get_engine()
File "/opt/virtualenvs/python3/lib/python3.8/site-packages/flask_sqlalchemy/__init__.py", line 555, in get_engine
options = self.get_options(sa_url, echo)
File "/opt/virtualenvs/python3/lib/python3.8/site-packages/flask_sqlalchemy/__init__.py", line 570, in get_options
self._sa.apply_driver_hacks(self._app, sa_url, options)
File "/opt/virtualenvs/python3/lib/python3.8/site-packages/flask_sqlalchemy/__init__.py", line 914, in apply_driver_hacks
sa_url.database = os.path.join(app.root_path, sa_url.database)
AttributeError: can't set attribute```
seems to be the same error
could it be because I'm using repl?
Is it a bad idea to skip the primary key?
What do you mean by skip?
Do you mean not include when inserting or not include in the table definition as a column?
For that you can use an auto increment
Ok thx
Hello guys, I'm having an issue when initiating a connection to MySQL command line client (version 5.1) from python IDLE. I don't have any user's other than root and it's offline. (The most basic setup- just self use & practice type)
So when I execute this command in script db=mysql.connector.connect(host='localhost', user='root', password='sql',database='s1')
A long error comes. Below is the error that was at bottom (in python shell)
ProgrammingError: 1045 (28000): Access denied for user 'root'@'localhost' (using password: NO)
Can anyone please help me with this issue?
@silent temple shouldn't it be mysql.connector.connect( ... )?
Oops my mistake! I wrote that wrong here. It's right in the python idle
@silent temple just a long shot after som quick googling: try to run 'pip install pymysql --upgrade' if that is the package that you're using
Okay I am trying it
@compact solar Installed it, now do I have to import `mysql.connector' ?
i think it's just import pymsql
Oh ok
then i guess change to pymysql.connect( .. )
maybe i set you off on a totally different route now but yeah...
Done, It's showing this error operational error 1045
Access denied for user 'root'@'localhost' (using password: No)
@silent temple guess you might have to try to reset your root pass then.. can't help you with that but maybe you get some idea here: https://stackoverflow.com/questions/2995054/access-denied-for-user-rootlocalhost-using-passwordno
Ok I will try that. Thanks a lot for your time and help.
hello! in sqlite3 why that's is wrong?
sql = ''' INSERT OR IGNORE INTO my_account (username) VALUES ?
SELECT * FROM my_account WHERE username = ?
'''
cur = conn.cursor()
cur.execute(sql, (username,))
conn.commit()
why is reasons returning None? when i do regular queries outside of this code it returns the string
reason = await bot.pg.fetchrow('SELECT reason FROM blacklist WHERE user_id = $1', ctx.author.id)
await ctx.send(f"You have been blacklisted from this bot for {reason['reason']}. Please contact MrKomodoDragon#7975 to be unblacklisted")
@compact solar It's working now. I have no clue how 😆
Hi there,
Any experience user in dealing with many-to-many sqlalchemy relationship filtering? I´m getting stuck on how to make work a filter in the associated table
@commands.command()
@commands.has_permissions(manage_messages=True)
async def modlogs(self, ctx, user : discord.Member):
db = sqlite3.connect("main.sqlite")
cursor = db.cursor()
cursor.execute(f"SELECT * FROM modlogsdb WHERE guild_id = {ctx.guild.id} AND user_id = {user.id}")
result = cursor.fetchall()
if result is None:
await ctx.channel.send(f"{user.id} has no logs.")
if result is not None:
cursor.execute(f"SELECT reason FROM modlogsdb WHERE guild_id = {ctx.guild.id} AND user_id = {user.id} AND type = 'Warning'")
result = cursor.fetchall()
for reason in result:
await ctx.channel.send(f"`Type:` Warning `Reason:` {str(result[0])}")
db.commit()
cursor.close()
db.close()
I'm currently having issues finding a way getting different types of data and paste it on a message.
For example
After running the command I get this
Even tho there are different types of reasons
maybe someone here can help me. I have a school project server and the only way i can manipulate the mysql server is by ssh into the server then logging in with a localhost DB account. is there a way or tool that i can interface with the DB? non-localhost connections to the database port is blocked. I cannot make a new user nor can i change the firewall settings.
depending on the db there's Workbench for MySQL and PGAdmin for PostgreSQL
its a mysql db
anyone here run integration tests with a real db? 🙂
@torn sphinx how mongo doesn’t help?
dont type here if u dont have ansdwers or questions related to dbs
This is database 🤦♂️
yes i know ur not helping so dont chat here
Depends where your bot is hosted
east
Then use the closest one
Im asking how a database doesn’t help, isnt that related to the topic of this channel? Lmao
Im not too sure but u can also selfhost
Read rules
If ur in a vps that the one with the least amount of latency
Whats a vps
Depending on the hoster he can't install custom things
Where you host your bot
Virtual Private Server
ik what it is lol
but idk how to do it
Then why’d u ask
ik what it is lol
Whats a vps

if u cant help dont chat
How do to it? There are installation docs I believe
"How do to it"
how do to it
Lmaoo
There are installation docs I believe
@torn sphinx dont tell me what to do, shut up.
Well guess we all can’t help you 🤷
@dull scarab he said shutup please ban em
😂
He can only make 15 commands ban him!
U ask me for codes😬
If you can't even search on google and expect us to do everything, then that's not the correct place. Sorry.
And now you're being cringe
one code that i alr had but it had an error
Instead of arguing here you could be looking at the documentation already and getting started
#databases message ban him!!!
i am
Your just flooding the chat
one ok
im trynna figure out whats closer
That has nothing to do with databases
No but since we cant help we wont chat right
cluster does
Latency? Not really
You should ask questions that are relevant
Clusters i can understand though
Then what are you doing here?
😂

i would be done if he was here
This guy ask something then say he already know the answer
Maybe your friend is not the reference?
And probably knows as much as you about databases?
I never said i know?
Ok as ur friend
"Ok as ur friend"
Only one flooding sincr the beginning is you, just as side note
Ask*
I dont care man!!!
Stop flooding!!
Insulting and then deleting is even worse
@brave bridge @lusty igloo Mute them!!
Man I thought u was nice
@dull scarab @analog sigil Man, these people are harassing me please ban them
they are being mean
@lusty igloo May you deal with this guy please? He's flooding/spamming, mass pinging moderators, being a troll and insulting. (He deleted it, I believe you have logs.)
Thanks in advance and sorry for disturbing you 
i dont like it
Yeah pls.
@torn sphinx please don't ping all the mods individually. If you'd like mod attention, reach out over @rugged locust.
@torn sphinx and everyone really, please stop accusing others of things, and trying to pick fights
If someone bothers you, don't deal with them. If they break rules, feel free to reach out to us.
Sorry man. My fault
Never tried in any way, just wanted to help him and then he got mad and started insulting me for no reason, he realized he said to someone else not to insult so he deleted his message, because we don't guide him on how to do stuff that he can definitely search and don't think it's that hard.
Ok.
But let's move on ^^
k
Ik this may start a flame war but really how would onw know when to use postgresql vs mysql.
I currently use sqlite as I don't really do too much data shifting, just data storage
would incrementing a counter of the number of installs of a software be counted as telemetry?
Hey guys! I created a method to insert emails (data type of the field is Text) in a db but whenever I try to add an email it returns this error. How do I solve it?
try wrapping the {} in quotes
"{var}"
It worked! Thanks💪 😉
👍
Can can anyone tell me can I add new row in a sqlite3 table?
yes @calm prawn
INSERT INTO tablename(column_name)
VALUES (value1, value2, etc...);
guys what's the problem here? I can't find it
!e
@glossy perch
print("foo"
"bar")
print("foo "
"bar")
print("UPDATE USER"
"SET EMAIL=foo")
print("UPDATE USER "
"SET EMAIL=foo")
@brave bridge :white_check_mark: Your eval job has completed with return code 0.
001 | foobar
002 | foo bar
003 | UPDATE USERSET EMAIL=foo
004 | UPDATE USER SET EMAIL=foo
Besides, you should never format SQL queries as strings: if someone enters '; DROP TABLE user;-- as their email, you're in trouble.
You should do:
cursor.execute("UPDATE user SET email=?", (new_email,))
This will format it correctly, and it's not prone to injection
Also, what do you actually want to do? This query will set a new email on all users
@brave bridge ooh ok i didn't know that! Thanks👀 😅
okay thanks
yes ik, i will put sql WHERE EMAIL="example@example.com"
np bud 😉
Can you provide a minimal example?
I mean databses are cool but do they get you a job
lol
i mean its tough
but at the same time easy
Well, most applications operate on some persistent data.
Quick question guys, are there any data migration tools available in python. I am aware of alembic but it’s for schema migrations. I was hoping for something specific to data migration which can also gel together with SQLAlchemy.
Don’t want to use alembic in a hacky way for data migration at the moment.
Thanks but these are schema migration tools. I was looking for data migration tools in production once the schema is migrated.
idk why this doesn't work. Can you help me guys?
The code looks fine. Are you sure your looking at the correct data?
Also the value of the email will be case sensitive in such comparison
yes ik
yes, for example i can add emails to db but i cannot delete those
Well as I said the query is fine, so there is some other issue.
Also looks like you have a global cursor. You should try to avoid that and create/dispose the cursor locally in the function
oh i didn't know that... Thx 🙂
i'l try to create the cursor locally in the function and use it, but that's not might be the problem
idk
Yes I’m aware the cursor is not the issue. That was a side note.
As I said before it’s probably due to the value you pass to the query
@glossy perch To debug add: conn.set_trace_callback(print)
Which will print queries being executed.
Also add print(cursor.rowcount) to the function
And show output of both
Replace the delete with “select count(*)”
And add: print(cursor.fetchall()) to the function
And show output
So that confirms no value exists in that database/table with the one your provided.
Although you think it does, it clearly does not. Maybe you are connecting to the wrong table/db or viewing the wrong data like I’ve said before.
hmm that's wierd! Because i just run the method(which is in the same class as the delete melthod) to create the new email and it worked perfectly
Create is different to select/delete
Just curious, what have you defined loop.run_until_complete as?
im doing db's now looking into them a lot so i was wondering if im using the free version of mongodb does the db only hold a certain amount of storage
cause im doing a economy bot so like will it run out of storage to hold my digital currency
whats the nosql equivalent os sqlite
pure python just for unit testing kind of deal
loop.run_until_complete(future)```
Run until the *future* (an instance of [`Future`](asyncio-future.html#asyncio.Future "asyncio.Future")) has completed.
If the argument is a [coroutine object](asyncio-task.html#coroutine) it is implicitly scheduled to run as a [`asyncio.Task`](asyncio-task.html#asyncio.Task "asyncio.Task").
Return the Future’s result or raise its exception.
Thanks
Hi I need help with some postgres stuff (asyncpg). In my Api (fastapi) I run one or two queries on some specific endpoints but the asyncpg spits out this message asyncpg.exceptions.TooManyConnectionsError: sorry, too many clients already after like 10 requests to the api. The only queries that are run are:
async def exc(pool, query):
async with pool.acquire() as con:
async with con.transaction():
return await con.execute(*query)
``` (+ another function which is con.fetch instead of .execute)
I tried printing out `print(await con.fetch('SELECT COUNT(*) from pg_stat_activity;'))` this (the amount of connections) and this appereantly increases by 10 by each request which is very weird because thats the only requests that are being sent?
and then it dies when it reaches 100 connections but I dont see why it is 100 connections in the first place, im using the pooling system
hello
this collection have the field LibSousFamille
and others foreign keys : LibFT, LibDM
how can i display the data of these foreign keys?
ALTER TABLE table ADD COLUMN column_name type restraint --I think
okay thanks
row != column
let me try
oh woops read it wrong
INSERT INTO table (the, coloumns, you, want, to, add) VALUES (the, values, for, each, column);
It adds new value
I mean new column
oh right so yeah the first one
could someone please help?
I'm using PyMODM with mongodb, but I don't see documentation on the objects attribute for Models anywhere.
Hey guys. Does the cluster location like aws, azure, etc improve ur ping? | ping if answer
Yes
Which one should i use?
Chose the one closest to where your users will be
Hi all,
I am trying to decode some specific fields (already encoded base64) across the collections while retrieving from mongodb using pymongo. Is there anyway to achieve this in pymongo / python? Any help would be appreciated.
Hello guys, I want to build my own passwordmanager in python. I think about using Sqlite. Is it a good db for my project?
or should i use something else?
You can use any database for this, as long as you hash the passwords before storing them in the database.
very noice
Can someone explain me why session.query(BDay).filter(BDay.last_notify != datetime.today()).all() gives me 0 rows and session.query(BDay).filter(BDay.last_notify.isnot(datetime.today())).all() gives me the expected.. The BDay.last_notify is null in this case
hi
Hey @torn sphinx!
Uh-oh! It looks like your message got zapped by our spam filter. We currently don't allow .txt attachments, so here are some tips to help you travel safely:
• If you attempted to send a message longer than 2000 characters, try shortening your message to fit within the character limit or use a pasting service (see below)
• If you tried to show someone your code, you can use codeblocks
(run !code-blocks in #bot-commands for more information) or use a pasting service like:
hey guys. im getting an error. error: https://paste.pythondiscord.com/enebeqeyuz.sql
guys i have a quick question: Should I commit the file that contains the credentials encryption methods to DB, on github? (The repo is public)
is it necessary to close a database and what happens if i dont?
I think yes, because if you dont your db could corrupt for some reason and if that happens you probably need to insert all the data manually.
Basically, close the db for the data safety 🙂
Generally nothing will happen unless you are in the middle of a transaction in which case it will be rolled back.
You can create an .env file for holding sensitive data/ or configuration options.
The env is kept locally and not checked into version control
Hie guys
I'm hopping to be assisted anyhow
I'm working on a project it's a GUI based system(PyQt5) and I need this app to access database on a network
I've decided to use django to host the db on a network
how best can i connect my GUI app with the database and be able to do general database stuff(updating,deleting,etc)
the server and database already up and running and accessible across the network
Where is the GUI app running?
On the same machine as the db or separate?
GUI app will be installed on multiple computers all connected on the same network
Ok so you can have the django as an api which the clients (gui app) will connect to.
And so the django app will be the one communicating with the database.
Oooh ok such that GUI app sends requests then django will return the response?
Exactly
ahh aight thanks :D
thank you very much
What does this mean? MongoDB
you're trying to use indexing on a Cursor
https://gist.github.com/anand2312/840aeb3e98c3d7dbb3db8b757c1a7ace#sidenote-why-arent-finds-awaited here are examples of working with Cursor objects
How do I cast a string column to Decimal/Float in sqlalchemy during query
PostgreSQL or MySQL or MariaDB
@burnt turret Then why am I getting this error?
MongoDB! ^^
Why should I use NoSQL(MongoDB)
Ok
What are you working on?



