terse stump May 31, 2018, 10:32 AM

#

you are mixing Grafana with Graphite (this is a data source)

stoic pendant May 31, 2018, 10:33 AM

#

ah, i see

terse stump May 31, 2018, 10:34 AM

#

so use the expertice here to figure out what data source to use for your problem, then check if grafana supports it. 😄

stoic pendant May 31, 2018, 10:34 AM

#

k, gotcha 😄

stoic pendant May 31, 2018, 10:50 AM

#

@viral crag What would you suggest using, if you had to choose one or two, for my project?

viral crag May 31, 2018, 11:24 AM

#

We use rethinkdb

#

but postgresql is a great SQL-based database

stoic pendant May 31, 2018, 2:26 PM

#

thanks, i'll give these a look and get back to you 😃

orchid charm Jun 1, 2018, 12:20 AM

#

            self.bot.c.execute('SELECT SCN from Profile where UID = ?', (ctx.message.author.id,))
            result = self.bot.c.fetchone()

#

    if result >= bet:
TypeError: '>=' not supported between instances of 'tuple' and 'int'

#

(406,)

Why is the result a tuple?

#

No matter what i do it comes as a tuple

#

In other cogs this does not happen though

deep glacier Jun 1, 2018, 12:28 AM

#

because you may fetch more than one item (per row)

#

and having the library automatically do (a,) -> a for single column selects would be inconsistent

hard rapids Jun 1, 2018, 12:24 PM

#

hey guys

#

could i drop a question here?

#

ive got a pretty dumb question but i cant for the life of me find an answer

unreal tartan Jun 1, 2018, 12:30 PM

#

bot.tags['ask']

delicate fieldBOT Jun 1, 2018, 12:30 PM

#

ask

Asking good questions will yield a much higher chance of a quick response:

• Don't ask to ask your question, just go ahead and tell us your problem.
• Try to solve the problem on your own first, we're not going to write code for you.
• Show us the code you've tried and any errors or unexpected results it's giving
• Keep your patience while we're helping you.

You can find a much more detailed explanation on our website.

hard rapids Jun 1, 2018, 12:31 PM

#

yeah alright then

#

AttributeError: 'pyodbc.Cursor' object has no attribute 'callproc'

#

im basiavlly getting that error

#

when following this documentation

#

http://docs.sqlalchemy.org/en/latest/core/connections.html#calling-stored-procedures

#

and im clueless as to why an almost direct replica doesnt work in my scripts

ornate abyss Jun 1, 2018, 6:45 PM

#

what is pyodbc

#

@hard rapids

#

Oh nvm i found the repo

#


This is not yet supported since there is no way for pyodbc to determine which parameters are input, output, or both.

You will need to call stored procedures using execute(). You can use your database's format or the ODBC escape format. For more information, see the Calling Stored Procedures page.```

hard rapids Jun 1, 2018, 7:35 PM

#

@ornate abyss thanks I will look into it more tomorrow

#

but it seems weird that they include it in the docs without there being actual supporrt to me....

ornate abyss Jun 1, 2018, 8:05 PM

#

That was taken from their docs

novel wharf Jun 2, 2018, 12:47 AM

#

@light summit How's Redis with expiring data? Is there a default or do I have to specify?

#

I'm having some pretty big problems with client sessions that get stuck in the database. If they or the server crash, they aren't removed properly.

#

I'd actually like to call a redis.flushall() everytime the server restarts, but I have no idea how to pull that off nicely in Django.

torn sphinx Jun 2, 2018, 10:41 AM

#

Hello. I am trying to create a currency system. I have 2 questions. How can I add coins to people database without having to reset the database? and then how can I make it add up coins and not replace the previous one? Fir example if he levels up he gets 10 coins. on the next level he gets 40 coins. but it wont add 40 to 10 it will jsut replace 10 to 40 it shouldbe 50

#

lv.update({'crystals': 10}, Levels.id == message.author.id)

#

How can I instert to everyones data ````crystals` :0```

ornate abyss Jun 2, 2018, 2:49 PM

#

@novel wharf You can specify a TTL on keys with redis if thats what you mean?

#

@torn sphinx I dont understand your question

novel wharf Jun 2, 2018, 2:52 PM

#

I want to make a simple user list, or any list really of connected clients. If expire only works on keys I'm not sure how it's helpful.

light summit Jun 2, 2018, 3:02 PM

#

@novel wharf you're wanting to completely erase your database every time it restarts?

#

What?

novel wharf Jun 2, 2018, 3:03 PM

#

Because I only use Redis for temporary game sessions

light summit Jun 2, 2018, 3:03 PM

#

I thought you were storing play records in it, too

novel wharf Jun 2, 2018, 3:03 PM

#

Nah, that's stored in Django's models

light summit Jun 2, 2018, 3:03 PM

#

õ.ô

novel wharf Jun 2, 2018, 3:04 PM

#

But I guess we figured it would be faster to use Redis and expire whatnot for game sessions that are to disappear the moment they're finished.

#

Like storing userdata and statistics in the regular database, and using Redis for disposable data. The game has to be stored somewhere while it's active

sick badge Jun 2, 2018, 3:25 PM

#

hey I'm doing db management through python and i'm getting an error that says db is locked whenever i try and change anything

#

is there a way i can fix that

#

I'm using sqlite3

ornate abyss Jun 2, 2018, 3:29 PM

#

@novel wharf When you say a list, do you literally mean the list type in redis, and would you want the entire list to expire or each individual element of that list to expire at their own rates?

novel wharf Jun 2, 2018, 3:32 PM

#

"users":
    {
        "name": "Golen",
        "session": "<sessionId>",
        "time": ...
    }
    ...

#

This is what I'd like to store, and is how I'm currently doing it. Whenever a client connects they create a new consumer websocket connection, but I have no idea how to track that unless I do it myself.

#

And since you can visit the same page with multiple tabs, I use sessionId to separate the users to avoid some issues.

#

Then using my own timestamp thing to keep track of expired sessions, which happens all too often, such as when the client crashes or server resets.

ornate abyss Jun 2, 2018, 3:34 PM

#

So how are you currently storing

    {
        "name": "Golen",
        "session": "<sessionId>",
        "time": ...
    }

#

like each connection

novel wharf Jun 2, 2018, 3:35 PM

#

redis.set("users", userlist)

#

That dict is one user. Imagine a list of 10 such.

ornate abyss Jun 2, 2018, 3:36 PM

#

Ah i see

#

Yea you cant individually set a TTL per set item with redis

#

Could you not instead have the sessionID as the key and use a hash?

novel wharf Jun 2, 2018, 3:37 PM

#

https://github.com/django/channels_redis

GitHub

django/channels_redis

channels_redis - Redis channel layer backend for Django Channels

#

So I'm actually using this, and they're doing a handful of redis stuff behind the scenes.

#

Like zadd whenever a new user connects, and a lot of hashing. I don't understand half of the source.

#

Here they add a TTL to each group_add thing, such that a user is disconnected within certain time, although it doesn't detect dead clients.

#

So if you play around for a while, everything starts slowing down immensely because of redis trying to group_send messages to a million dead clients.

#

And if you decreate the TTL time to something decently small, the package doesn't have a way of updating it in case a client pings or whatnot. They could be in the middle of a game and get kicked.

#

When you say using session as the key and use a hash, do you mean redis hset?

#

Like perhaps hset("users", sessionId), and then hscan to retrieve all sessions.

ornate abyss Jun 2, 2018, 4:05 PM

#

Sorry was away for a bit

#

Well hset yea because then you can have each individual connection have its own TTL and expire through redis

novel wharf Jun 2, 2018, 4:07 PM

#

That sounds really handy!

#

How would I increase the expiry upon a ping? I want to keep it low, but refresh it every so often.

ornate abyss Jun 2, 2018, 4:17 PM

#

There is a Redis command called EXPIRE

#

127.0.0.1:6379> set t 10
OK
127.0.0.1:6379> expire t 100
(integer) 1
127.0.0.1:6379> ttl t
(integer) 97
127.0.0.1:6379> ttl t
(integer) 95
127.0.0.1:6379> ttl t
(integer) 94
127.0.0.1:6379> ttl t
(integer) 94
127.0.0.1:6379> expire t 100
(integer) 1
127.0.0.1:6379> ttl t
(integer) 99

#

Just as like a little example

#

apologies for the late reply again

novel wharf Jun 2, 2018, 4:20 PM

#

Don't worry 'bout it

orchid charm Jun 2, 2018, 5:18 PM

#

In SQL is there a way to wait for the current things that are reading/writing to the DB to finish?

#

Cause i got a "reroll" loop and every hour it does some SQL stuff

#

But it usually locks when another SQL event is happening at the same itme

#

Like a user chatting, or using one of my commands, ect.

ornate abyss Jun 2, 2018, 5:21 PM

#

You will have to wait to acquire the lock, so im not sure what you mean

#

@orchid charm

orchid charm Jun 2, 2018, 5:22 PM

#

?

#

Well what is locking my SQL then if it ain't the Every hour DB function and a user executing SQL stuff at the same time?

#

📎 Screen_Shot_2018-06-02_at_10.23.41_AM.png

#

@ornate abyss

ornate abyss Jun 2, 2018, 5:26 PM

#

im not sure how you thought i was saying nothing is locking your db??

orchid charm Jun 2, 2018, 5:27 PM

#

Then what were you saying XD

#

Brain.exe never worked in the first place

ornate abyss Jun 2, 2018, 5:28 PM

#

I was saying, basically i thought you were thinking it didnt wait to try to acquire the lock (it does) however i didnt know that you were meeting the timeout

#

which iirc is 5 seconds by default

orchid charm Jun 2, 2018, 5:28 PM

#

So how would i fix this

#

And prevent it from locking

ornate abyss Jun 2, 2018, 5:29 PM

#

A quick workaround could be try to increase the timeout

orchid charm Jun 2, 2018, 5:30 PM

#

How would i increase the SQL timeout?

#

and what is the disadvantages to increasing the timeout

ornate abyss Jun 2, 2018, 5:30 PM

#

when you do the sqlite.connect("some_db", timeout=x)

#

when its waiting to send the connection, that timeout will be blocking

#

so it could take up to 10 seconds for the next line of code to be executed

#

assuming you choose 10 to be your timeout

orchid charm Jun 2, 2018, 5:31 PM

#

Ah

#

Well thats p simple 👍

#

Ima do that

ornate abyss Jun 2, 2018, 5:31 PM

#

yea, if you still get exceptions then youll have to handle them and retry the queries at a later date

#

Or if this becomes a massive issue, migrate away from sqlite

dull scarab Jun 2, 2018, 5:32 PM

#

Won't it become an issue if you're using it with an async bot?

ornate abyss Jun 2, 2018, 5:33 PM

#

tbh i didnt even notice it was a bot

#

Yes it will

orchid charm Jun 2, 2018, 5:33 PM

#

Ah

#

So how would i do it and not have that be a issue then

ornate abyss Jun 2, 2018, 5:34 PM

#

I think migrating away from sqlite in this instance is your best bet

dull scarab Jun 2, 2018, 5:34 PM

#

👌

ornate abyss Jun 2, 2018, 5:34 PM

#

Using something like postgres with an async library will work best

orchid charm Jun 2, 2018, 5:34 PM

#

Well what works almost exactly like SQL and can write multiple times at once?

ornate abyss Jun 2, 2018, 5:34 PM

#

sqlite is based on postgres so the SQL queries themselves wont need changing

dull scarab Jun 2, 2018, 5:34 PM

#

Not "at once", but seemingly

orchid charm Jun 2, 2018, 5:34 PM

#

Also how exactly do i migrate to postgres ;-; ive been trying XD

#

I am using a Mac OSX

ornate abyss Jun 2, 2018, 5:35 PM

#

Are you running this bot on your mac?

orchid charm Jun 2, 2018, 5:35 PM

#

Yes

#

Soon i will probably get a VPS tho

ornate abyss Jun 2, 2018, 5:35 PM

#

I see, well youll need to install postgres and set that up

orchid charm Jun 2, 2018, 5:35 PM

#

postgres.app?

#

If so i think i already got it

ornate abyss Jun 2, 2018, 5:35 PM

#

https://www.postgresql.org

PostgreSQL: The world's most advanced open source database

The official site for PostgreSQL, the world's most advanced open source database

orchid charm Jun 2, 2018, 5:36 PM

#

@ornate abyss so i should use postgres.app?

ornate abyss Jun 2, 2018, 5:37 PM

#

i dont know what that is

orchid charm Jun 2, 2018, 5:37 PM

#

📎 Screen_Shot_2018-06-02_at_10.37.09_AM.png

ornate abyss Jun 2, 2018, 5:37 PM

#

what i linked is the postgres website, there is a download tab

#

looks good

orchid charm Jun 2, 2018, 5:37 PM

#

Ok opening it

ornate abyss Jun 2, 2018, 5:38 PM

#

Then youll need to find an async library to interface with it

orchid charm Jun 2, 2018, 5:38 PM

#

📎 Screen_Shot_2018-06-02_at_10.37.58_AM.png

#

@ornate abyss Which one would work most like SQL?

#

SQL3*

#

Also is commiting needed

ornate abyss Jun 2, 2018, 5:47 PM

#

I don't understand the question

dull scarab Jun 2, 2018, 5:48 PM

#

https://gist.github.com/Altarrel/cfc7b22e72426f5ced6f87caa6920fd6

Gist

Example code for connecting to and using a postgres database using...

Example code for connecting to and using a postgres database using discord.py and asyncpg.

orchid charm Jun 2, 2018, 5:49 PM

#

@ornate abyss Commiting is Saving to a Database

ornate abyss Jun 2, 2018, 5:49 PM

#

No I didn't mean that part

#

And I know what committing is

orchid charm Jun 2, 2018, 5:49 PM

#

Oh then what library would work most like SQL

ornate abyss Jun 2, 2018, 5:49 PM

#

That doesn't make sense sorry

#

SQL is a language in its own right

orchid charm Jun 2, 2018, 5:50 PM

#

¯_(ツ)_/¯

ornate abyss Jun 2, 2018, 5:50 PM

#

You use SQL to interact with postgres or well any relational database

dull scarab Jun 2, 2018, 5:50 PM

#

Do you know what sql stands for weeb

orchid charm Jun 2, 2018, 5:50 PM

#

sqlite

ornate abyss Jun 2, 2018, 5:51 PM

#

SQL is structured query language

#

Sqlite is just a single file database

#

But yea, the library you choose is entirely up to you

#

Just make sure it's async really

orchid charm Jun 2, 2018, 5:52 PM

#

How to use asyncpg correctly then cause i heard it is good

ornate abyss Jun 2, 2018, 5:52 PM

#

Surely it has documentation?

dull scarab Jun 2, 2018, 5:53 PM

#

I sent an example on how to implement asyncpg, connect and query a few messages up

#

In relation to d.py

ornate abyss Jun 2, 2018, 5:53 PM

#

Not all heros wear capes

dull scarab Jun 2, 2018, 5:54 PM

#

feelskawaiiman

orchid charm Jun 2, 2018, 5:54 PM

#

Ive already tried using that example to try and use Postgresql

ornate abyss Jun 2, 2018, 5:56 PM

#

Just gonna have to keep reading the docs and trying

#

Dunno what else to suggest

orchid charm Jun 2, 2018, 5:58 PM

#

Oh yeah, so how does increasing the timeout affect a d.py rewrite bot?

#

sqlite3

#

Does it just make it a bit slower?

ornate abyss Jun 2, 2018, 6:07 PM

#

No, in the event of a timeout it will block your entire bot

#

Basically means for that timeout period your bot won't respond to anything

torn sphinx Jun 2, 2018, 6:24 PM

#

Can anyone help me make this shorter? The bot replies slow because of so many if statements I think.
https://hastebin.com/xicegidiwo.py

novel wharf Jun 2, 2018, 6:34 PM

#

Even though it's unoptimized code, the if-statements cost next to nothing.

#

If you wish to make them faster, you could group them together with elif instead of a new if every time. I doubt you can be two ranks at once, can you?

#

Although I supposed you've done it just in case someone somehow gets enough xp to bypass several ranks, in which case you might only want to display the latest rank anyhow.

#

if lvl == "RANK" and xp >= XP_REQ:
    lv.update({'level': "NEXT_RANK"}, Levels.id == message.author.id)
    lv.update({'rank': "NEXT_RANK_IMAGE"}, Levels.id == message.author.id)
    embed=discord.Embed(title="Tanki Online", url="...", color=0x00ffff)
    embed.set_thumbnail(url='...')
    embed.add_field(name=":sparkles: Rank up! :sparkles:", value="Congratulations **{}**! Your rank is now **NEXT_RANK**\n   ‍   ".format(message.author.display_name))
    await bot.send_message(message.channel, embed=embed)

#

This particular piece of code is repeated about 44 times, but with only a few changes. Try move this into a function and use loops! You'd need to store the data for RANK and XP_REQ, etc, elsewhere.

#

@torn sphinx

torn sphinx Jun 2, 2018, 6:52 PM

#

So I will just use elif instead of if everytime. okay thanks

novel wharf Jun 2, 2018, 6:54 PM

#

What's with Field Marshal that only requires 100 xp for the next rank?

#

As for your ranks, are they all in order, or are there any branches?

torn sphinx Jun 2, 2018, 7:07 PM

#

I fixed the 100xp in marshal rank I forgot to change that from copy pasting. Also Yeah they are in order

#

Btw I want to create a curreny system but now I cant add for example coins because they already have a database. Or I can instert coins to their database?

novel wharf Jun 2, 2018, 7:07 PM

#

There is a really nice way you can decrease all this code to just the code I pasted above.

#

If you can insert ranks, you should be able to insert coins the same way?

torn sphinx Jun 2, 2018, 7:09 PM

#

I did made to instert ranks when they didn have a profile set so the bot would do. If the user is not in the database then

    lv.insert({'id': message.author.id, 'xp': 1, 'level': 'Recruit', 'premium': 'No', 'rank': 'image_link'})

But now they alreadyare in the database. :/

novel wharf Jun 2, 2018, 7:10 PM

#

Right, at the bottom.

#

But they have a rank, don't they? What if you also give them 'coins': 0?

#

Then you update that the same way as you do with ranks~

#

coins = asd["coins"]
coins += 20
lv.update({'coins': coins}, Levels.id == message.author.id)

torn sphinx Jun 2, 2018, 7:16 PM

#

I did add coins at the bottom but it didnt instert them in my profile. it will only add coins to people that talk for the first time I mean when they send their first message so the bot will add them in the database.
Because I made the bot check if the user is in the database if lv.contains(Levels.id == message.author.id):. If its not then instert him. else: lv.insert({'id': message.author.id, 'xp': 1, 'level': 'Recruit', 'coins': 0, 'premium': 'No', 'rank': 'https://i.imgur.com/tlyR0dt.png'})

#

But if someone is already in the database it wont add coins

novel wharf Jun 2, 2018, 7:17 PM

#

Right, that makes sense. You have a lot of users with the old database.

torn sphinx Jun 2, 2018, 7:17 PM

#

yeah

#

about 3.5k users

novel wharf Jun 2, 2018, 7:17 PM

#

Oh boy.

torn sphinx Jun 2, 2018, 7:17 PM

#

📎 unknown.png

novel wharf Jun 2, 2018, 7:18 PM

#

That's why you need to check whether a user has coins, and then you can add it! Consider this:```python
if "coins" not in asd:
lv.update({'coins': 0}, Levels.id == message.author.id)

torn sphinx Jun 2, 2018, 7:18 PM

#

isn't is insert?

#

not update

novel wharf Jun 2, 2018, 7:18 PM

#

Oh, sure.

torn sphinx Jun 2, 2018, 7:18 PM

#

or it doesnt matter?

#

ok

novel wharf Jun 2, 2018, 7:19 PM

#

Alternatively, if you have the database as a file someplace, you could write a short python script to update it and insert "coins":0 for everyone.

torn sphinx Jun 2, 2018, 7:20 PM

#

Well I am not that pro with python so I would just use the first method you told me Xd

#

Thanks though

novel wharf Jun 2, 2018, 7:21 PM

#

Cmon, should be easy peasy~ You're in a python server.

#

As for the ranks. https://pastebin.com/raw/HPjSxQH7

torn sphinx Jun 2, 2018, 7:22 PM

#

wow you did all this for me?

novel wharf Jun 2, 2018, 7:23 PM

#

If you store them in a list like this in python, you can do the rank check with a simple for-loop and test if a user has reached past a new rank.

#

I'm using macros in my editor, so it takes only a minute or two~

torn sphinx Jun 2, 2018, 7:24 PM

#

damn lol It took me 2 days to do what you say in the hastebin XD Actualy 1 day for writing. cuz I spent 1 day trying to figur out how to make a leveling up system

novel wharf Jun 2, 2018, 7:26 PM

#

for rank in reversed(ranks):
    if xp >= rank["xp"] and my_rank != rank:
        # New rank!
        return

#

And here's a quick example of how you could use that ranks list. Loop over it backwards, check if the xp is past the required xp, then update!

#

Gonna brb for a minute.

torn sphinx Jun 2, 2018, 7:28 PM

#

Damn dude. You are so helpful. People in disord.py are so mean lol

#

Thanks

#

a lot

dull scarab Jun 2, 2018, 7:28 PM

#

d.py is straight forward, and expect you to know python

#

Here it's more about helping you learn most of it

novel wharf Jun 2, 2018, 7:31 PM

#

So what does the database look like? Is it json?

#

Let's make a script to add coins to it.

orchid charm Jun 2, 2018, 7:34 PM

#

async def check_all_members():
    bot.c.execute('SELECT UID from Profile')
    rows = bot.c.fetchall()
    for member in bot.get_all_members():
        if not any(row[0] == member.id for row in rows):

So i got a problem with this, it is making Multiple of the same "Accounts"

torn sphinx Jun 2, 2018, 7:34 PM

#

@novel wharf yes json

#

Let me upload it to sendspace

novel wharf Jun 2, 2018, 7:35 PM

#

As long as it doesn't contain sensitive information.

#

I dont need to see it really. You can try opening it in a new Python script.

orchid charm Jun 2, 2018, 7:36 PM

#

async def check_all_members():
    bot.c.execute('SELECT UID from Profile')
    rows = bot.c.fetchall()
    for member in bot.get_all_members():
        if not any(row[0] == member.id for row in rows):
            bot.c.execute('INSERT INTO Profile (UID, XP, LEVEL, SCROLLS, SCN, Snails, LevelUp, Rank, GCN, XPBooster, SCNBooster, InkBay) VALUES(?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?, ?)',(member.id, 5, 1, 0, 0, 0, 100, 1, 0, 0, 0, 0))

Forgot to put in the insert

#

Anyways, do you know what i am doing incorrectly? it is making multiple of the same row

torn sphinx Jun 2, 2018, 7:36 PM

#

I Dmed it to you

novel wharf Jun 2, 2018, 7:42 PM

#

Try printing row[0] and member.id

#

Could be something silly like string vs int comparison

orchid charm Jun 2, 2018, 7:49 PM

#

async def check_all_members():
    bot.c.execute('SELECT UID from Profile')
    rows = bot.c.fetchall()
    for member in bot.get_all_members():
        if not any(row[0] == member.id for row in rows):
            bot.c.execute('SELECT UID From Profile where UID = ?', (member.id,))
            result = bot.c.fetchone()
            if not result:
                print('Woomy!')

#

TabError: inconsistent use of tabs and spaces in indentation

#

No Matter what i do it gives this error?

torn sphinx Jun 2, 2018, 7:50 PM

#

fix your identation

#

youre mixing up tabs and spaces

#

use one or the other

orchid charm Jun 2, 2018, 7:53 PM

#

Also, How to get x amount rows with the highest value

#

Its like a "Leaderboard"

#

Also i am making a "Owned Items" Command
How would i get all the rows with a Users ID in them and put each row into a variable

#

Sqlite3

dull scarab Jun 2, 2018, 8:01 PM

#

Sort by value, get x first-

#

Query members ID, and just put it in a list/dict or simply stick to the response object

orchid charm Jun 2, 2018, 8:03 PM

#

Oh?

dull scarab Jun 2, 2018, 8:04 PM

#

 fetchall()

    Fetches all (remaining) rows of a query result, returning a list.```

#

So you don't need to convert to a list, but you can still make it a dict if you need to assign values to specific names

orchid charm Jun 2, 2018, 8:07 PM

#

so fetchall
then how to insert parts of it into variables?

#

result[1] result[2] ect?

dull scarab Jun 2, 2018, 8:07 PM

#

Is each item stored as a row connected to a member?

orchid charm Jun 2, 2018, 8:08 PM

#

yeah

dull scarab Jun 2, 2018, 8:08 PM

#

Then yes.

orchid charm Jun 2, 2018, 8:08 PM

#

This is how it looks like
UID (ctx.message.author.id)
Amiibo(AmiiboName)

#

How to check how many rows it grabbed? @dull scarab

dull scarab Jun 2, 2018, 8:18 PM

#

len(results)

orchid charm Jun 2, 2018, 8:18 PM

#

Ah

#

@dull scarab Also how to convert the tuple it is giving to something else? like a Str

#

or do i do something like str(result[1)

vestal apex Jun 2, 2018, 8:38 PM

#

these seem like really trivial questions you could answer yourself in a matter of seconds

dull scarab Jun 2, 2018, 8:39 PM

#

You don't have to test on your actual bot either

#

open the python interpreter

# You know it's a tuple, create a dummy
>>>test_tuple = (1, 2, "a")

# Now simply test...
>>>print(test_tuple[0])
1

# Keep in mind print calls str() on it's arguments

>>>type(test_tuple[0])
<class 'int'>
>>>type(test_tuple[2])
<class 'str'>

>>>test_tuple[1]
2

>>>str(test_tuple[1])
"2"```

rain karma Jun 2, 2018, 9:12 PM

#

Could someone come along side me and help me structure my tables for a project? I understand that what I'm asking for is vague and time consuming, however, I have been trying to sketch out my table structure for a few weeks now and I still can't seem to get things well organized

novel wharf Jun 3, 2018, 11:48 AM

#

Give us a description in here of what you're trying to do.

torn sphinx Jun 4, 2018, 12:22 PM

#

Hey! Im trying to get my crypto miner machine to rapport its hashrate and gpu temps so a simpla database on the internet! Any tips on what librarys i whould look in to ?

vestal apex Jun 4, 2018, 5:02 PM

#

not sure what this has to do with databases...

#

and it probably depends on what software you're using to mine

compact owl Jun 4, 2018, 7:45 PM

#

hello! I am currently learning about databases and websites and stuff with python and I just started creating a simple database to understand how it works.

#

I am having a problem with the insert function as part of the database code

#

I am using "/t" as a delimiter to seperate the key from the value after being read.

#

and I was trying to find ways to encode the backslash as a different sequence of characters to prevent it from corrupting my table

#

def insert(filename, key, value):
    file = open(filename, "r")
    ogn = file.read()
    key = key.replace('''\\''', '''\\\\''')
    value = value.replace('''\\''', '''\\\\''')
    if key in ogn:
        print("This key has already been used.")
    elif value in ogn:
        print("This value has already been used.")
    else:
        print(key)
        print(value)
        file = open(filename, "a")
        file.write(key + '/t' + value + ' /n')
    file.close()

insert("data.txt", "prsdf6@outlook.com", "Rashim Kesa\ni")

#

I tried doing it such a way that the backslash would be replaced by the escape character for backslash

#

or do I need to change it to somehting else

dull scarab Jun 4, 2018, 8:17 PM

#

I wouldn't call appending to a txt file a database.

#

Also, you're opening 2 instances of the file, and only closing one from what I can tell

#

Are you getting any errors @compact owl

compact owl Jun 4, 2018, 8:44 PM

#

nope

#

well a simplified database

#

I guess

#

the text file is the table

dull scarab Jun 4, 2018, 8:45 PM

#

Mkai, and what's the issue?

compact owl Jun 4, 2018, 8:46 PM

#

if you look at where I called the function

#

I purposely put a backslash next to the "n" in teh name

#

I am also using /t as my delimiter for when I read off the text file

#

so I am trying to figure out ways to turn the backslash into two backslashes before storing

#

and then decoding after reading from the txt file

#

so I am trying to use the replace() string method

#

but for whatever reason, the value remains unchanged

#

I tried assigning the "value" parameter's value to another variable and then applying replace() to that

#

but I am unable to change the single backslash to two backslashes

#

def insert(filename, key, value):
    file = open(filename, "r")
    ogn = file.read()
    file.close()
    key1 = key
    value1 = value
    key1.replace("\\","\\\\")
    value1.replace("\\", "\\\\")
    if key in ogn:
        print("This key has already been used.")
    elif value in ogn:
        print("This value has already been used.")
    else:
        print(key1)
        print(value1)
        file = open(filename, "a")
        file.write(key1 + "\t" + value1 + "\n")
    file.close()

#

any help would be appreciated

dull scarab Jun 4, 2018, 8:51 PM

#

How do you want this to look in your data.txt file?
Kesa\ni

#

like Kesa\\ni?

compact owl Jun 4, 2018, 8:51 PM

#

kesa\ni

#

sorry the latter

#

Kesa\\ni

#

in the txt file itself

#

but when I write my read function

#

I need to be able to change it back to Kesa\ni

#

and print that to console

#

def insert(filename, key, value):
    file = open(filename, "r")
    ogn = file.read()
    file.close()
    key1 = key
    value1 = value
    key1.replace("\\","\\\\")
    value1.replace("\\", "\\\\")
    if key in ogn:
        print("This key has already been used.")
    elif value in ogn:
        print("This value has already been used.")
    else:
        print(key1)
        print(value1)
        file = open(filename, "a")
        file.write(key1 + "\t" + value1 + "\n")
    file.close()

insert("data.txt", "prsdf6@outlook.com", "Rashim Kesa\ni")

#

here is the full code

dull scarab Jun 4, 2018, 8:56 PM

#

Well, your text viewer will view \n as a new line, but you could just convert that back to \n with py def read(filename): with open(filename, "r") as data: data_string = data.read().replace("\n", "\\n")

compact owl Jun 4, 2018, 8:57 PM

#

okay

dull scarab Jun 4, 2018, 8:57 PM

#

My data.txt

📎 unknown.png

#

prints
prsdf6@outlook.com/tRashim Kesa\ni /n

compact owl Jun 4, 2018, 8:57 PM

#

hmm

#

so its fine?

dull scarab Jun 4, 2018, 8:58 PM

#

the \n is still in the txt, sort of

#

But it's viewed as a new line

compact owl Jun 4, 2018, 8:58 PM

#

so I can just leave it the way it is?

#

and read it and get the result I want?

dull scarab Jun 4, 2018, 8:59 PM

#

You could also pass write a raw string

compact owl Jun 4, 2018, 8:59 PM

#

yeah that's another thing

dull scarab Jun 4, 2018, 8:59 PM

#

r"somestring\n that wont be formated"

compact owl Jun 4, 2018, 8:59 PM

#

it keeps giving me weird stuff when I type r in front of my strings

dull scarab Jun 4, 2018, 9:00 PM

#

def insert(filename, key, value):
    file = open(filename, "r")
    ogn = file.read()
    if key in ogn:
        print("This key has already been used.")
    elif value in ogn:
        print("This value has already been used.")
    else:
        print(key)
        print(value)
        file.close()
        file = open(filename, "a")
        file.write(key + '/t' + value + ' /n')
    file.close()

insert("data.txt", r"prsdf6@outlook.com", r"Rashim Kesa\ni")```I changed your code slightly, and it writes `prsdf6@outlook.com/tRashim Kesa\ni /n`  to my data.txt now

compact owl Jun 4, 2018, 9:00 PM

#

okay here's another question

dull scarab Jun 4, 2018, 9:00 PM

#

def insert(filename, key, value):
    file = open(filename, "r")
    ogn = file.read()
    file.close()
-    key1 = key
-    value1 = value
-    key1.replace("\\","\\\\")
-    value1.replace("\\", "\\\\")
    if key in ogn:
        print("This key has already been used.")
    elif value in ogn:
        print("This value has already been used.")
    else:
        print(key1)
        print(value1)
        file = open(filename, "a")
        file.write(key1 + "\t" + value1 + "\n")
    file.close()

-insert("data.txt", "prsdf6@outlook.com", "Rashim Kesa\ni")
+insert("data.txt", r"prsdf6@outlook.com", r"Rashim Kesa\ni")```

compact owl Jun 4, 2018, 9:01 PM

#

hmm

dull scarab Jun 4, 2018, 9:01 PM

#

I used the old code you sent. whopsie

compact owl Jun 4, 2018, 9:01 PM

#

is there a way to change the strings into raw strings within the function?

#

also

#

what if the email has a "\t" in it

#

how would I change that backslash to something else before storing

#

then decode after reading?

#

thanks for the help btw

dull scarab Jun 4, 2018, 9:05 PM

#

    key = f"{key!r}"
    value = f"{value!r}"```

#

That should convert your input to a raw string, I think

compact owl Jun 4, 2018, 9:05 PM

#

wats this?

#

hmm

#

can I leave it as just a string and it would still be fine without the raw string?

dull scarab Jun 4, 2018, 9:06 PM

#

It's more or less the same as ```py
key = "{!r}".format(key)

or old style

key = "%r"%key```

#

as far as i know

compact owl Jun 4, 2018, 9:06 PM

#

cool

dull scarab Jun 4, 2018, 9:07 PM

#

Are you even allowed to use \ in emails

compact owl Jun 4, 2018, 9:07 PM

#

yea

#

you are

#

but just as a scenario

#

how would I edit my code

#

to encode the backslash

#

which is what I was trying to do earlier

dull scarab Jun 4, 2018, 9:08 PM

#

I'll be honest, I think this would be a lot easier if you converted it to json or some other data storage method

compact owl Jun 4, 2018, 9:08 PM

#

lmao it would

dull scarab Jun 4, 2018, 9:08 PM

#

That way you could just fetch the keys, and do what you wanted with those values

compact owl Jun 4, 2018, 9:08 PM

#

I guess

#

yeah

#

but I want to know how now

dull scarab Jun 4, 2018, 9:09 PM

#

Hm

compact owl Jun 4, 2018, 9:09 PM

#

and then I will be off your back

#

cause for whatever reason

#

when I try to change the parameters within the function

dull scarab Jun 4, 2018, 9:09 PM

#

Well, you are also using /t, not \t

compact owl Jun 4, 2018, 9:09 PM

#

sorry

#

i meant \t

dull scarab Jun 4, 2018, 9:10 PM

#

well currently your data looks like this 'prsdf6@outlook.com'\t'Rashim Kesa\ni' \n

#

Right?

compact owl Jun 4, 2018, 9:10 PM

#

yeah

dull scarab Jun 4, 2018, 9:11 PM

#

So you could just use the ' to seperate key and value

compact owl Jun 4, 2018, 9:11 PM

#

what if the email was like bob\t@gmail.com or something

#

I know, but I am curious about how it would work if I would use \t as delimiter. sorry if I am being stubborn

#

If i were to use any other character

#

this problem wouldn

#

t

dull scarab Jun 4, 2018, 9:11 PM

#

Honestly, I don't know

compact owl Jun 4, 2018, 9:12 PM

#

hmm

#

yeah its weird

dull scarab Jun 4, 2018, 9:12 PM

#

Most of the answers I've given so far I've personally googled and learned myself now

compact owl Jun 4, 2018, 9:12 PM

#

lmao

#

I have tried to research string literals

#

and raw strings

#

but they don't work really

#

when I am trying to change backslash to escape characer for backslash

dull scarab Jun 4, 2018, 9:12 PM

#

But feel free to experiment and I'm sure people would be happy to help you solve your erros etc

compact owl Jun 4, 2018, 9:13 PM

#

yup

#

thank you for the help

#

I appreciate it

dull scarab Jun 4, 2018, 9:13 PM

#

Also, you'd probably get more and faster help in #help-coconut - 4

compact owl Jun 4, 2018, 9:13 PM

#

yea

#

that's what I was going to do next

#

XD

#

Again, thanks

dull scarab Jun 4, 2018, 9:13 PM

#

I'm off to bed though, Good luck

compact owl Jun 4, 2018, 9:13 PM

#

gn

torn sphinx Jun 5, 2018, 5:51 PM

#

Anyone got any recommendations for videos to watch on Databases with Python3 so I can get up to speed?

#

Used databses but only in Microsoft Access, never with Python or SQL

ionic pecan Jun 5, 2018, 5:52 PM

#

Videos? No

#

The SQLAlchemy tutorial is good

sweet nebula Jun 5, 2018, 8:47 PM

#

is there a way to incorporate a database into an app using python. The database should be app-side and shouldn't require internet access to be able to acquire data from the database. Instead the database should be able to save data given by the user onto the device directly???? (IDRK) and will be able to access that data at a later date. IF this is at all possible that would be great.

ornate abyss Jun 5, 2018, 9:45 PM

#

SQLite

#

@sweet nebula

sweet nebula Jun 5, 2018, 9:46 PM

#

should i just research into SQlite and i should find whag i need or is there any specifics i should focus on?

ornate abyss Jun 5, 2018, 9:47 PM

#

Well SQLite is a single file database, it still uses SQL to form its queries, be that saving/gathering data. Python has a builtin module for dealing with SQLite https://docs.python.org/3/library/sqlite3.html

midnight verge Jun 6, 2018, 8:11 AM

#

When working with SQLAlchemy, is there any way to apply a default filter to a relationship? For instance I have:

class Foo(Base):
    __tablename__ = 'foos'

    id = Column(Integer, primary_key=True)
    bars = relationship('Bar', back_populates='foo', lazy='dynamic')

class Bar(Base):
    __tablename__ = 'bars'

    id = Column(Integer, primary_key = True)
    foo_id = Column(Integer, ForeignKey(Foo.id)
    foo = relationship('Foo', back_populates='bars')
    deleted = Column(Boolean, default=False)

What I would like to be able to do, with assert isinstance(foo, Foo), is automatically filtering foo.bars so that deleted bars (i.e. with deleted is True) are not returned by default.
I could not manage to find a nice way to do this without using a proxy function, any idea? 🤔

unreal tartan Jun 6, 2018, 8:14 AM

#

@midnight verge See the documentation at https://bitbucket.org/zzzeek/sqlalchemy/wiki/UsageRecipes/PreFilteredQuery

midnight verge Jun 6, 2018, 8:36 AM

#

@unreal tartan looks perfect, thanks 👍

glass bough Jun 8, 2018, 10:40 PM

#

so, when I want to select all from a table where column_name = string

#

correct?

dull scarab Jun 8, 2018, 10:40 PM

#

wdym?

glass bough Jun 8, 2018, 10:42 PM

#

nvm, I'll work on it

#

hard to explain

#

I want to select all from a table where in a column it is equal to a string, which I assume would be written as

#

"SELECT * FROM tableWHERE column = {}".format(string)

dull scarab Jun 8, 2018, 10:44 PM

#

Doesn't your lib have ? variables for queries?

#

If so i strongly recommend using it

glass bough Jun 8, 2018, 10:47 PM

#

got it

#

was missing a comma

ornate abyss Jun 9, 2018, 11:59 AM

#

You dont want to use string interpolation to format a query

sweet nebula Jun 9, 2018, 9:24 PM

#

is it possible to store images in a sqlite databse?

worn stream Jun 9, 2018, 9:31 PM

#

I don't think so

#

possibly if Base64 encode it

sweet nebula Jun 9, 2018, 9:34 PM

#

so if i base64 encode it before entering it to the datbase i can save it as TEXT, then when i need the data i can call it and then decode it back to an image for display?

hidden ginkgo Jun 9, 2018, 9:45 PM

#

Should be possible.

vestal apex Jun 9, 2018, 9:46 PM

#

you can, you probably shouldn't

sweet nebula Jun 9, 2018, 9:47 PM

#

XD

#

but idk what else to do

vestal apex Jun 9, 2018, 9:54 PM

#

can't save to the filesystem?

orchid charm Jun 9, 2018, 11:56 PM

#

How to Insert multiple selects into multiple variables with one query

vestal apex Jun 10, 2018, 12:00 AM

#

could you clarify what you mean by "multiple selects"?

orchid charm Jun 10, 2018, 2:53 AM

#

Multiple "Queries"

vestal apex Jun 10, 2018, 3:43 AM

#

so you want to perform multiple queries... in one query?

vestal apex Jun 10, 2018, 3:59 AM

#

it's possible you want the IN clause of SELECT queries, but without additional information I have no idea if that's what you need or not

dull scarab Jun 10, 2018, 2:14 PM

#

Saw the suggestion about RethinkDB in #discord-bots , is there any big benefits to migrating over using Postgresql?

ionic pecan Jun 10, 2018, 2:19 PM

#

well idk

#

I use postgreSQL personally and its really great

#

It probably depends on your usecase

#

I remember that rethinkdb advertised thie realtime query result push, but I haven't seen that used anywhere yet

#

I found a comparison of core features here: https://db-engines.com/en/system/PostgreSQL%3BRethinkDB

PostgreSQL vs. RethinkDB Comparison

Detailed side-by-side view of PostgreSQL and RethinkDB

dull scarab Jun 10, 2018, 2:36 PM

#

Ah, thank you

ornate abyss Jun 10, 2018, 4:50 PM

#

You really want to have a long think if you are using a NoSQL database

ionic pecan Jun 10, 2018, 4:55 PM

#

wdym?

compact owl Jun 10, 2018, 8:41 PM

#

Yay I am using SQL now

ornate abyss Jun 10, 2018, 10:03 PM

#

Saw the suggestion about RethinkDB RethinkDB is a NoSQL DB

#

NoSQL DB's (as im sure youve seen striking say a lot) are used a lot when a regular RDBMS could be used

compact owl Jun 11, 2018, 4:24 AM

#

In a SQLite database, how would I check if a table exists? I need to make a table only once.

vestal apex Jun 11, 2018, 4:30 AM

#

you could use a CREATE TABLE IF NOT EXISTS query

compact owl Jun 11, 2018, 4:53 AM

#

okay

#

thanks

subtle warren Jun 11, 2018, 1:00 PM

#

https://hastebin.com/uruhawijif.py Can someone tell me how I can do something like this?

#

essentially have a relationship that comes from two columns in a table

#

(player1 and player2)

#

also I don't like the duplication. If I want to add more information about a player, I need to copy/paste it and add 2 to it

#

is there a better way to do it?

#

I am 100% certain it will always only stay two players (it's a fighting game), so generalizing it with a player-game-association table to have an arbitrary amount seems like overkill

#

and I would have to have code in some places that checks if there was ever more than 2 players in a game etc. and I don't even want it to get to that

#

this is sqlalchemy btw, but I thought people that know it, would recognize

subtle warren Jun 11, 2018, 1:38 PM

#

if someone can help me later, please @ me!

unreal tartan Jun 11, 2018, 1:57 PM

#

If a User needs game-specific data you only have 2 options: the duplicate options where you store it all in the game, or the user-game association table like you suggested yourself.

Both are valid options, yes you have duplicate column names, it's a denormalized model in the first solultion. But denormalizing data gives back the benefit of being quicker if you need all the data, because there's no joins.
Normalizing has the benefit of having no duplication, cleaner, but a slight performance hit because you need joins to retreive all data.

I'd probably start off with the 'clean' model and see if you have any true issues with speed. If not, I see no need to denormalize.

Game:
- game data
- players 

Player:
- Game
- User
- extra data for player for a specific game

User:
- name

@subtle warren

subtle warren Jun 11, 2018, 2:02 PM

#

if I was to go with the denormalized model, do you know how I could make that relationship work?

#

(I will probably take your advice and go for the association table GameUser or something)

#

also is there maybe a nice way to make the database or sqlalchemy ensure constraints like "only two users per game"?

unreal tartan Jun 11, 2018, 2:04 PM

#

Well it's basically what you have in the code you posted but you would literally have like

class Game(Base):
    __tablename__ = "games"

    id = Column(Integer, primary_key=True)
    player1Id = Column(Integer, ForeignKey("users.userId"))
    player1 = relationship("User", backref="games")
    player1_hp = Column(Integer)    
    player2Id = Column(Integer, ForeignKey("users.userId"))
    player2 = relationship("User", backref="games")
    player2_hp = Column(Integer)

So you just duplicate the hp column for each player. It's inefficient for data storage, but efficient for querying.

subtle warren Jun 11, 2018, 2:06 PM

#

I need to query all games of a user every time I want to recalculate the rank, so I think querying is more important than storing

#

but I will still go with it, because it's less ugly to potentially copy&paste 8 columns or something

#

thank you so much for your help @unreal tartan !

unreal tartan Jun 11, 2018, 2:08 PM

#

Can't quickly think of a way to easily put the 2-player constraint on the DB

subtle warren Jun 11, 2018, 2:10 PM

#

I guess I will just check it whenever get the players of a game

unreal tartan Jun 11, 2018, 2:12 PM

#

You could still do player1 and player2 in the game.

#

but the relevant data would be in the player table.

#

Game:
- game data
- player1
- player2

Player:
- Game
- User
- hp
- extra data for player for a specific game

User:
- name

subtle warren Jun 11, 2018, 2:16 PM

#

oh jeez, that sounds like a good solution too

#

in what way is denormalization in this case bad?

#

I mean: what kind of stuff is problematic with this structure?

#

or can go wrong more easily

#

*this specific structure

unreal tartan Jun 11, 2018, 2:21 PM

#

Well let's say you denormalize and put player back into game basically

Game:
- game data
- User1
- user1_hp
- User2
- user2_hp

User:
- name

There's nothng terribly bad, but you just have duplciate column names.
It's acceptable with 2 players, but you can imagine the repetitiveness with more.

The main problem comes that now if you like to add an extra field in, let's say, a weeks time. You have to do add it X times depending on players (2 in this case)
Same for deleting. A delete of a single atttribute, means deleting more than 1 field in the database.

#

It's not a very bad case really. It'd get worse if you merged the User's table as well.
After 2 games you might have this:

Game
id | player1_hp | player2_hp | player1_name | player2_name
0  | 0          | 60         | xxx          | yyy
0  | 56         | 0          | xxx          | yyy

#

Clearly you see there's duplicate data in the table: the names.

subtle warren Jun 11, 2018, 2:24 PM

#

yeah, absolutely, but I was more talking about the last suggestion of yours

unreal tartan Jun 11, 2018, 2:25 PM

#

Yea there it's not all that bad except for future database changes that you have to do twice.

subtle warren Jun 11, 2018, 2:25 PM

#

where all the data is in an extra table and I juse have two references to rows of that table in a game

#

I think I will take that gamble

unreal tartan Jun 11, 2018, 2:25 PM

#

Oh in my last example there's even less things bad.

#

I think it might even be better to have player1_id and player2_id.

subtle warren Jun 11, 2018, 2:26 PM

#

why do you think that?

unreal tartan Jun 11, 2018, 2:27 PM

#

Because then it satisfies the maximum of 2 players requirement.

subtle warren Jun 11, 2018, 2:27 PM

#

yeah, that's why I like it

#

okidoki!

#

thank you so much for your help again! also for writing it up with so much detail!

subtle warren Jun 11, 2018, 3:20 PM

#

I think in all the examples with an extra table i need to save the user and the player, so I can easily get User.games

#

but maybe that's not worth the duplication

torn sphinx Jun 12, 2018, 11:11 AM

#

does anyone know if sqlalchemy has unique random indentifiers for the databases it connects to?

#

or maybe not random

#

basically I want to see if my two applications connect to the same database

ionic pecan Jun 12, 2018, 11:17 AM

#

welll, look at your auth credentials?

torn sphinx Jun 12, 2018, 11:18 AM

#

I don't have any. It's a sqlite database. And my two applications are two docker containers

#

main flask app container and celery container

#

i suspect they each create their own db.sqlite file

unreal tartan Jun 12, 2018, 11:21 AM

#

If you don't specify any connection parameters, that sounds like a fair assumption

#

Use docker run -it <image name> /bin/sh to go into both docker images and see if you can find the sqlite files.

torn sphinx Jun 12, 2018, 11:26 AM

#

I'm changing over to devops channel

orchid charm Jun 13, 2018, 3:44 AM

#

        self.bot.c.execute('Select Censor from CensorList where GID = ?', (message.guild.id,))
        result = self.bot.c.fetchall()
        print(result)
        print(len(result))
        print(message.guild.id)

[]
0
432711178117054475

📎 Screen_Shot_2018-06-12_at_8.44.40_PM.png

#

It is not grabbing the info, why?

#

    async def on_message(self, message):
        self.bot.c.execute('Select Censor from CensorList where GID = ?', (message.guild.id,))
        result = self.bot.c.fetchall()
        print(result)
        print(len(result))
        print(message.guild.id)
        censorint = 0
        censormessage = message.content
        for item in result:
            if result[censorint][0] in censormessage:
                censormessage = censormessage.replace(result[censorint][0], '*' * len(result[censorint][0]))
        if censorint > 0:
            await message.channel.send(f'{censormessage}')

#

This is the full event

#

It is in a Discord.py rewrite cog

#

But this is mostly a Database question so i put it here

orchid charm Jun 13, 2018, 5:37 AM

#

NVM. Fixed it.

rancid bronze Jun 13, 2018, 6:56 PM

#

hey yall i'm moving to production pretty soon -- i'm putting my db on my vps where the webapp is hosted -- how should I backup the db?

shadow sphinx Jun 13, 2018, 8:16 PM

#

ok so i have a tkinter program
and i want to store data from it into a sqlite database
but everytime i close the app
the tables get cleared
how do i prevent this
(i have the database made into a class)

#

📎 imbd.py

#

heres the code

low berry Jun 13, 2018, 9:16 PM

#

When using parameterized SQL with python does it automatically put the single quotes around the argument?
ex:
SELECT * FROM ss_orders where ao_MergedIds like %s
turns into:
SELECT * FROM ss_orders where ao_MergedIds like '1234'
?

ionic pecan Jun 13, 2018, 10:11 PM

#

pretty sure that's the case, although if you're using something like SQLAlchemy you can print the query it sends

#

doesnt parametrization use ? though

fringe gale Jun 14, 2018, 6:45 AM

#

code

def add(num1, num2):
    return num1 + num2    

def main():
    operation = input("what calculation do you want to make (+,-,*,/):")
    if(operation != "+" and operation != "-" and operation != "*" and operation != "/"):
            #if invalid operation
            print("you must enter a valid operation")
        else:
            var1 = int(input("Enter num1: "))
            var2 = int(input("Enter num2: "))
            #adds var1 and var2
            if (operation == "+"):
                print(add(var1, var2))

            
main()

error:

C:\Users\Vadim\Desktop>python test.py
  File "test.py", line 10
    else:
        ^
IndentationError: unindent does not match any outer indentation level```

#

what am i missing?

#

wrong chat whoops

#

just noticed

unreal tartan Jun 14, 2018, 6:56 AM

#

(continued in #help-kiwi )

shadow sphinx Jun 14, 2018, 8:09 AM

#

@fringe gale ur else is indented to far in

#

remove and

fringe gale Jun 14, 2018, 8:09 AM

#

@shadow sphinx already fixed it, thanks though 😃

shadow sphinx Jun 14, 2018, 8:09 AM

#

k

bright pelican Jun 14, 2018, 8:09 AM

#

📎 unknown.png

shadow sphinx Jun 14, 2018, 8:54 AM

#

ok so i have a tkinter program
and i want to store data from it into a sqlite database
but everytime i close the app
the tables get cleared
how do i prevent this
(i have the database made into a class)

#

📎 imbd.py

#

heres the code

unreal tartan Jun 14, 2018, 8:58 AM

#

you don't actually execute the commit methods

#

You just retrieve them from the object

shadow sphinx Jun 14, 2018, 10:42 AM

#

That means?

unreal tartan Jun 14, 2018, 10:43 AM

#

you should add ()

cinder pelican Jun 14, 2018, 2:59 PM

#

using flask-sqlalchemy, and anytime I commit a second user it gives me an IntegrityError

UNIQUE constraint failed: user.email [SQL: 'INSERT INTO user (username, email, first_name, last_name, password) VALUES (?, ?, ?, ?, ?)']

#

I checked the database and it does add the record

#

should I try to try/except this and do a rollback? what is even happening?

ionic pecan Jun 14, 2018, 3:11 PM

#

are you sure it adds the record?

#

if the constraint fails, it probably shouldn't ..

#

can you show your model definition?

cinder pelican Jun 14, 2018, 3:14 PM

#

class User(db.Model):
    """
    User model for database
    """
    id = db.Column(db.Integer, primary_key=True)  # Unique identifier
    username = db.Column(db.String(16), unique=True, nullable=False)
    email = db.Column(db.String(120), unique=True, nullable=False)
    first_name = db.Column(db.String(256), nullable=False)
    last_name = db.Column(db.String(256), nullable=False)
    password = db.Column(db.String(60), nullable=False)

    def __repr__(self):
        return (f'User({self.username}', f'{self.email}', f'{self.first_name}', f'{self.last_name}', f'{self.password})')

#

def on_register(form_data):
    """
    Fired when user passes validation.
    """
    new_user = User(
        username=form_data['username'],
        email=form_data['email'],
        first_name=form_data['first_name'],
        last_name=form_data['last_name'],
        password=form_data['password']
    )
    db.session.add(new_user)
    db.session.commit()

#

this is what I do when someone registers

shadow sphinx Jun 14, 2018, 4:29 PM

#

how to i get rid of the brackets when i pull data out of the table

#

using sqlite

#

        c.execute("SELECT * FROM Favs")
        x=c.fetchall()
        self.favlist=[]
        for i in x:
            self.favlist.append(i)
            self.favlist.append("##")

patent glen Jun 14, 2018, 4:33 PM

#

what do you mean get rid of the brackets

shadow sphinx Jun 14, 2018, 4:48 PM

#

📎 unknown.png

#

@patent glen

#

the brackets and shit around the entrys

patent glen Jun 14, 2018, 4:49 PM

#

what exactly are you doing to display that

#

it looks like your "i" is actually a data row with three columns, not a string

#

but also you're creating a list

#

what do you want it to look like

shadow sphinx Jun 14, 2018, 4:50 PM

#

a straight line

#

with all of the rows in it

patent glen Jun 14, 2018, 4:51 PM

#

ok but what do you want between the title, the date, and the description

shadow sphinx Jun 14, 2018, 4:51 PM

#

space

patent glen Jun 14, 2018, 4:51 PM

#

ok

#

how are you putting the favlist in the window

shadow sphinx Jun 14, 2018, 4:51 PM

#

def addintodatabase(name,rdate,plot):
    try:
        db.execute("INSERT INTO Favs (Name,release_date,Plot) VALUES(?,?,?);",(name,rdate,plot))
        db.commit()
    except Exception as e:
        removefromdatabase(name,rdate)

patent glen Jun 14, 2018, 4:52 PM

#

i would suggest doing self.favlist.append(' '.join(i)), and then when you display it using '##'.join(self.favlist) instead of whatever you're doing

#

i said how are you putting it in the window, not how are you putting it in the database

shadow sphinx Jun 14, 2018, 4:52 PM

#

oh

#

    def showfavs(self):
        self.favwin=Toplevel(self)
        c.execute("SELECT * FROM Favs")
        x=c.fetchall()
        self.favlist=[]
        for i in x:
            self.favlist.append(i)
            self.favlist.append("##")
        self.favlist=str(self.favlist).replace("##","\n")
        favlabel=Label(self.favwin,text=self.favlist)
        favlabel.pack()

#

using a label

patent glen Jun 14, 2018, 4:53 PM

#

or just '\n'.join (why are you using ##?)

shadow sphinx Jun 14, 2018, 4:53 PM

#

idk i did that once before it just showed \n

patent glen Jun 14, 2018, 4:54 PM

#

so like python for i in x: self.favlist.append(' '.join(i)) self.favlist="\n".join(self.favlist)

shadow sphinx Jun 14, 2018, 4:56 PM

#

ty

cinder pelican Jun 15, 2018, 12:35 AM

#

weird.. database fixed itself 🤔

dull scarab Jun 15, 2018, 7:24 AM

#

Could someone give me a quick rundown on relational keys in postgres

#

Say I have 2 tables, one for people and one for pets and I put a pets id as a relational key in people. Do I still have yo querry pets etc to fetch info about a pet assigned a person

subtle warren Jun 15, 2018, 1:34 PM

#

@unreal tartan I was here a few days before and I hope it's not rude to ping you. I asked about how to create a relationship from two columns, like this: https://hastebin.com/eharuhasuq.py

#

and I did a lot of other stuff since then and just now could really try it out and it errors

#

sqlalchemy.exc.ArgumentError: Error creating backref 'games' on relationship 'Game.clientUser': property of that name exists on mapper 'Mapper|User|users'

#

this seems to me the clientUser relationship can not be created, because hostUser has already created a "games" property

#

so there must be some other way to do this (if there is at all)

#

I guess specifying a custom (primaryjoin?) for a games property in User could be an option.

#

games = relationship("Game", primaryjoin="or_(Game.hostUserId==User.userId, Game.clientUserId==User.userId)") like this?

#

this does not error, but it's gonna take a while to really try it out. this was more a "just so you know" kind of thing

gleaming frost Jun 15, 2018, 5:15 PM

#

Dont store passwords 😠

#

Without hash and salt

gusty spindle Jun 15, 2018, 5:55 PM

#

^ this.

dull scarab Jun 15, 2018, 8:19 PM

#

Does anyone know what i'd need to do to connect 2 tables?

#

So i dont have to query both, and find matching results

viral crag Jun 15, 2018, 8:29 PM

#

That's called a join query

#

Left join, right join, inner join

#

Kind of an advanced topic but useful to know

dull scarab Jun 15, 2018, 8:32 PM

#

I've used inner join a bit during my database and datahandling course

#

And know kinda... how it works

#

Basically creates a new table with other tables right?

dull scarab Jun 16, 2018, 4:55 PM

#

Can you store data is postgresql in the form of a dict?

#

I'm setting up configs to be saved in my db, with lot's of different types for each key.. like lists, strings etc

#

Or would i be better off just making a json file for it

quick marlin Jun 16, 2018, 4:56 PM

#

if your config is read-only (as it should be) then why would you want a db for it

dull scarab Jun 16, 2018, 4:57 PM

#

I guess config is the wrong term then

quick marlin Jun 16, 2018, 4:57 PM

#

a json file should do, though for configs something with comments, like yaml, may be better

dull scarab Jun 16, 2018, 4:57 PM

#

But generally stuff my bot needs to keep track of like blacklisted text channels etc

#

Blacklisted channels in general, and that would just be (text, anyarray) for storage, but I thought i'd create it.. a bit more flexible for future things

#

Actually... is a db query faster than checking for element in list?.. could just make a blacklisted table and call it a day GWcmeisterPeepoShrug

knotty snow Jun 16, 2018, 6:52 PM

#

element in list time complexity is O(n)

#

if you used a set or dictionary, it's O(1), though. blacklisted table is a good solution.

dull scarab Jun 16, 2018, 6:58 PM

#

How efficient are databases in general?

#

Would it ever be considered bad to create a table for just a bit of data?

knotty snow Jun 16, 2018, 7:00 PM

#

databases are efficient for large amounts of data, but they also can add overhead to your program. it's probably not worth the time or effort to seriously add a database for a small amount of data

#

and it may actually end up slowing down your program ( inter process communication is slower than having something in local memory)

#

but if it's for learning about databases, go for it

dull scarab Jun 16, 2018, 7:01 PM

#

I'd like to know more about it, like down to the bare bones. If you got any reads that would be great

knotty snow Jun 16, 2018, 7:03 PM

#

do you want to learn how databases work internally orfrom a outside perspective

dull scarab Jun 16, 2018, 7:03 PM

#

Stuff like, what happens with open connects, with queries, executions, opening/quering/closing etcetc

#

Just get a good feel for databases behind the scenes, and from an outside perspective

#

Probably not the deepest of deep layers within database builds Monksass

knotty snow Jun 16, 2018, 7:04 PM

#

maybe something you can try before you read about is to try and implement a database yourself

dull scarab Jun 16, 2018, 7:04 PM

#

I have multiple with my discord bots, and studied datastructure (which had basic sql) at uni a year ago

knotty snow Jun 16, 2018, 7:05 PM

#

never watched this, but it's by the creator of SQLite, so i imagine it's in depth

#

https://www.youtube.com/watch?v=Z_cX3bzkExE

YouTube

tcation

How SQL Database Engines Work

Dr. Richard Hipp, creator of SQLite, presents "How SQL Database Engines Work" at OpenSQLCamp 2008. The description: To many programmers, SQL RDBMSes are a ma...

▶ Play video

dull scarab Jun 16, 2018, 7:06 PM

#

Neat

#

I'll make sure to give it a watch

torn sphinx Jun 16, 2018, 7:13 PM

#

guys i'm sorry to ask this here but I have an Excel related question

#

may I ask here?

knotty snow Jun 16, 2018, 7:34 PM

#

it's vaguely a database

gusty spindle Jun 16, 2018, 7:41 PM

#

@torn sphinx if it's about interacting with excel spreadsheets using python then yes, you could ask here. if it's just about excel in general, it would be better to go to #ot0-fear-of-python, especially as it's already almost not about databases anyways :P

grim reef Jun 16, 2018, 9:28 PM

#

Would it be a bad idea to use google spreadsheet to store large amounts of data?
If so, does anyone have a suggestion for a database that can be stored on a cloud and accessed by two or more pc's at once?

#

just @ me when anyone responds thanks

ionic pecan Jun 16, 2018, 9:59 PM

#

Postgres would work

#

Depends on your usecase though

knotty snow Jun 16, 2018, 10:04 PM

#

https://www.quora.com/Is-there-any-free-SQL-Server-hosting-available @grim reef

Is there any free SQL Server hosting available? - Quora

Yes. GearHost.com provides free hosting for both MS SQL and MySQL. The limit on free database MSSQL is 10 MB. You can also host your ASP.NET applications on their platform for free. Best for testing and small scale applications. I am using GearH...

ionic pecan Jun 16, 2018, 10:11 PM

#

that's not what they asked

grim reef Jun 17, 2018, 1:13 AM

#

Postgres is a local database is it not? what happens if the hdd of the pc its on goes?

#

Is it possible to Postgres's data on something like OneDrive?

vapid cairn Jun 17, 2018, 2:41 AM

#

i am working with postgres in my bot and had a questions about how to handle the initial creation of tables and then altering the table at a later date to add columns that you didn't plan for. Right now, I only have one table and it shouldn't need to change, but I wanted to create another table and over time i think there will be columns i will want to add/remove. My one table i currently have I created with something like "CREATE TABLE IF NOT EXISTS table_name ( row_id SERIAL PRIMARY KEY, etc.....).

Would a better method to create tables be to create a blank table (or maybe with just a primary key ) then add the columns with "ALTER table_name ADD COLUMN IF NOT EXISTS".....that way if i need to add another row, i can just another ALTER command? I hope this makes sense...i may just be missing something simple since i really don't know sql that well.

dull scarab Jun 17, 2018, 5:29 AM

#

First rule of databases, plan it out :'P.

#

I aint got your answer though SadHug3

dull scarab Jun 17, 2018, 10:19 AM

#

How would i Increment a value in a row, while decrementing another value? Transfer money from person a to b in one query

#

Just pass 2 queries in one execute?

#

await bot.db.execute("""
        UPDATE users
            SET money = money + $1
        WHERE id = $2 AND guild_id = $4;

        UPDATE users
            SET money = money - $1
        Where id = $3 AND guild_id = $4;""",
        amount, fid, tid, gid)``` Or is there a fancier way of doing it

ionic pecan Jun 17, 2018, 11:42 AM

#

@vapid cairn you really shouldn‘t be doing this from your bot programmatically, use migrations

vapid cairn Jun 17, 2018, 2:16 PM

#

@ionic pecan can you point me to to some docs to understand what you mean by that 😬

ionic pecan Jun 17, 2018, 2:43 PM

#

Like, instead of dynamically changing your tables from your program, you should manage the tables it uses through something else

#

Alembic is really good at this http://alembic.zzzcomputing.com/en/latest/

#

Django also has an amazing migrator built-in

#

Depends on what you're using

orchid charm Jun 18, 2018, 5:53 AM

#

In SQLITE, how to wait for All Processes to be done before doing something?

patent glen Jun 18, 2018, 6:54 PM

#

what do you mean by that

orchid charm Jun 18, 2018, 6:54 PM

#

@patent glen Wait for All other Sqlite things to be finished

#

Like, i got a on_message event in d.py that does SQLite

#

And another event that uses SQLite

#

Then a command that uses SQLite

#

I want it to wait until those "Processes" are complete

patent glen Jun 18, 2018, 6:56 PM

#

this really sounds like more of a discord.py question (but are you even using sqlite asynchronously?)

#

but note that sqlite does support transactions (you may need to use separate connections), if that ends up being relevant to the problem you're trying to solve

orchid charm Jun 18, 2018, 6:58 PM

#

@patent glen How to use Seperate Connections?

#

Also, how to use SQLite asynchronously

#

Is it Aiosqlite? if so i don't know how to use XD

#

Also this is a Database Question, it just happens to be in a D.PY Bot

patent glen Jun 18, 2018, 6:59 PM

#

if you're not doing that and you're not doing threads, then it's unlikely you'll have two things going on at the same time anyway

orchid charm Jun 18, 2018, 6:59 PM

#

Threads?

patent glen Jun 18, 2018, 6:59 PM

#

and by separate connections i meant just calling sqlite3.connect each time you have an operation instead of keeping one open for the lifetime of the program

orchid charm Jun 18, 2018, 6:59 PM

#

Oh

#

So i redefine it every time

patent glen Jun 18, 2018, 6:59 PM

#

ok back up a bit

orchid charm Jun 18, 2018, 6:59 PM

#

Creating a new "connection"

patent glen Jun 18, 2018, 7:00 PM

#

you say you want it to wait until those processes are complete

#

but i'm not clear on why you think it's not already waiting

orchid charm Jun 18, 2018, 7:00 PM

#

Database Locks happen too often

#

Cause of so much SQLite

patent glen Jun 18, 2018, 7:00 PM

#

if you're not using aiosqlite

#

....what errors are you actually seeing

orchid charm Jun 18, 2018, 7:00 PM

#

Database is locked

#

Database is closed

patent glen Jun 18, 2018, 7:00 PM

#

is this database shared between your bot and other processes?

orchid charm Jun 18, 2018, 7:01 PM

#

Yeah

#

1 Database

#

For Multiple "Cogs"

patent glen Jun 18, 2018, 7:01 PM

#

ok hmm

#

ok it also looks like i was a bit overly optimistic about the capabilities of sqlite, hold on

orchid charm Jun 18, 2018, 7:03 PM

#

Once we figure this out i will scrap everything i coded and work on the thing i was making from Square 1

#

Cause #1 Alot of my Code in this bot was never updated cause its a Very Old Bot #2 Terrible Coding #3 Insane Amount of SQLite and database locking

patent glen Jun 18, 2018, 7:04 PM

#

ok i think your issue probably is caused by keeping the connection (well, the transaction) open too long.

#

are you remembering to call commit after each operation?

#

(well, each logical stopping point that it's in a consistent state, anyway)

orchid charm Jun 18, 2018, 7:05 PM

#

Yeah

#

I do Commit after every Operation

#

The Only Operation that really is buggy is one that uses a While Loop Gonna be fixed in the new ver of the bot

patent glen Jun 18, 2018, 7:06 PM

#

ok maybe you do need to be opening and closing the connection each time instead of just holding it open

orchid charm Jun 18, 2018, 7:07 PM

#

Oh?

#

How does that work

#

And what does Closing a connection do exactly

#

And how many connections can be open at once?

patent glen Jun 18, 2018, 7:07 PM

#

ok from what i can tell

orchid charm Jun 18, 2018, 7:08 PM

#

I am a idiot

patent glen Jun 18, 2018, 7:08 PM

#

the database is locked error should only show up when another process has the database open and in a write transaction for longer than five seconds.

orchid charm Jun 18, 2018, 7:08 PM

#

Oh

#

Ive tried using Timeout

dusk sparrow Jun 18, 2018, 7:08 PM

#

5 seconds. HOly smokes

orchid charm Jun 18, 2018, 7:08 PM

#

It just makes stuff slow

dusk sparrow Jun 18, 2018, 7:08 PM

#

🤔

orchid charm Jun 18, 2018, 7:08 PM

#

Discord.py Bot

#

Oof

patent glen Jun 18, 2018, 7:08 PM

#

...that just makes it wait longer than five seconds, it won't help if the other process is keeping it forever

orchid charm Jun 18, 2018, 7:09 PM

#

Still it slows stuff down

dusk sparrow Jun 18, 2018, 7:09 PM

#

why exactly is a db used for bot prgarmming?

orchid charm Jun 18, 2018, 7:09 PM

#

In a D.py bot

dusk sparrow Jun 18, 2018, 7:09 PM

#

or rather what is a db used fr

#

?

orchid charm Jun 18, 2018, 7:09 PM

#

@dusk sparrow Profile system

patent glen Jun 18, 2018, 7:09 PM

#

anything you want the bot to keep track of

orchid charm Jun 18, 2018, 7:09 PM

#

And Holding Info

dusk sparrow Jun 18, 2018, 7:09 PM

#

oh

#

and it's slow because of the query

#

or the size of the db

#

or sqlite?

patent glen Jun 18, 2018, 7:10 PM

#

they're having file locking problems, which probably means that one process is keeping it locked forever because it doesn't commit the transaction and/or close the connection

orchid charm Jun 18, 2018, 7:10 PM

#

I don't exactly know why it gets so slow, maybe cause so many processes at once?

dusk sparrow Jun 18, 2018, 7:10 PM

#

Hmm

patent glen Jun 18, 2018, 7:10 PM

#

how many processes are you accessing this database with

orchid charm Jun 18, 2018, 7:10 PM

#

The on_message event handles the most important bot stuff

dusk sparrow Jun 18, 2018, 7:10 PM

#

so this is not a central db but a db for your app I assume

orchid charm Jun 18, 2018, 7:11 PM

#

Ima show you the code for on_message

patent glen Jun 18, 2018, 7:11 PM

#

uhh

#

do you know what a process is

orchid charm Jun 18, 2018, 7:11 PM

#

I just call Query/Write/Reads "Processes"

patent glen Jun 18, 2018, 7:11 PM

#

that's not what a process is

#

what i meant is how many instances of your bot (or other programs) are using this database

#

at the same time

orchid charm Jun 18, 2018, 7:12 PM

#

https://hastebin.com/agagahajar.swift

#

Also only one Instance of bot is using stuff, multiple .py files are using this database

#

And that is my on_message

patent glen Jun 18, 2018, 7:13 PM

#

ok for one thing you're not committing after the insert

#

(you don't have to do it after every insert, but you should do it after the account create operation as a whole, at the end)

orchid charm Jun 18, 2018, 7:14 PM

#

    while len(shoesset) < 6:
        bot.c.execute('Select ShellaFresh1 from Botinfo')
        ShellaFresh1 = bot.c.fetchone()
        bot.c.execute('Select ShellaFresh2 from Botinfo')
        ShellaFresh2 = bot.c.fetchone()
        bot.c.execute('Select ShellaFresh3 from Botinfo')
        ShellaFresh3 = bot.c.fetchone()
        bot.c.execute('Select ShellaFresh4 from Botinfo')
        ShellaFresh4 = bot.c.fetchone()
        bot.c.execute('Select ShellaFresh5 from Botinfo')
        ShellaFresh5 = bot.c.fetchone()
        bot.c.execute('Select ShellaFresh6 from Botinfo')
        ShellaFresh6 = bot.c.fetchone()
        bot.c.execute('Update BotInfo set ShellaFresh1 = ?',(random.choice(shoes_set),))
        bot.c.execute('Update BotInfo set ShellaFresh2 = ?',(random.choice(shoes_set),))
        bot.c.execute('Update BotInfo set ShellaFresh3 = ?',(random.choice(shoes_set),))
        bot.c.execute('Update BotInfo set ShellaFresh4 = ?',(random.choice(shoes_set),))
        bot.c.execute('Update BotInfo set ShellaFresh5 = ?',(random.choice(shoes_set),))
        bot.c.execute('Update BotInfo set ShellaFresh6 = ?',(random.choice(shoes_set),))
        bot.conn.commit()

This is probably the most Problamatic thing, but i think i got a idea how to fix it

patent glen Jun 18, 2018, 7:14 PM

#

what is bot.c and what is bot.conn? is c the cursor?

orchid charm Jun 18, 2018, 7:15 PM

#

bot = commands.Bot(command_prefix='//')
bot.conn = sqlite3.connect('profiles.db')
bot.c = bot.conn.cursor()

patent glen Jun 18, 2018, 7:15 PM

#

ok

#

where exactly are you getting the database locked error, anyway? it should only happen in connect, which you only do once, right?

orchid charm Jun 18, 2018, 7:15 PM

#

It happens when alot of Sqlite is being used

patent glen Jun 18, 2018, 7:16 PM

#

ok is there anything other than your bot using this database

orchid charm Jun 18, 2018, 7:16 PM

#

Like lets say my 1 Hour Reroll thing happens, i do a command, and on_message all at the same time

patent glen Jun 18, 2018, 7:16 PM

#

i don't understand how you can get the error if there is only one process with only one connection

orchid charm Jun 18, 2018, 7:16 PM

#

It would Almost Guranteed Lock

patent glen Jun 18, 2018, 7:16 PM

#

huh???

#

what function is raising the exception, exactly

#

what is a 1 hour reroll

orchid charm Jun 18, 2018, 7:17 PM

#

It Updates Database info for multiple things

#

And "Rerolls" their info

patent glen Jun 18, 2018, 7:17 PM

#

ok what does 1 hour mean in this context

orchid charm Jun 18, 2018, 7:17 PM

#

If it is a hour

#

So

#

Like 1:00PM

patent glen Jun 18, 2018, 7:18 PM

#

like, after an hour? with no activity before

orchid charm Jun 18, 2018, 7:18 PM

#

2:00PM

patent glen Jun 18, 2018, 7:18 PM

#

ok, i think i get it

orchid charm Jun 18, 2018, 7:18 PM

#

It always does it on the hour mark

patent glen Jun 18, 2018, 7:18 PM

#

ok maybe it's bad to leave a transaction open that long

orchid charm Jun 18, 2018, 7:18 PM

#

if now.minute == 0:

#

                self.bot.c.execute('Update BotInfo set Headspace1 = ?',(random.choice(headgear_set),))
                self.bot.c.execute('Update BotInfo set Headspace2 = ?',(random.choice(headgear_set),))
                self.bot.c.execute('Update BotInfo set Headspace3 = ?',(random.choice(headgear_set),))
                self.bot.c.execute('Update BotInfo set Headspace4 = ?',(random.choice(headgear_set),))
                self.bot.c.execute('Update BotInfo set Headspace5 = ?',(random.choice(headgear_set),))
                self.bot.c.execute('Update BotInfo set Headspace6 = ?',(random.choice(headgear_set),))
                self.bot.c.execute('Update BotInfo set YeOldeClothShoppe1 = ?',(random.choice(shirt_list),))
                self.bot.c.execute('Update BotInfo set YeOldeClothShoppe2 = ?',(random.choice(shirt_list),))
                self.bot.c.execute('Update BotInfo set YeOldeClothShoppe3 = ?',(random.choice(shirt_list),))
                self.bot.c.execute('Update BotInfo set YeOldeClothShoppe4 = ?',(random.choice(shirt_list),))
                self.bot.c.execute('Update BotInfo set YeOldeClothShoppe5 = ?',(random.choice(shirt_list),))
                self.bot.c.execute('Update BotInfo set YeOldeClothShoppe6 = ?',(random.choice(shirt_list),))
                self.bot.c.execute('Update BotInfo set ShellaFresh1 = ?',(random.choice(shoes_set),))
                self.bot.c.execute('Update BotInfo set ShellaFresh2 = ?',(random.choice(shoes_set),))
                self.bot.c.execute('Update BotInfo set ShellaFresh3 = ?',(random.choice(shoes_set),))
                self.bot.c.execute('Update BotInfo set ShellaFresh4 = ?',(random.choice(shoes_set),))
                self.bot.c.execute('Update BotInfo set ShellaFresh5 = ?',(random.choice(shoes_set),))
                self.bot.c.execute('Update BotInfo set ShellaFresh6 = ?',(random.choice(shoes_set),))
                self.bot.conn.commit()

#

This is what it is Updating

patent glen Jun 18, 2018, 7:20 PM

#

i don't even see that function in your paste

#

but there are places in the paste you sent that don't commit

orchid charm Jun 18, 2018, 7:20 PM

#

That is a on_message

#

This is a loop

patent glen Jun 18, 2018, 7:21 PM

#

ok but

#

any of your problems could be caused by any code that ran before that failing to commit

orchid charm Jun 18, 2018, 7:21 PM

#

https://hastebin.com/nufelifepe.swift

#

This is the Loop

patent glen Jun 18, 2018, 7:21 PM

#

so you need to fix all the places you didn't commit

orchid charm Jun 18, 2018, 7:21 PM

#

Also when you were talking about opening and closing connections, how to do that

patent glen Jun 18, 2018, 7:22 PM

#

forget that for now

#

you'd have to rewrite the whole thing

orchid charm Jun 18, 2018, 7:22 PM

#

Yeah i know

#

I am rewriting the whole bot

patent glen Jun 18, 2018, 7:22 PM

#

and i'm not sure it's necessary

#

there are at least two or three places in on_message you need to add commit

orchid charm Jun 18, 2018, 7:24 PM

#

Well my bot already has stupid code

patent glen Jun 18, 2018, 7:25 PM

#

anyway i meant instead of having a global bot.conn and bot.c

orchid charm Jun 18, 2018, 7:25 PM

#

Define it in every cog?

patent glen Jun 18, 2018, 7:25 PM

#

create the connection and the cursor in each function that uses it, and then commit and close the connection at the end

#

cog?

orchid charm Jun 18, 2018, 7:26 PM

#

Cog is a .py file

#

That has a class

patent glen Jun 18, 2018, 7:26 PM

#

no, each function

orchid charm Jun 18, 2018, 7:26 PM

#

Oh

#

Any downsides to that?

patent glen Jun 18, 2018, 7:26 PM

#

so that it only connects when it runs the function, and disconnects at the end of the function (or when it's done using the database)

orchid charm Jun 18, 2018, 7:26 PM

#

Oh it disconnects when it is done automatically?

patent glen Jun 18, 2018, 7:26 PM

#

...you need to call close()

orchid charm Jun 18, 2018, 7:26 PM

#

so

#

conn.close

patent glen Jun 18, 2018, 7:27 PM

#

yeah

orchid charm Jun 18, 2018, 7:27 PM

#

Sweet

#

Thanks Mate

patent glen Jun 18, 2018, 7:27 PM

#

just fixing the commits might deal with 90% of the problem though

hollow tiger Jun 18, 2018, 8:38 PM

#

why no mongodb

orchid charm Jun 18, 2018, 9:04 PM

#

@hollow tiger Wat 👀

orchid charm Jun 19, 2018, 12:59 AM

#

@modest haven Wait autocommit?

#

How to Enable 👀

#

This would be so damn useful

orchid charm Jun 19, 2018, 4:42 AM

#

How to "Query" Multiple things at once and Insert them into Multiple Variables

#

In SQLite

#

Instead of something like this
cur.execute('SELECT SOMETHING FROM SOMETHING')
result1 = cur.fetchone()
cur.execute('SELECT SOMETHINGELSE FROM SOMETHINGELSE')
result2 = cur.fetchone()

Just do it in one Query and insert the info into the variables all at once

ionic pecan Jun 19, 2018, 6:49 AM

#

select x, y from z

#

look up sqlite sql syntax

#

you can then just unpack the result

#

x, y = cur.fetchone()

orchid charm Jun 19, 2018, 6:51 AM

#

so i just use comma

#

Sweet this works Perfectly

#

This is gonna save me so much Code

#

Does it work the same with Set?

#

If i wanted to set multiple things to multiple things in one query

orchid charm Jun 19, 2018, 8:59 PM

#

So Ally was talking about a Autocommit, how to do that?

dull scarab Jun 20, 2018, 3:55 AM

#

async def dump_log(bot):
    logs = bot.msg_log
    query = """
        INSERT INTO messages
        (msg_id, author_id, channel_id, guild_id, content, date)
        VALUES($1, $2, $3, $4, $5, $6)
        ON CONFLICT(msg_id, author_id)
        DO NOTHING
            """
    args = list(logs.items())

    async with bot.db.transaction():
        await bot.db.executemany(query, args)
    bot.msg_log = []``` Would this be a good way to dump a log of my bot? Or are the more efficient ways to do so with asyncpg

#

Mostly thinking about this part py async with bot.db.transaction(): await bot.db.executemany(query, args)

#

This happens every 5 min, or shutdown of bot

ionic pecan Jun 20, 2018, 11:33 AM

#

hmm

#

apart from preparing the insert i don't think so

#

https://magicstack.github.io/asyncpg/current/api/index.html#prepared-statements

dull scarab Jun 20, 2018, 11:48 AM

#

Had to change it back, since i couldn't directly convert a record to list it seemed

#

async def dump_log(bot):
    logs = bot.msg_log
    query = """
        INSERT INTO messages
        (msg_id, author_id, channel_id, guild_id, content, date)
        VALUES($1, $2, $3, $4, $5, $6)
        ON CONFLICT(msg_id, author_id)
        DO NOTHING
            """
    args = []

    for log in reversed(logs):
        msg = log["msg"]
        if not msg:
            msg = "<Media transmitted>"
        args.append((log["id"], log["author"], log["channel"], 
                        log["guild"], msg, log["date"]))

    async with bot.db.acquire() as connection:
        async with connection.transaction():
            await bot.db.executemany(query, args)
    bot.msg_log = []```

#

So not as pretty anymore

ionic pecan Jun 20, 2018, 11:49 AM

#

hmm

#

are you building some ingestor kind of bot?

dull scarab Jun 20, 2018, 11:49 AM

#

Also bot.db is a pool

#

Not connectiong, so i'd have to fetch the connection to create a statement wouldn't i?

#

It's a twitch stream bot

#

So keeping logs for moderation

ionic pecan Jun 20, 2018, 11:50 AM

#

ohh

dull scarab Jun 20, 2018, 11:50 AM

#

All encrypted and secure, worry not dblSmile

ionic pecan Jun 20, 2018, 11:50 AM

#

i'm not entireley sure myself, I use peewee_async as an ORM in my bot

dull scarab Jun 20, 2018, 11:51 AM

#

It's not a big issue if it's not super efficient, was just curious if anything could be done at this moment

#

It's a rare feature, only available to administrators

#

So won't be used a lot

lavish ferry Jun 20, 2018, 11:22 PM

#

i need help

#

my bot is hosted on heroku, and there I also hosted the .json file, where the points of the people on my server are stored, the problem that every 1 day, my .json file resets, and this is bad because the points of the people of my server resets too, I would put in a database, could someone help me?

cinder pelican Jun 20, 2018, 11:24 PM

#

bot.tags.get ask

delicate fieldBOT Jun 20, 2018, 11:24 PM

#

ask

Asking good questions will yield a much higher chance of a quick response:

• Don't ask to ask your question, just go ahead and tell us your problem.
• Try to solve the problem on your own first, we're not going to write code for you.
• Show us the code you've tried and any errors or unexpected results it's giving
• Keep your patience while we're helping you.

You can find a much more detailed explanation on our website.

cinder pelican Jun 20, 2018, 11:25 PM

#

have you looked into what sort of databases you want to use?

lavish ferry Jun 20, 2018, 11:26 PM

#

no, I wanted one that stores my json and keeps updating forever, without resetting

cinder pelican Jun 20, 2018, 11:34 PM

#

even though SQLite doesn’t have support with async it shouldn’t slow it down too much, unless your bot is really big

lavish ferry Jun 20, 2018, 11:34 PM

#

know if there is another site that I can host my bot and that does not reset my files? why the heroku resets, and that's awful

#

glitch ?

#

.... great help

torn sphinx Jun 20, 2018, 11:56 PM

#

heroku is meant to be for testing not actually putting stuff there

dull scarab Jun 21, 2018, 2:07 PM

#

Can I so similarity searches in postgres?

#

Like sort by closest match to string query, and return a limit of 10 for top 10 matches

pallid schooner Jun 21, 2018, 2:31 PM

#

Hey

#

Im working on a employee db app

#

but

#

I dont know how to store user input to a db

#

Any help?

fair laurel Jun 21, 2018, 2:49 PM

#

Have you tried searching for any tutorials on Google?

pallid schooner Jun 21, 2018, 4:18 PM

#

@fair laurel I see things about relational db

#

But it doesnt help with what I have in mind

fair laurel Jun 21, 2018, 4:25 PM

#

What do you have in mind?

hollow tiger Jun 22, 2018, 6:46 AM

#

Are there any databases that store data in a dictionary format? I currently use a huge json file and my Discord bot is starting to slow down

#

I'm also looking for a database service that's easy to use since i'm new to databases

restive violet Jun 22, 2018, 6:49 AM

#

yaml maybe?

#

or just sqlite

hollow tiger Jun 22, 2018, 6:50 AM

#

but like in a dictionary format

#

{'like':'this'}

#

not a table

#

yaml?

restive violet Jun 22, 2018, 6:54 AM

#

do you know how a dictionary works?

wispy fable Jun 22, 2018, 6:55 AM

#

@hollow tiger If you want a more permanent solution, use an sql database

#

iirc, sqlite3 is included with python

restive violet Jun 22, 2018, 6:56 AM

#

some sqlite comes with it

#

i think it's 3

hollow tiger Jun 22, 2018, 6:56 AM

#

yeahh but like my current bot stores data in a dict format and i don't wanna have to change everything into a table format

wispy fable Jun 22, 2018, 6:56 AM

#

An actual .db file is special because when reading or writing to it, the entire file does not need to be loaded into memory, only the part that is being modified does

restive violet Jun 22, 2018, 6:56 AM

#

idk what you mean dict format

wispy fable Jun 22, 2018, 6:56 AM

#

You can make a table behave like a dict

#

But if you want a performance improvement, then this is an obstacle you'll have to overcome

hollow tiger Jun 22, 2018, 6:57 AM

#

well

restive violet Jun 22, 2018, 6:57 AM

#

sql does a lot of clever shit under the hood for speed

wispy fable Jun 22, 2018, 6:57 AM

#

If you're not using an actual database, then your performance gains will only be marginal

hollow tiger Jun 22, 2018, 6:57 AM

#

idk how i'd implement it

#


    "Rob and Many's hangout": {
        "apps": {
            "junior mod or junior admin": {
                "intro": "Apply for Junior mod or junior admin: Many_Roblox2\nThanks for applying for Junior mod or junior admin, Many_Roblox2! Remember to answer all the questions truthfully.",
                "app": [
                    "Will you follow the rules?",
                    "How active are and how mature are you?",
                    "What can you bring to the server?",
                    "Do you have any ideas to improve the community server?",
                    "Do you have any experience in moderating a discord server?",
                    "Why do you wanna become a moderator?",
                    "How would you handle a issue (A troller for example)",
                    "Will you respect everyone?",
                    "How active are you? From a scale to 1 to 10."
                ]
            },
            "suggestions for the server": {
                "intro": "Apply for Suggestions for the server: {user}\nThank you for applying for Suggestions for the server! Answer all questions truthfully.",
                "app": [
                    "Do you have any suggestions?",
                    "Are you an staff member?",
                    "What is your suggestion?",
                    "Do you think we will add it?",
                    "Do you understand you are not aloud to rage if it's gets denied."
                ]
            },
            "cancel": {
                "intro": "cancel"
            }
        },
        "prefix": [
            "/",
            "-"
        ],
        "members": {
            "gagi12": {},
            "Many_Roblox2": {},
            "TheDutchSwagBoi": {}
        }
    }```

#

that's for one item i have in my dict

#

    "\ud83d\udd25Server's_United\ud83d\udd25": {
        "apps": {},
        "prefix": [
            "/"
        ],
        "members": {}
    }```

wispy fable Jun 22, 2018, 6:58 AM

#

You could also opt to split all the data into multiple json files to emulate what an db file does

hollow tiger Jun 22, 2018, 6:58 AM

#

that's true

#

it's just that my data is like a huge tree structure

#

and simply having it in a grid i think wouldn't be enough

restive violet Jun 22, 2018, 6:58 AM

#

you need to load it into memory

#

that's what will give you speed

#

reading and writing to some file will be a big bottleneck

wispy fable Jun 22, 2018, 6:59 AM

#

Your sql table could be just two columns, both strings representing a key and the contents of each key's location

hollow tiger Jun 22, 2018, 6:59 AM

#

well i was thinking for one db file

#

each table would represent a server

#

but idk how i'd go from there

restive violet Jun 22, 2018, 6:59 AM

#

read up on sqlite

#

people do it all the time

hollow tiger Jun 22, 2018, 7:00 AM

#

how do ppl do it with a huge structure :(

#

here's how my data's organized:

wispy fable Jun 22, 2018, 7:01 AM

#

We're not going to do your reorganizing for you

restive violet Jun 22, 2018, 7:01 AM

#

c.execute('insert into tablename values (?,?,?)', [dict['id'], dict['name'], dict['dob']])

wispy fable Jun 22, 2018, 7:01 AM

#

But we'd be more than happy to help with specific issues

hollow tiger Jun 22, 2018, 7:02 AM

#

if i were to use tables, how would I put a tree in?

#

a multilevel dict

#

if i figure out how to do that then i'd be glad to try to learn sqlite

restive violet Jun 22, 2018, 7:02 AM

#

use executemany

hollow tiger Jun 22, 2018, 7:02 AM

#

but atm idek how i'd put data in

#

executemany?

restive violet Jun 22, 2018, 7:03 AM

#

https://stackoverflow.com/a/31921824

Stack Overflow

Python : How to insert a dictionary to a sqlite database?

I have a sqlite database with a table with following columns :

id(int) , name(text) , dob(text)
I want to insert following dictionary to it :

{"id":"100","name":"xyz","dob":"12/12/12"}
Dictionary

hollow tiger Jun 22, 2018, 7:04 AM

#

i mean like how i'd organize a dictionary if there's more dicts and lists inside

#

that example only has one big dict

#

but i kinda have dicts and lists in the big dict

#

and it's like lists in dicts that are in a bigger dict

#

i can't think of an organized way to keep the data in a table format

restive violet Jun 22, 2018, 7:08 AM

#

why not just combine data of x depth in and just append to db

#

and when you read it out you split it up

hollow tiger Jun 22, 2018, 7:21 AM

#

what are all the benefits of a database anyways?

#

compared to having multiple json files storing data

versed coyote Jun 22, 2018, 7:22 AM

#

Performance, scalability, integrity and concurrency guarantees, redundancy

hollow tiger Jun 22, 2018, 7:23 AM

#

performance?

#

that's a lot of things i do'nt get

#

imma research

#

"integrity and concurrency guarantees, redundancy"

#

sry i'm pretty dumb i'm still learning

restive violet Jun 22, 2018, 7:25 AM

#

sql is not simple under the hood

#

does a lot of cool stuff

#

but the idea is to stay in memory as much as possible

versed coyote Jun 22, 2018, 7:28 AM

#

SQL is a language

#

Look up ACID

#

https://en.m.wikipedia.org/wiki/ACID

ACID

In computer science, ACID (Atomicity, Consistency, Isolation, Durability) is a set of properties of database transactions intended to guarantee validity even in the event of errors, power failures, etc. In the context of databases, a sequence of database operations that satis...

#

One of several fundamental concepts behind relational databases

#

Which is what SQL is a language for

hollow tiger Jun 22, 2018, 7:46 AM

#

i heard about NoSQL?

midnight verge Jun 22, 2018, 7:53 AM

#

@hollow tiger NoSQL are databases that do not use SQL, in general, do not use NoSQL DBs unless you really need it

#

(protip: you do not need it 99% of the time)

hollow tiger Jun 22, 2018, 7:54 AM

#

why not?

midnight verge Jun 22, 2018, 7:57 AM

#

Because SQL provides structure and constraints

#

(among other things)

hollow tiger Jun 22, 2018, 7:58 AM

#

i heard big companies used mongodb

#

y they didn't hav problems?

midnight verge Jun 22, 2018, 7:59 AM

#

actually some did http://www.sarahmei.com/blog/2013/11/11/why-you-should-never-use-mongodb/

Sarah Mei

Why You Should Never Use MongoDB

Disclaimer: I do not build database engines. I build web applications. I run 4-6 different projects every year, so I build a lot of web applications. I see apps with different requirements and diff…

#

and companies using MongoDB do not only use MongoDB most of the time, and only use Mongo for a very specific part of their database

dull scarab Jun 22, 2018, 7:59 AM

#

I read in a recent thread about some firm having issue with mongo as well

#

They migrated to postgres and never looked back

hollow tiger Jun 22, 2018, 7:59 AM

#

oh

midnight verge Jun 22, 2018, 8:00 AM

#

tl;dr: do not use NoSQL unless you need it (and this is usually not the case)

hollow tiger Jun 22, 2018, 8:01 AM

#

i just have a huge structured database

dull scarab Jun 22, 2018, 8:01 AM

#

https://dzone.com/articles/why-we-moved-from-nosql-mongodb-to-postgresql Here's the post actually

dzone.com

Why We Moved From NoSQL MongoDB to PostgreSQL - DZone Database

We have been very happy with Postgres since we migrated from NoSQL MongoDB and we are no longer struggling with managing our database.

hollow tiger Jun 22, 2018, 8:01 AM

#

not database

#

big json file with a dictionary in it

#

but it has lists in dicts in a bigger dict

midnight verge Jun 22, 2018, 8:02 AM

#

if your dictionnary items follow a structure, then you can very easily store the data in a SQL data

hollow tiger Jun 22, 2018, 8:03 AM

#

it sorta does

midnight verge Jun 22, 2018, 8:03 AM

#

then go with SQL DB 👍

hollow tiger Jun 22, 2018, 8:03 AM

#

here's a base thing

#

"Testing": {
    "apps": {},
    "prefix": [
        "/"
    ],
    "members": {}
}```

#

that's one item of the biggest dict

#

each dict item will be a table for me

#

but

#

sometimes

midnight verge Jun 22, 2018, 8:03 AM

#

depends

hollow tiger Jun 22, 2018, 8:04 AM

#

"apps": {
            "crew": {
                "intro": "Thank you for applying to the COVL crew, {applier}! Please fill these qeustions truthfully and dont hesitate to ask one of the staff if there is qeustions!",
                "app": [
                    "What is your Gamertag?",
                    "What is your social club account name?",
                    "What is your age?",
                    "From wich country are you?",
                    "Do you speak English or Dutch?",
                    "What is your gender?",
                    "Why do you want to be in \"COVL\"?",
                    "Do you agree to have a saved outfit of the crew?",
                    "How often are you online playing?"
                ]
            }
        },```

#

this is what "apps" became in a server

midnight verge Jun 22, 2018, 8:04 AM

#

yeah

restive violet Jun 22, 2018, 8:05 AM

#

@hollow tiger can i ask if you know what differs between a list and dict

hollow tiger Jun 22, 2018, 8:05 AM

#

['this','is','a','list']

restive violet Jun 22, 2018, 8:05 AM

#

under the hood though

hollow tiger Jun 22, 2018, 8:05 AM

#

{'this':'is','a':'dict'}

#

under the hood?

restive violet Jun 22, 2018, 8:05 AM

#

they are very different

hollow tiger Jun 22, 2018, 8:06 AM

#

well i kinda like to think of dicts as lists but u can name each item instead of having 0, 1, 2, and 3

#

etc

restive violet Jun 22, 2018, 8:06 AM

#

see that's why i asked

#

dicts operate totally different

#

easier question would be

hollow tiger Jun 22, 2018, 8:06 AM

#

:o

restive violet Jun 22, 2018, 8:06 AM

#

how does a set differ from a list

hollow tiger Jun 22, 2018, 8:06 AM

#

set?

restive violet Jun 22, 2018, 8:06 AM

#

myset = set(1,5,6,7)

hollow tiger Jun 22, 2018, 8:06 AM

#

idk what a set is

restive violet Jun 22, 2018, 8:07 AM

#

ok so

#

when you want to find an item in a list

#

what do you do

hollow tiger Jun 22, 2018, 8:07 AM

#

like[3] for the 2nd item

restive violet Jun 22, 2018, 8:07 AM

#

nah but you don't know which index

#

just value inside

#

["asd","sdfh","gyu","dfd4"] find "gyu"

hollow tiger Jun 22, 2018, 8:08 AM

#

like

#

.index

#

()

#

["asd","sdfh","gyu","dfd4"].index("gyu")

#

2

restive violet Jun 22, 2018, 8:09 AM

#

ehh aside from that

#

what is common method

#

.index is rly just abstraction of it

hollow tiger Jun 22, 2018, 8:09 AM

#

uhmawei

#

i do'nt get what u mean

restive violet Jun 22, 2018, 8:09 AM

#

you do for....

#

for k,item in enumerate(mylist):
if item == "gyu": return k

#

right?

hollow tiger Jun 22, 2018, 8:10 AM

#

oh yea

#

that works too

restive violet Jun 22, 2018, 8:11 AM

#

index is same just abstracted

#

now.. how about a dict

#

how do you find something in a dictionary

#

the value i mean

hollow tiger Jun 22, 2018, 8:11 AM

#


for k,item in enumerate(mydict):
   if item == "gyu": return k``` ig?

restive violet Jun 22, 2018, 8:11 AM

#

nah given a key i mean

hollow tiger Jun 22, 2018, 8:12 AM

#

if key == 'gyu'

#

return item

restive violet Jun 22, 2018, 8:12 AM

#

if dict[key] == 'gyu' right?

hollow tiger Jun 22, 2018, 8:12 AM

#

yeah

restive violet Jun 22, 2018, 8:12 AM

#

the point is

#

from a dict you shouldn't/don't have to iterate through and check this and that

#

it'll get it instantly with the key

#databases

or old style