solid void Apr 30, 2020, 7:30 AM

#

@elfin geyser no that seems really slow

#

I'd say it should take much less than a second

elfin geyser Apr 30, 2020, 7:55 AM

#

@solid void Do you have any idea why it so slow?

elfin geyser Apr 30, 2020, 8:54 AM

#

I have tried connection to mongodb cloud from my computer, and from heroku cloud (which was a little faster, but not significantly)

obsidian leaf Apr 30, 2020, 11:03 AM

#

ive got a snakes and ladders game - and ive got 2 tables players and game - players has a FK for the gameID - and I want to have a winner column in game that is a FK for playerID - is this acceptable or is there a better way to do it?

pale oriole Apr 30, 2020, 12:14 PM

#

@pale oriole try %s instead of ?
@celest blaze works Thanks!

woeful tusk Apr 30, 2020, 1:27 PM

#

@obsidian leaf check out database normalization for some good strategies on PK, FK organization.

#

heres a good starting point: https://www.essentialsql.com/get-ready-to-learn-sql-11-database-third-normal-form-explained-in-simple-english/

Essential SQL

Kris Wenzel

Database Third Normal Form Explained in Simple English - Essential SQL

In this simple to understand article, we'll dig into third normal form and figure out what transitive dependencies really are and why we care!

past current Apr 30, 2020, 5:04 PM

#

Hello guys, is there anyone who know how to run your django rest frame work api on the amazon servers?

boreal steppe Apr 30, 2020, 7:53 PM

#

Did anyone try using marshmallow flask and SQLalchemy together? I can't seem to get my query results to serialize 😩

#

from flask import Flask, request, render_template
from flask_sqlalchemy import SQLAlchemy
from flask_marshmallow import Marshmallow
from sqlalchemy import or_

app = Flask(__name__)

DB_URL = 'postgresql+psycopg2://{user}:{pw}@{url}/{db}'.format(user='user',pw='pw',url='url',db='db')
app.config['SQLALCHEMY_DATABASE_URI'] = DB_URL

db = SQLAlchemy(app)
ma = Marshmallow(app)

class Model(db.Model):
    __tablename__ = 'models'
    username = db.Column(db.String(200), primary_key=True)

class ModelSchema(ma.SQLAlchemyAutoSchema):
    class Meta:
        model = Model

@app.route("/api/getModels", methods=['POST'])
def get_models():
  filters = request.get_data()
  f = json.loads(filters)
  filters = f['filters']
  query = db.session.query(Model).filter(or_(*[getattr(Model, selection ) >=25 for selection in filters])).all()
  final_result = ModelSchema().dump(query)
  # final_result returns {} :(

runic pilot Apr 30, 2020, 8:12 PM

#

try using the "many" flag .dump(query, many=True)

granite oasis May 1, 2020, 12:36 AM

#

Hey can someone help me with mySQL
SELECT DISTINCT e.fName, e.lName FROM Employee e, Dependant d WHERE e.salary > (select AVG(salary) FROM Employee) AND d.ESSN != e.SSN;
Im having an issue where the: *AND d.ESSN != e.SSN; * is not working and print everything and not excluding things.

trim knoll May 1, 2020, 1:16 AM

#

I have a sqlite table with the following columns: id, timestamp, type, info (int, datetime, text, blob).
I want to do something like

select * from table
where timedifference < datetime.timedelta(minutes=30)

and timedifference = timestamp - datetime.datetime.utcnow()

I cant formulate how would I do this in my head. Help will be appreciated

pseudo hill May 1, 2020, 1:45 AM

#

Can someone explain to me why this wont work?

#

mycursor = db.cursor()
def pswdcheck():
    mycursor.execute("SHOW TABLES")
    for x in mycursor:
        global dbpswd
        dbpswd = x
    upswd = input("What is the pasword?\n")
    if dbpswd == "('" + upswd + "',)":
        print("test")
        main()

elfin geyser May 1, 2020, 3:33 AM

#

hi, i am working with mongodb cloud (pymongo driver) and i have there a collection of 1000 documents (200mb overall).
Each document is as follows:

{"_id": 1,
 "a": oneLineDict, #Object
 "b": { #Object
     "1": ListOfHundredsSmallDicts1, #Array of Objects
     "2": ListOfHundredsSmallDicts2,
     ...,
     "up to 15": ListOfHundredsSmallDicts15},
 "_id":2, #doc 2
 }```
My code:
```py
BIG_LIST = mycol.find({"_id": {"$gte": 100, "$lte": 199}})
start_time = time.time()
for api in BIG_LIST:
    print(int(time.time() - start_time))
    break #just for this test
    # only the first loop takes so long. The rest appear immediately.
    # but every 100 it stop for few minutes.

> 65```
It takes over a minute to run this single line `for api in BIG_LIST`! Is there any way i can make it any faster?
I have tried to connect to mongodb cloud from my computer, and from heroku cloud (which was a little faster, but not significantly)

#

print(mycol.find({"_id": {"$gte": first, "$lte": last}}).explain())
> {'queryPlanner': {'plannerVersion': 1, 'namespace': 'mydatabase.mycollection', 'indexFilterSet': False, 'parsedQuery': {'$and': [{'_id': {'$lte': 100}}, {'_id': {'$gte': 199}}]}, 'winningPlan': {'stage': 'FETCH', 'inputStage': {'stage': 'IXSCAN', 'keyPattern': {'_id': 1}, 'indexName': '_id_', 'isMultiKey': False, 'multiKeyPaths': {'_id': []}, 'isUnique': True, 'isSparse': False, 'isPartial': False, 'indexVersion': 2, 'direction': 'forward', 'indexBounds': {'_id': ['[100, 199]']}}}, 'rejectedPlans': []}, 'executionStats': {'executionSuccess': True, 'nReturned': 100, 'executionTimeMillis': 0, 'totalKeysExamined': 100, 'totalDocsExamined': 100, 'executionStages': {'stage': 'FETCH', 'nReturned': 100, 'executionTimeMillisEstimate': 0, 'works': 101, 'advanced': 100, 'needTime': 0, 'needYield': 0, 'saveState': 0, 'restoreState': 0, 'isEOF': 1, 'docsExamined': 100, 'alreadyHasObj': 0, 'inputStage': {'stage': 'IXSCAN', 'nReturned': 100, 'executionTimeMillisEstimate': 0, 'works': 101, 'advanced': 100, 'needTime': 0, 'needYield': 0, 'saveState': 0, 'restoreState': 0, 'isEOF': 1, 'keyPattern': {'_id': 1}, 'indexName': '_id_', 'isMultiKey': False, 'multiKeyPaths': {'_id': []}, 'isUnique': True, 'isSparse': False, 'isPartial': False, 'indexVersion': 2, 'direction': 'forward', 'indexBounds': {'_id': ['[100, 199]']}, 'keysExamined': 100, 'seeks': 1, 'dupsTested': 0, 'dupsDropped': 0}}, 'allPlansExecution': []}, 'serverInfo': {'host': '*****', 'port': ****, 'version': '4.2.6', 'gitVersion': '***'}, 'ok': 1.0, '$clusterTime': {'clusterTime': Timestamp(1588233776, 1), 'signature': {'hash': b'*****', 'keyId': ***}}, 'operationTime': Timestamp(1588233776, 1)}
> (5 seconds)```

alpine patio May 1, 2020, 5:46 AM

#

Is there a way to lock mysql database? From inserting data into it?

boreal steppe May 1, 2020, 10:02 AM

#

try using the "many" flag .dump(query, many=True)
@runic pilot That was it! Thank you 🙂

grand lark May 1, 2020, 10:27 AM

#

Anyone know free mysql database hosts with remote connections????

vague haven May 1, 2020, 2:55 PM

#

i get this error

    raise CommandInvokeError(exc) from exc
discord.ext.commands.errors.CommandInvokeError: Command raised an exception: TypeError: 'method' object is not subscriptable

from this code

results = collectionOwner.find["_id": 0]

pls help

#

im using mongoDB

#

nwm

#

find the problem

lavish ferry May 1, 2020, 6:13 PM

#

SELECT user_id, box_id, square_id, circle_id
    FROM items WHERE '%_id' = '62184'```
how can I detect which of the items above has such a value?

gloomy bone May 1, 2020, 7:21 PM

#

what library are you using ? @lavish ferry

vague haven May 1, 2020, 7:59 PM

#

is there a way to ping a mongoDB?

sudden ocean May 1, 2020, 8:33 PM

#

Can someone recommend what to do when you have a database set up which includes shops, their products, sales, customers, and receipts? Since receipts have foreign key tie in to products, but shops can delete their products at any time, should we record all the product information in the receipts as well? Or should we soft delete products and maintain the foreign key link between receipts and products?

#

I worry about soft deleting because it's tough to maintain and do without breaking several things like on_delete primitives 🙂

#

But I also worry that it doesn't make sense to duplicate so much information across two tables

bronze sail May 1, 2020, 8:42 PM

#

I'm fairly new at SQL and was wondering what are the pros/cons for creating and using a function as compared to using the WITH clause in T SQL?

lavish ferry May 1, 2020, 8:43 PM

#

using asyncpg, but this is in my sql query

sudden ocean May 1, 2020, 9:02 PM

#

@lavish ferry so any of the ids might have that value?

#

If that query doesn't work you can try

SELECT user_id, box_id, square_id, circle_id
    FROM items WHERE '62184' IN (user_id, box_id, square_id, circle_id);

rugged root May 1, 2020, 9:13 PM

#

how do I get row count in python? I tried:

rowcount = mycursor.execute('SELECT * FROM bot_guilds')

But that prints "None"
I also tried:

rowcount = mycursor.execute('SELECT COUNT(*) FROM bot_guilds')

#

That also prints "None"

lavish ferry May 1, 2020, 9:20 PM

#

nice

rich trout May 1, 2020, 9:26 PM

#

.fetchone()

#

mycursor.execute("stuff").fetchone()

torn sphinx May 1, 2020, 10:08 PM

#

has anyone here used psycopg2 to access postgres? i'm not able to connect and i'm not sure what exactly i'm doing incorrectly

#

FATAL: Peer authentication failed for user "postgres"
that's the exception

#

i'm using the default user postgres just to get connected then i'll try with a different user/role

#

postgres=# \conninfo
You are connected to database "postgres" as user "postgres" via socket in "/var/run/postgresql" at port "5432".

#

if anyone has any ideas please @ or dm me

frank needle May 2, 2020, 5:18 AM

#

How do I use SQLite3? I've never used SQL before and I just started needing it for a dpy bot

rain wagon May 2, 2020, 7:17 AM

#

import sqlite3
conn = sqlite3.connect("file")
conn.execute("SQL STATEMENT")

#

@frank needle

frank needle May 2, 2020, 7:17 AM

#

Hmm

rain wagon May 2, 2020, 7:17 AM

#

This is an example without value, but a quick pointer

frank needle May 2, 2020, 7:17 AM

#

Ok

rain wagon May 2, 2020, 7:18 AM

#

If file does not exist, a new database is created, otherwise it is opened

frank needle May 2, 2020, 7:18 AM

#

Alright

rain wagon May 2, 2020, 7:18 AM

#

https://www.sqlitetutorial.net/sqlite-python/

SQLite Tutorial

admin

SQLite Python

This section guides you how to work with the SQLite database using Python sqlite3 module.

#

https://en.wikipedia.org/wiki/Database_normalization

Database normalization

Database normalization is the process of structuring a relational database in accordance with a series of so-called normal forms in order to reduce data redundancy and improve data integrity. It was first proposed by Edgar F. Codd as part of his relational model.
Normalization...

#

Some stuff to read

torn sphinx May 2, 2020, 9:09 AM

#

i am storing a list in database which contains (name, value, date,amount). i would like to make a map inside database using the date provided. and whenever i do select * from table, i should be able to categorize them according to its date. like (today's list) = maybe 5 records. yesterday's ...

rain wagon May 2, 2020, 9:12 AM

#

@torn sphinx Are you storing the list in one field?

torn sphinx May 2, 2020, 9:12 AM

#

no

#

same table each record, in sperate rows

rain wagon May 2, 2020, 9:13 AM

#

Then what is the exact issue?

#

Your question does not really reflect that well

torn sphinx May 2, 2020, 9:13 AM

#

i wanna retrieve records, categorized according to their date

rain wagon May 2, 2020, 9:14 AM

#

SELECT field1,field2,field3 FROM tablename WHERE date >= 2020-02-02

#

This would be general way to go about it

#

assuming the field type is DATE

torn sphinx May 2, 2020, 9:14 AM

#

no

rain wagon May 2, 2020, 9:15 AM

#

no what?

torn sphinx May 2, 2020, 9:16 AM

#

i wanna do select * but later on categorize according date. like if belongs to today getting in one list. if belongs to other date should be stored on its relevant list

rain wagon May 2, 2020, 9:17 AM

#

Then modify the query above using various date ranges

#

SQL only returns what you query

#

It does not have a concept of using different "lists" to return data

#

Or query a lot and sort using python

torn sphinx May 2, 2020, 9:18 AM

#

we can't use map in database ? to give each date a key then retrieve using that key ?

#

like all records which is saved today should be saved under key named "today"

rain wagon May 2, 2020, 9:19 AM

#

And what if it is 0:00 on "today"? How are you going to update it?

#

But no, there is no such concept

#

just query using the actual date

torn sphinx May 2, 2020, 9:19 AM

#

oh yea

#

maybe a make a new column for storing that key

#

anyways. tqu

rain wagon May 2, 2020, 9:20 AM

#

that would be redundant data and violate the normal form

#

The date already reflects today

#

Some database systems may have a variable called TODAY to use in a query

#

The system you are proposing makes the database unreliable if a field is not updated, it violates the normal form and it would be clunky and inperformant to use.

torn sphinx May 2, 2020, 9:22 AM

#

hm

#

ok i will try to think about it

rain wagon May 2, 2020, 9:22 AM

#

If you want all data from today, SELECT field1, field2 from tablename WHERE date == "2020-05-02"

torn sphinx May 2, 2020, 9:23 AM

#

not only today. i want to list all records. but show in category, like today, yesterday, ...

rain wagon May 2, 2020, 9:24 AM

#

As I said, sort it with Python. Have a look at datetime

torn sphinx May 2, 2020, 9:24 AM

#

okkk

#

np

night socket May 2, 2020, 11:43 AM

#

Hi.
Could someone help me with my problem with SQLAlchemy.
I am trying to implement post-comments database and in comment table I have 2 primary key columns - comment_id and post_id (with relationship and foreign key to post table).
I want to know if there is way to make db set comment_id not always incresingly but starting from 1 when post_id changes.
For example:
When I have post_id 1 and add 3 comments (with ids 1, 2, 3 respectively), but when I add post with post_id 2 I want to get comment_id not 4 but 1.
Is there a way to do it?

#

I think was possible in postgres in some ways but I can not figure it out how to make the same for sqlalchemy

undone roost May 2, 2020, 3:43 PM

#

anyone help me with alembic migrations

#

it gives error in running alembic upgrade head

#

sqlalchemy.exc.nosuchmoduleerror can't load plugin sqlalchemy.dialects:driver

undone roost May 2, 2020, 6:06 PM

#

from configuration import app
from core.expense.models import user # added my model here

alembic_config = config.get_section(config.config_ini_section)
alembic_config['sqlalchemy.url'] = app.config['SQLALCHEMY_DATABASE_URI']
engine = engine_from_config(
    alembic_config,
    prefix='sqlalchemy.',
    poolclass=pool.NullPool)

this is the code to be added to use --autogenerate and it says it will work...but i don't have a config file so i imported my app and models differently but i don't know where to import get_section from instead of config

trail cedar May 2, 2020, 7:22 PM

#

CREATE TABLE ban list(
guild_id integer,
member_id integer,
ban_date datetime,
end_date datetime)

this is in sql lite why its not working

vocal moon May 2, 2020, 7:27 PM

#

    mycursor.execute(f"SELECT * FROM wp_users WHERE id = 1")
    await ctx.send(mycursor.fetchall())

why does this not send anything?

celest blaze May 2, 2020, 9:11 PM

#

just a guess: are you using an async sql client? If not, I don't know what happens

safe pagoda May 2, 2020, 10:30 PM

#

The ones that have used django for backend. What are your opinions?

#

I'm planning to go through backend with django

#

I'll like to know how it fits with React

elfin geyser May 2, 2020, 10:34 PM

#

https://discordapp.com/channels/267624335836053506/342318764227821568/705622919971340350

stray thorn May 2, 2020, 11:30 PM

#

Its kinda hard to know exactly where i should be asking this question, but does anyone have any tips on where to go looking for ontologies that have ID annotations? Also looking for an ontology that contains something you're looking for, but dont know what each ontology has is really annoying

torn sphinx May 3, 2020, 12:50 AM

#

I am just wondering, what should I use redis for?

#

And is it okay if I use Postgresql and Redis both?

#

Please ping me if you respond

elfin geyser May 3, 2020, 2:15 AM

#

http://prntscr.com/s9y8gw
I am having problems with cursor iteration (PyMongo & MongoDB) [https://discordapp.com/channels/267624335836053506/342318764227821568/705622919971340350]
So I thought maybe the problem is with my data, so I decided to do a test with the Sample Dataset of mongodb - http://prntscr.com/s9y9mh
I am running the following code for this test:

mycol = client["sample_airbnb"]["listingsAndReviews"]
a = mycol.find()
start_time = time()
for api in a:
    pass
print(time() - start_time)

It print 212.84761023521423 ....
Isn't 3.5 minutes for such a task too much? Is there anything i can do about it?

cinder sierra May 3, 2020, 2:35 AM

#

how many documents are in that collection?

elfin geyser May 3, 2020, 2:35 AM

#

http://prntscr.com/s9y8gw

Lightshot

Screenshot

Captured with Lightshot

cinder sierra May 3, 2020, 2:36 AM

#

hardware and connection speed also affect that time

#

can't remember exactly how pymongo handles a cursor iterator

elfin geyser May 3, 2020, 2:39 AM

#

I thought it should be a matter of a few milliseconds

cinder sierra May 3, 2020, 2:42 AM

#

i mean usually it is but i don't know anything about your setup

#

if you're on dialup with a pentium 2, it'll take a while

elfin geyser May 3, 2020, 2:49 AM

#

"a while" is still less than 3.5 minutes i guess

cinder sierra May 3, 2020, 2:50 AM

#

that definitely shouldn't take that long but why it is i'm not sure honestly man

elfin geyser May 3, 2020, 2:51 AM

#

well, thanks, i am still searching google, no luck yet

rigid terrace May 3, 2020, 8:33 AM

#

Im looking for advice on repairing a CSV file that somehow ended up in disarray during a platform migration. I just created this post in r/learnpython but thought you guys might be able to help as well: https://www.reddit.com/r/learnpython/comments/gcn61k/looking_for_advice_on_automating_the_repair_of_a/ Any advice is appreciated.

r/learnpython - Looking for advice on automating the repair of a CS...

0 votes and 0 comments so far on Reddit

woeful tusk May 3, 2020, 2:58 PM

#

hello @rigid terrace

#

I just look at your screenshot

#

I think this looks not so bad to fix

#

pseudo code:
Read in each row and split into an array
If rnc1 exists then process each column into a dict with key = rncn and val = value
if rnc1 does not exist then youre still processing previous rn-1c1 value so simply process each column into the dict again for rn-1cn .

#

you can easily check for exisitance of key/value to make sure you dont squash previous value (but from your data it looks like simply be duplcate)

#

if rnc1 already exits then you still need to process incase there are updates

#

if you can create a csv file using the data from your screenshot then I will write a quick script for you

#

but I am going into the woods for the afternoon and I return in 7 hours to check your reply

quartz tiger May 3, 2020, 3:09 PM

#

Hello, I'm trying to connect my genre and video together but i'm getting this error TypeError: Additional arguments should be named <dialectname>_<argument>, got 'backref'

I'm not very good at databases so I have no clue how to actually do it and just found it on some random stackoverflow question which I can't find anymore. Here's my models

Genre model

class Genre(db.Model):
    __tablename__ = "genres"

    id = db.Column(db.Integer, primary_key=True)
    name = db.Column(db.String(50), nullable=False)

    def __repr__(self):
        return '<Genre %r>' % self.name

Video model

class Video(db.Model):
    __tablename__ = 'videos'

    # Values which are important
    id = db.Column(db.Integer, primary_key=True)
    title = db.Column(db.String(100), unique=False, nullable=False)
    description = db.Column(db.String(250), unique=False, nullable=True)
    views = db.Column(db.Integer, default=0, nullable=True)
    genre = db.Column('Genre', backref='video_genre', lazy=True)
    
    def __repr__(self):
        return '<Video %r>' % self.title

quartz tiger May 3, 2020, 3:27 PM

#

nvm fixed it

noble oak May 3, 2020, 5:37 PM

#

Are there any websites or something that I can start to learn databases? I'm working on a discord.py bot and databases would be quite helpful

celest blaze May 3, 2020, 5:51 PM

#

the postgresql documentation has a surprisingly good tutorial

north loom May 3, 2020, 6:15 PM

#

how do I call/name these unnamed columns after rotation? df has no attributable columns when I print df.columns as well.

📎 Screen_Shot_2020-05-03_at_1.00.06_PM.png

#

using pandas

sharp idol May 3, 2020, 6:33 PM

#

Is there a way to have MySQL have default values for JSON?

#

Because it is annoying to have to define default json in my code to just insert it

sudden ocean May 3, 2020, 6:58 PM

#

@noble oak just search for 'learn sql' there are tons of resources online

grand trout May 3, 2020, 7:54 PM

#

I'm starting to think I've found a bug in django because this doesnt make sense. So, theres two models relevant to this, solve and challenge (removed almost everything thats irrelevant)

class Challenge(models.Model):
    name = models.CharField(max_length=36, unique=True)
    auto_unlock = models.BooleanField(default=False)
    unlocks = models.ManyToManyField('self', related_name="unlocked_by", blank=True, symmetrical=False)

class Solve(models.Model):
    team = models.ForeignKey(Team, related_name='solves', on_delete=CASCADE)
    challenge = models.ForeignKey(Challenge, related_name='solves', on_delete=CASCADE)

There are 53 challenges in the database, but when I run this, it outputs 74:

solved_challenges = Solve.objects.filter(team=team).values_list('challenge')
        print(len(Challenge.objects.annotate(
                    unlocked=Case(
                        When(unlocked_by__in=solved_challenges, then=Value(True)),
                        default=Value(False),
                        output_field=models.BooleanField()
                    )
                )))```
And this only happens when a when statement is referencing solved_challenges

rigid terrace May 4, 2020, 1:23 AM

#

@woeful tusk I sent you a direct message!

woeful tusk May 4, 2020, 1:24 AM

#

ok I just finished

rigid terrace May 4, 2020, 1:28 AM

#

no worries - i just didnt know if you saw it.

#

not familiar with DMs on Discord

woeful tusk May 4, 2020, 1:29 AM

#

data = [
# Note: header row comented out.
# 'c1,c2,c3,c4,c5,c6,c7',
'c1r2,,c3r2,c4r2,c5r2,c6r2,c7r2',
',c2r2,,,c5r2,,',
',,,,,,',
',,,,,,',
'c1r3,,c3r3,c4r3,c5r3,c6r3,c7r3',
'c1r3,c2r3,,,c5r3,,',
'c1r3,,,,,,',
',,,,,,',
'c1r4,,c3r4,c4r4,c5r4,c6r4,c7r4',
'c1r4,c2r4,,,c5r4,,',
',,,,,,',
',,,,,,',
'c1r5,,c3r5,c4r5,c5r5,c6r5,c7r5',
'c1r5,c2r5,,,c5r5,,'
]

#

I made that data array for testing earlier. I just copied the data from the image you posted

#

Next the script processes each row and this is what the debug output looks like:

0 ['c1r2', '', 'c3r2', 'c4r2', 'c5r2', 'c6r2', 'c7r2']
0 ['c1r2', 'c2r2', '', '', 'c5r2', '', '']
0 ['c1r2', '', '', '', '', '', '']
0 ['c1r2', '', '', '', '', '', '']
1 ['c1r3', '', 'c3r3', 'c4r3', 'c5r3', 'c6r3', 'c7r3']
1 ['c1r3', 'c2r3', '', '', 'c5r3', '', '']
1 ['c1r3', '', '', '', '', '', '']
1 ['c1r3', '', '', '', '', '', '']
2 ['c1r4', '', 'c3r4', 'c4r4', 'c5r4', 'c6r4', 'c7r4']
2 ['c1r4', 'c2r4', '', '', 'c5r4', '', '']
2 ['c1r4', '', '', '', '', '', '']
2 ['c1r4', '', '', '', '', '', '']
3 ['c1r5', '', 'c3r5', 'c4r5', 'c5r5', 'c6r5', 'c7r5']
3 ['c1r5', 'c2r5', '', '', 'c5r5', '', '']

#

so that normalized each row with the first col

#

Then the sanitized output is:

0 ['c1r2', 'c2r2', 'c3r2', 'c4r2', 'c5r2', 'c6r2', 'c7r2']
1 ['c1r3', 'c2r3', 'c3r3', 'c4r3', 'c5r3', 'c6r3', 'c7r3']
2 ['c1r4', 'c2r4', 'c3r4', 'c4r4', 'c5r4', 'c6r4', 'c7r4']
3 ['c1r5', 'c2r5', 'c3r5', 'c4r5', 'c5r5', 'c6r5', 'c7r5']

#

So all the rows match up and the cols as well

#

however I did need to make some assumptions:
Data is treated as strings
The column header row is commented out
the verfy first data row has an entry in col1

trail blade May 4, 2020, 4:22 AM

#

Hello, quick question about sqlite3 database entries

#

does sqlite3 automatically add a column for entry #?

sudden ocean May 4, 2020, 4:23 AM

#

@grand trout look into subquery and other forms of join / subquery in Django because I suspect that what's happened here is you are doing one query and unioning both the solve and the challenge tables, so that's why you get an output higher than 53

trail blade May 4, 2020, 4:24 AM

#

#create table
cursor.execute("""CREATE TABLE all_table (
                alert_type text,
                zone int,
                info float,
                oid int
                )""")


def submit():
    # create database or connect to one
    conn = sqlite3.connect('all_alerts.db')

    # create a cursor
    cursor = conn.cursor()



    #insert into table
    cursor.execute("INSERT INTO all_table VALUES (:alert_type, :zone, :info, :oid)",
              {
                    'alert_type': type.get(),
                    'zone': zone.get(),
                    'info': info.get(),
                    'oid': delete_box.get()
              }


              )

    # commit changes
    conn.commit()

    # close connection
    conn.close()

#using tk, but this entry box is where I enter ID number

delete_box = tk.Entry(main, width=30)
delete_box.grid(row=5, column = 1)
delete_box_label = tk.Label(main, text = 'Select ID')
delete_box_label.grid(row=5, column=0)``` 
there are also entry boxes for alert_type, zone, and info. oid is supposed to be an ID number, that is associated with that database entry
I expect my database to have only 4 columns
but when I enter everything in, i get this...
[('high temp', 1, 15.0, 1, 1)]
What is that fifth value?
If i'm only using .get for 4 entry fields, why does my database end up having 5 columns? that fifth column increments up by one for every additional database entr

safe pagoda May 4, 2020, 11:18 AM

#

From where can I learn Django for free?

#

What's the most complete documentation / book

idle ridge May 4, 2020, 11:23 AM

#

why programming is so hard
i hate it at the same time i love it
seeing results but the process is pain in ass

rough lintel May 4, 2020, 11:55 AM

#

should i use pytables or is there something better y'all would recommend for writing a bot?

fresh tundra May 4, 2020, 12:00 PM

#

hi

valid cobalt May 4, 2020, 4:01 PM

#

what are the best db's to use?

torn sphinx May 4, 2020, 4:41 PM

#

what are the best db's to use?
@valid cobalt I think the starting point would be SQLite Browser easy and powerful and free 😄

open night May 4, 2020, 4:57 PM

#

Anyone here available to help me our real quick?

#

SQL = '''
        INSERT INTO Customer(FirstName, LastName, City, Country, Phone)
             VALUES (?,?,?,?,?)
      ;'''

def NewCustomer(FirstName, LastName, *args):
    try:
        cursor = conn.cursor()
        cursor.execute(SQL,(FirstName, LastName, City, Country, Phone))
    except Exception as err: 
        print('%sBe aware: %s' % (SQL, str(err)))
        pass
    finally:
        print'Closing connection')
        conn.commit()
        
        
    return "The message you want to return"

#

how would i go about making the *args optional?

#

if i run this now, nothing will be appended to database if i'm missing City, Country or Phone

woeful tusk May 4, 2020, 5:23 PM

#

give them default values

open night May 4, 2020, 5:33 PM

#

eg None?

zealous schooner May 4, 2020, 6:33 PM

#

Hello is anyone familiar with Wordpress?

#

I’m trying to host only a database on their and connect it to my website that is on another server(Heroku)

#

Could Django be a alternative content management system

rain wagon May 4, 2020, 7:13 PM

#

why do you need a cms? @zealous schooner

#

for blogs, static wins over wordpress

#

Static HTML has no attack angle, the hosting is dead cheap. I pay 3€/month and 7€/year for my domain

zealous schooner May 4, 2020, 7:15 PM

#

It’s for this company, they need a cms to update information. But the thing is they’ve used Wordpress previously before so... they’d rather that. The website is already coded up, with flask and a database. And they’re both hosted on heroku. Question is, is it possible to JUST integrate the database from heroku into the Wordpress and host it there for CMS

rain wagon May 4, 2020, 7:16 PM

#

https://devcenter.heroku.com/articles/connecting-to-heroku-postgres-databases-from-outside-of-heroku

Connecting to Heroku Postgres Databases from Outside of Heroku | He...

Considerations for connecting to a Heroku Postgres database from outside of Heroku, including SSL and credential details.

#

Heroku Postgres databases are designed to be used with a Heroku app. However, except for private and shield tier databases, they are accessible from anywhere and may be used from any application using standard Postgres clients. For private databases, outside access can be enabled using trusted IP ranges.

zealous schooner May 4, 2020, 7:17 PM

#

I mean I have all the SQL so I could dump it anywhere. My question is, could wordpresss only host databases?

#

Without a website included

#

And we could use that server to connect to my website(which is on another server)

rain wagon May 4, 2020, 7:19 PM

#

That does not even make sense

zealous schooner May 4, 2020, 7:19 PM

#

I don’t know much about Wordpress

rain wagon May 4, 2020, 7:19 PM

#

How would Wordpress "host" a database?

#

It's not an OS

zealous schooner May 4, 2020, 7:20 PM

#

Oh okay

rain wagon May 4, 2020, 7:20 PM

#

Why not run a REST API on Heroku?

#

Then you can pull whatever data you need via that

#

IIRC Wordpress can do some REST as well, but it's wordpress centric, so whatever your needs are it may or may not work

#

https://developer.wordpress.org/rest-api/

WordPress Developer Resources

REST API Handbook | WordPress Developer Resources

The WordPress REST API provides an interface for applications to interact with your WordPress site by sending and receiving data as JSON…

zealous schooner May 4, 2020, 7:22 PM

#

You think Django could be a good alternative?

#

for cms

#

They just need a user friendly reliable feature where they can update/delete from the database

rain wagon May 4, 2020, 7:25 PM

#

I don't know enough about the project to give any recommendation, but for simple database updates there are middleware options that are far more secure than a public interface

#

BUt if it is a Flask App, why does it not have an admin panel?

#

You can even do my special brand of admin panel which involves vpn tunnels and using local addresses

#

so the admin panel does not face the public

zealous schooner May 4, 2020, 7:31 PM

#

I’m using MySQL as a dB, how do you get an admin panel?

rain wagon May 4, 2020, 7:34 PM

#

I mean the app

#

CMS means Content Managment System, so why do you need an extra way to manage the content?

zealous schooner May 4, 2020, 7:36 PM

#

It’s not for me, it’s for the company. So after the website is in production, they would want to be able to modify information from the db but through Wordpress if possible

#

I’m not too familiar with Wordpress

rain wagon May 4, 2020, 7:37 PM

#

I don't even know how to respond to that

zealous schooner May 4, 2020, 7:38 PM

#

Sorry 😂

rain wagon May 4, 2020, 7:38 PM

#

So they had someone make an app with Flask but want to manage it with Wordpress

zealous schooner May 4, 2020, 7:38 PM

#

Yeah :/

#

Kind of

#

but you can’t bring that code to Wordpress right? lol if that even makes sense

woeful tusk May 4, 2020, 7:39 PM

#

manage db thru wordpress?

zealous schooner May 4, 2020, 7:39 PM

#

Yeah

woeful tusk May 4, 2020, 7:39 PM

#

i dont even understand that.

rain wagon May 4, 2020, 7:39 PM

#

I think am done for today. I am going to bed.

woeful tusk May 4, 2020, 7:39 PM

#

just use something like squirrelSQL

#

or one of the many guis

#

maybe hire a real DBA for a day/week to sort it out for them

rain wagon May 4, 2020, 7:40 PM

#

This is what happens when managers without IT knowledge make fancy presentations and use buzzwords they know jack about

zealous schooner May 4, 2020, 7:41 PM

#

I’ll look into it but thanks both of you

#

I appreciate it

#

👍👌

rain wagon May 4, 2020, 7:42 PM

#

My newest hate is "serverless"

#

well someone runs a server somewhere

woeful tusk May 4, 2020, 7:43 PM

#

and "cloud computing" "cloud storage"

rain wagon May 4, 2020, 7:43 PM

#

well, that actually makes sense

#

cloud computing means being able to spawn a couple of VM when you need them

#

but serverless is just bullshit³, it still involves a server

rich trout May 4, 2020, 7:47 PM

#

I would argue that serverless implies lack of server management--you provide just the application, and someone else just runs it. But yeah, it's a big heavily buzzwordy. And lots of people don't seem to understand the restrictions.

#

AWS Lambda is a good example. You execute packaged functions in a small variety of languages, but other than them being on a server somewhere you don't have to think about the server at all. There is no storage, no persistence, and you simply pay for your total utilized resources rather than for "Space". Which for some things is great, but you'd need an external DB server or similar for proper storage.

woeful tusk May 4, 2020, 8:44 PM

#

💯

stray kernel May 4, 2020, 10:27 PM

#

hey guys! Im trying to find a data for a project. Im interested in cryptocurrency. Want to find a legit source not keggle. Anybody give me some suggestion?

safe pagoda May 4, 2020, 10:48 PM

#

Hi

#

Where can I learn Django?

#

If it's cheap or for free better

real timber May 5, 2020, 12:22 AM

#

@safe pagoda i think there's a tutorial called 'django for girls' which is an ok intro

torn sphinx May 5, 2020, 3:18 AM

#

@safe pagoda Have you tried Two Scoops of Django?

#

https://www.feldroy.com/products/two-scoops-of-django-1-11

storm wigeon May 5, 2020, 3:30 AM

#

I have a desire to make a database for a project that stores user_settings, account info, etc.
Due to a certain nature of the project (for reasons) I would like to be able to use interpolation
e.g. f"{foo}:{bar}" for the database itself, allowing the users to make their own foo bar
in their own way. A customizable database.

This db would be accessible from users logging into a website, and this is where they would
enter their foo and bar.

There has been discussion with associates as to which seems to be most practical, and what might be considered bad practice. 3 ideas have been formed and are requesting qualified feedback.

Idea #1 - SQL
SQL is a well known standard and provides serialization.
But with interpolation you can have security risks.
citation: https://cmatskas.com/ef-core-string-interpolation-and-sql-injection/

Idea #2 - Python Dictionary
But I am honestly prefering to implement something similar to a python dictionary.
https://developer.rhino3d.com/guides/rhinopython/python-dictionary-database/

As far as I know, I would just need to ensure a serialization implementation with the py-dict,
and would be sacrificing speed compared to sql. I'm sure there are possible security vectors
associated with python dictionaries.

Idea #3 - Have python dictionary values converted into sql and use sql for db updates.
This would most likely only be used for sql that required interpolation

https://stackoverflow.com/questions/9336270/using-a-python-dict-for-a-sql-insert-statement

All feedback is appreciated

cmatskas

EF Core, String Interpolation and SQL Injection

EF Core has always provided support for inline SQL queries. This means that you could pass a T-SQL query to be executed through the current DbContext. A typical example would look like this: var term = "some search term"); var blogs = db.Blogs.FromSql($"SELECT * FROM dbo.Blogs

Using Python Dictionary as a database

This guide discusses using Python's Dictionary object to access nested data.

Stack Overflow

Using a Python dict for a SQL INSERT statement

I am trying to use a dict to do a SQL INSERT. The logic would basically be:

INSERT INTO table (dict.keys()) VALUES dict.values()
However, I am having a tough time figuring out the correct syntax ...

wind raft May 5, 2020, 4:19 AM

#

how do you get the total number of dates
and then store them in a numbered list
if the mongodb doc is:```
'5/2/20': '600'
'5/3/20': '610'
'5/4/20': '620'
'5/5/20': '630'

i want to store it as:```python
dates = ['5/2/20', '5/3/20', '5/4/20', '5/5/20']
userCounts = ['600', '610', '620', '630']

torn sphinx May 5, 2020, 4:23 AM

#

@wind raft total number of dates? What do you mean?

wind raft May 5, 2020, 4:24 AM

#

basically just convert the mongodb doc into a list like that

#

is my goal

#

through pymongo

#

@torn sphinx

remote harness May 5, 2020, 4:31 AM

#

speaking of MongoDB, @storm wigeon MongoDB sounds like what you're looking for. it's records are serialized as BSON...

{
  'id': 091282930, # static key:value
  'user_settings': { # static key, dynamic values
    'favorite_color': 'void',
    'favorite_band': 'Smash Mouth'
  },
  'custom_entries': { # static key, dynamic values
  }
}

lime grove May 5, 2020, 5:36 AM

#

Is it safe to put your credentials.json into your project directory when making an app for ios and android while using the google sheets and google drive api?
im planning to use a google sheet as sort of a database
its fine if people see the database i just dont want people to edit it or poke around in my google drive
basically the app is a map of community food banks in the area, and the spreadsheet has all the coordinates of the food banks

wind raft May 5, 2020, 5:51 AM

#

why does python motorDict = (collection.find({'serverid': guildId})) equal AsyncIOMotorCursor(<pymongo.cursor.Cursor object at 0x7f87fc2876a0>)

#

for PyMongo (motor engine)

storm wigeon May 5, 2020, 5:54 AM

#

thanks @remote harness, I'll check it out and give it a try. From what I've found on the internet seems like a practical solution to the problem.

sudden ocean May 5, 2020, 8:28 AM

#

@lime grove it is not safe to do this. Any API keys or other credentials ARE visible to users who download your app if you store them in code, especially for users on Android who have their phones rooted. But anyone in possession of your app binary can decompile it and poke through it

sinful stratus May 5, 2020, 1:41 PM

#

whats a good way to use and manage many URLs (strings, basically)? I have many static URLs (especially image URLs) that I use on my application often. I want to be able to use and manage them easily. I tried using json but It feels off and inefficient. any advices?

minor ruin May 5, 2020, 1:50 PM

#

Database

ripe helm May 5, 2020, 4:04 PM

#

Hi, im new to databases

#

is it possible to make a column that auto updates itself,

#

for example a total value

#

that takes 2 items from the row and adds them up

#

and is it possible to have a one value be a list?

brazen charm May 5, 2020, 4:07 PM

#

that depends on your db

#

some do some dont

ripe helm May 5, 2020, 4:08 PM

#

sqlite?

brazen charm May 5, 2020, 4:09 PM

#

with sqlite to store a list you'd need to have the column be a string

#

then use json to dump the list as a string and save that

ripe helm May 5, 2020, 4:10 PM

#

ah i see

brazen charm May 5, 2020, 4:10 PM

#

as sqlite doesnt have that capability to store lists by default

#

tho its not hard todo

ripe helm May 5, 2020, 4:10 PM

#

what about the other thing

brazen charm May 5, 2020, 4:10 PM

#

as for auto count

ripe helm May 5, 2020, 4:10 PM

#

yeah

brazen charm May 5, 2020, 4:10 PM

#

to have it do automatically im not sure with lite

#

but would be easier enough to just have a func do it

ripe helm May 5, 2020, 4:10 PM

#

yeah i know

#

it would just make life easei

brazen charm May 5, 2020, 4:10 PM

#

but wouldnt suprise me if there was the ability to sum values in sqlite

rain wagon May 5, 2020, 4:12 PM

#

If you want to store lists, look at nosql dbs

#

e.g. mongo

#

The question is, if you really need the upside and can live with the downside

#

A NoSQL db makes sense for e.g. large scale network monitoring like Icinga or PTRG, where you get a lot of values in very quick succession via SNMP

brazen charm May 5, 2020, 4:17 PM

#

noSQL tends to only be good for smaller datasets however

rain wagon May 5, 2020, 4:18 PM

#

Not really

brazen charm May 5, 2020, 4:19 PM

#

i mean pretty much, something like mongo is limited to 9MB per request

#

which on larger sets can mean more requests to get data

rain wagon May 5, 2020, 4:20 PM

#

Which is why I said if you get lots of values in quick succession

#

SNMP are lots of small values

#

So..

brazen charm May 5, 2020, 4:20 PM

#

yh

#

personally i use Postgres for my production based stuff or Datomic

#

ive used mongo before but was getting about 40ms response time from the api compared to pg's 5ms but probably could of optimised stuff a bit more to lower that number

rose plank May 5, 2020, 4:28 PM

#

is there any way i can access a google sheets without authentication

brazen charm May 5, 2020, 4:30 PM

#

not without api creds no

ripe helm May 5, 2020, 4:53 PM

#

what even is the syntax error here?

c.execute(f'INSERT INTO members (id, money, workc, jailtime, stealc, rpsc, bank, bankc, total, hbank, heistamount, items, inuse, storage, isworking, tokens) VALUES ({userid}, {person["money"]}, {person["workc"]}, {person["jailtime"]}, {person["stealc"]}, {person["rpsc"]}, {person["bank"]}, {person["bankc"]}, {person["total"]}, {person["hbank"]}, {person["heistamount"]}, {items}, {inuse}, {person["storage"]}, {person["isworking"]}, {person["tokens"]});')

ive been staring at it for 10 mins i actually cant find anything wrong

#

im trying to convert my json file into a db

brazen charm May 5, 2020, 4:56 PM

#

dont use f strings with sql

ripe helm May 5, 2020, 4:57 PM

#

thats the problem?

brazen charm May 5, 2020, 4:57 PM

#

makes them very liable to inject attacks

ripe helm May 5, 2020, 4:57 PM

#

im only running this program once

brazen charm May 5, 2020, 4:57 PM

#

no but this is just a headsup warning xD

ripe helm May 5, 2020, 4:57 PM

#

and thats it

#

ok

#

I just have an enormous json file i wanna convert into a database

brazen charm May 5, 2020, 4:57 PM

#

btw sqlite doesnt need ;

ripe helm May 5, 2020, 4:57 PM

#

i tried without it it still didnt work

brazen charm May 5, 2020, 5:00 PM

#

it doesnt look wrong

#

whats the error message?

ripe helm May 5, 2020, 5:00 PM

#

lemme see

#

  File "C:\Users\Zack Ghanbari\Desktop\Local Hierarchy\dbconverter.py", line 24, in <module>
    c.execute(f'INSERT INTO members (id, money, workc, jailtime, stealc, rpsc, bank, bankc, total, hbank, heistamount, items, inuse, storage, isworking, tokens) VALUES ({userid}, {person["money"]}, {person["workc"]}, {person["jailtime"]}, {person["stealc"]}, {person["rpsc"]}, {person["bank"]}, {person["bankc"]}, {person["total"]}, {person["hbank"]}, {person["heistamount"]}, {items}, {inuse}, {person["storage"]}, {person["isworking"]}, {person["tokens"]});')
sqlite3.OperationalError: near ",": syntax error

brazen charm May 5, 2020, 5:01 PM

#

damn i love sql errors lol

ripe helm May 5, 2020, 5:02 PM

#

my first sql program :(

brazen charm May 5, 2020, 5:02 PM

#

if you move it out of execute

#

that string

#

and print the string

#

and send that

ripe helm May 5, 2020, 5:02 PM

#

ok

#

wHAoh

#

i got

#

spammed

#

with 60 different messages

#

its cuz its in a for loop

#

i'll just send one

brazen charm May 5, 2020, 5:03 PM

#

oof

ripe helm May 5, 2020, 5:03 PM

#

INSERT INTO members (id, money, workc, jailtime, stealc, rpsc, bank, bankc, total, hbank, heistamount, items, inuse, storage, isworking, tokens) VALUES (659923837051207682, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, , , 2, False, 0);

#

ah wait

#

the items and inuse things were supposed to be empty strings

brazen charm May 5, 2020, 5:04 PM

#

those values that arnt anything are what r breaking it

ripe helm May 5, 2020, 5:04 PM

#

strange then

brazen charm May 5, 2020, 5:04 PM

#

0, , , 2,
----^ this bit

ripe helm May 5, 2020, 5:04 PM

#

yeah i saw

#

    userid = int(person["user"])
    items = ""
    inuse = ""
    for item in person["items"]:
        items = f'{items} {item}'
    for item in person["inuse"]:
        inusename = item["name"]
        inusetimer = item["timer"]
        inuse = f'{inuse} {inusename} {inusetimer}'```

#

this is how i defined it though..?

#

here is one dict from hierarchy as an example

brazen charm May 5, 2020, 5:05 PM

#

you sure the for loops arnt just skipping over it tho

ripe helm May 5, 2020, 5:05 PM

#

    "user": "259427180201639946",
    "money": 0,
    "workc": 0,
    "jailtime": 0,
    "stealc": 0,
    "rpsc": 0,
    "bank": 62,
    "bankc": 0,
    "total": 62,
    "hbank": 62,
    "heistamount": 0,
    "items": [],
    "inuse": [],
    "storage": 2,
    "isworking": "False",
    "tokens": 0
  },```

brazen charm May 5, 2020, 5:05 PM

#

are those loops actually looping>?

ripe helm May 5, 2020, 5:05 PM

#

well some of them no

#

most of the lists are empty

#

most of the items and inuse

brazen charm May 5, 2020, 5:06 PM

#

well if items and insue is empty

#

the string is gonna be empty

ripe helm May 5, 2020, 5:06 PM

#

i defined it with "" though

#

shouldnt an empty string be possible?

brazen charm May 5, 2020, 5:06 PM

#

not with how youve done it

#

why we dont use f strings along with the issue of sql attacks

#

we use ? to represent a value

#

then in execute you can do c.execute('somequery ?, ?, ?', (v1, v2, v3,))

ripe helm May 5, 2020, 5:07 PM

#

ah i see

#

thanks

ripe helm May 5, 2020, 7:18 PM

#

why doesnt this work..?

#

def read_value(table, where, what, value):
    conn = sqlite3.connect('hierarchy.db')
    c = conn.cursor()
    reading = c.execute('SELECT ? FROM ? WHERE ? = ?', (value, table, where, what))
    conn.close()
    return reading```

#

theres a syntax error in the reading statement

brazen charm May 5, 2020, 7:19 PM

#

it only works for vaues not table names etc...

ripe helm May 5, 2020, 7:19 PM

#

ohhhhhhhhh

#

ok

#

thx

#

@brazen charm would it be safe to use an f string lol

brazen charm May 5, 2020, 7:20 PM

#

if you dont need the tables to be dynamic it would be better hard coding it

rain wagon May 5, 2020, 7:20 PM

#

@ripe helm It can be if the table name cannot be influenced from the outside

brazen charm May 5, 2020, 7:20 PM

#

but f strings would work

rain wagon May 5, 2020, 7:21 PM

#

if the table name can be input from outside, then don't do it

brazen charm May 5, 2020, 7:21 PM

#

it would be unusual to sql injection yourself with table names bloblul

ripe helm May 5, 2020, 7:21 PM

#

im confused

brazen charm May 5, 2020, 7:21 PM

#

basically you'll be fine

#

as long as

#

the table name and column names

#

are not from some other user input

rain wagon May 5, 2020, 7:21 PM

#

@brazen charm Uhm, just end the statement and start your own

#

that is how injection works

ripe helm May 5, 2020, 7:21 PM

#

okkk

rain wagon May 5, 2020, 7:22 PM

#

; SELECT * from user_passwords; --

brazen charm May 5, 2020, 7:22 PM

#

@rain wagon ik, it was a joke, tho i dont know any real practical applications where the user would input the table name

rain wagon May 5, 2020, 7:23 PM

#

Ok, he is looking for a way to dynamically use the table name. That is fine in itself, however, if the table name depends on an input, it needs to be safeguarded

brazen charm May 5, 2020, 7:23 PM

#

yeah

rain wagon May 5, 2020, 7:23 PM

#

It does not matter at which point you inject, any injection is bad

brazen charm May 5, 2020, 7:23 PM

#

ik

ripe helm May 5, 2020, 7:26 PM

#

also

#

how do i like

#

get the value

#

running that gives me this:

#

<sqlite3.Cursor object at 0x03C12DE0>

brazen charm May 5, 2020, 7:29 PM

#

do cursor.fetchone() or somthing like cursor.fetchall()

ripe helm May 5, 2020, 7:31 PM

#

ok

#

wait

#

im confused what

brazen charm May 5, 2020, 7:33 PM

#

basically

#

when you do cursor.execute()

#

what ever the database returns

#

the cursor essentially contains that data

#

we do cursor.fetchone() for getting a single tuple (row) or cursor.fetchall() to get all the rows that match out WHERE value

#

there are other methods also

#

when you do cursor.fetchall() i would advise saving it to a variable as its a generator so once its iterated over once or referenced once you cant do it again iirc

ripe helm May 5, 2020, 7:37 PM

#

how do i use fetchone?

#

what are the parameters

brazen charm May 5, 2020, 7:38 PM

#

nothing

#

just ()

#

as it just returns a single tuple

ripe helm May 5, 2020, 7:40 PM

#

wait

#

how do you specify what to get then..

brazen charm May 5, 2020, 7:40 PM

#

in the query

#

e.g

#

SELECT * FROM mytable WHERE cats = 5

#

if we called fetchone()

#

after executing that query

#

it would return the first row where cats = 5

#

* means all in this case

#

so select the entire row

#

we could do
SELECT name FROM mytable WHERE cats = 5 and call fetchone(), this would return the value in column name in the row where cats = 5

ripe helm May 5, 2020, 7:46 PM

#

ohhh i see

#

thanks

#

sry for late responses

brazen charm May 5, 2020, 7:47 PM

#

dw

ripe helm May 5, 2020, 7:47 PM

#

how do i make it not a tuple

#

nevermind i did this

#

    c.execute(f'SELECT {value} FROM {table} WHERE {where} = {what}')
    reading = c.fetchone()
    conn.close()
    for x in reading:
        reading = int(x)```

#

idk if theres a more efficient way but it works

brazen charm May 5, 2020, 8:05 PM

#

@ripe helm int(reading[0])

ripe helm May 5, 2020, 8:26 PM

#

oh i guess that works too

#

thx

#

how would i update every row in a database?

#

i want to go through each row indivually and change it

brazen charm May 5, 2020, 8:42 PM

#

err

#

ig? you could do executemany

#

and pass it a list of querys

ripe helm May 5, 2020, 8:44 PM

#

um what

#

does that mean

#

i wanna iterate through it

brazen charm May 5, 2020, 8:44 PM

#

im not rlly sure if there is a good way of doing it

#

you could iterate over every row in the table and update it as u go

#

but that would probably be very bad practice

ripe helm May 5, 2020, 8:46 PM

#

why?

#

how do u do that anyways

#

i just wanna change an entire column to one value

brazen charm May 5, 2020, 8:49 PM

#

for what purpose

#

there is probs a better way of doing it

ripe helm May 5, 2020, 8:51 PM

#

Um

#

kinda hard to explain

runic pilot May 5, 2020, 8:51 PM

#

update tablename set column=value;

ripe helm May 5, 2020, 8:51 PM

#

oh thanks

runic pilot May 5, 2020, 8:51 PM

#

👍

ripe helm May 5, 2020, 9:39 PM

#

how can I get how many rows there are in a database?

#

in sqlite3

brazen charm May 5, 2020, 9:40 PM

#

quick and dirty method would be SELECT * FROM table and then do fetchall()

#

and get the len() of it

ripe helm May 5, 2020, 9:45 PM

#

ah ok thanks

#

is it possible to do something like SELECT id, total FROM table ?

brazen charm May 5, 2020, 9:54 PM

#

maybe

#

ive never tried

ripe helm May 5, 2020, 9:54 PM

#

yolo

reef hawk May 6, 2020, 1:57 AM

#

Is there such thing as appending to an existing list in postgres using asyncpg?

#

Basically, what I want to do is add all existing users in a member list row, in which I can check if member in list (the row) exists

obsidian mural May 6, 2020, 3:28 AM

#

oh sorry yes to who ever helped me with my question

#

the fact / dim statement was regarding star schema stuff

quartz imp May 6, 2020, 3:42 PM

#

does anyone have any experience with sqlalchemy??

#

if yes, I would like to know how to access an object outside a session instead of copying the object data into another datastructure

#

if that makes any sense

woeful tusk May 6, 2020, 7:07 PM

#

do you . mean database session or do you mean user session

rain wagon May 6, 2020, 7:16 PM

#

@ripe helm You should always do a qualified select instead of just select *

#

because that minimizes the risk of having data in a user form that the user isn't mean to see

brazen charm May 6, 2020, 7:28 PM

#

true

ripe helm May 6, 2020, 7:32 PM

#

wdym

brazen charm May 6, 2020, 7:34 PM

#

He means if you are displaying data in what ever method you want to give the bare minimum amount of data as possible to stop the potential for leaking other data

ripe helm May 6, 2020, 7:42 PM

#

thats what ive been doing

#

yee

vale mulch May 6, 2020, 8:47 PM

#

anyone have experience with mongo

#

I have a question about why data i'm entering is being modified

#

by mongo

tacit lark May 6, 2020, 9:41 PM

#

is there any good online courses to learn mySQL or postgre

quartz imp May 6, 2020, 9:48 PM

#

do you . mean database session or do you mean user session
@woeful tusk I mean database session, it's an orm

paper grail May 6, 2020, 10:15 PM

#

Hey if anyone has experience with SQLAlchemy trying to do a self referential join that only loads certain instances of the relationship on the 'child' node. However, when I add a contains eager load to the query it filters the relationship column down to one instance.

e.g.

class Manager(db.Model):
      id = db.Column(db.Integer, primary_key=True)
      user_id = db.Column(db.Integer, db.ForeignKey("user.id", ondelete="CASCADE"), index=True)
      location = db.Column(db.String())
      same_managers = db.relationship(
        "Manager",
        primaryjoin=db.and_(db.remote(user_id) == db.foreign(user_id), db.remote(id) != db.foreign(id)),
        uselist=True,
    )

manager_alias = db.aliased(Manager)

managers = (
    db.session.query(Manager).outerjoin((manager_alias, Manager.same_managers))
    .filter(manager_alias.location == 'CA')
    .options(db.contains_eager(Manager.same_managers.of_type(manager_alias)))
).first()

managers.same_managers == [Manager2] // should be multiple managers.

managers = (
    db.session.query(Manager).outerjoin((manager_alias, Manager.same_managers))
    .filter(manager_alias.location == 'CA')
).first()

managers.same_managers == [Manager2, Manager3, Manager4]

ripe helm May 7, 2020, 4:04 AM

#

What are the most important syntax differences between sqlite3 and aoisqlite?

lilac bolt May 7, 2020, 6:24 AM

#

this might be a rooky mistake, but I need help with these, "check to see if there already isnt a submission" functions

#

def ifexisting(emp):
    c.execute("""SELECT first FROM employees""")
    print(c.fetchall())
    print(emp.first)
    for i in c.fetchall():
        if i == emp.first:
            print("yes")
        else:
            print("no")

#

this is the code

#

for some reason it doesnt print either yes or no

#

this is what the two c.fetchall() and emp.first prints out

#

[('jason',)] jason

rich trout May 7, 2020, 7:31 AM

#

@lilac bolt the way a cursor works is that when you call .execute(), all the results are saved in the cursor. Then, when you call fetchone() or fetchall() it grabs all the results and empties the cursor. So your second fetchall() won't get anything because the first one has already consumed all the results. You should store the result in a variable before printing it and using it, so that you don't fall into that trap

heady elk May 7, 2020, 9:19 AM

#

postgresql need elephantsql?

#

I am new to db, I want to learn PostgreSQL

#

Where to start?

gloomy bone May 7, 2020, 9:51 AM

#

Where to start?
@heady elk Go to google and search for "PostgreSQL"

somber hatch May 7, 2020, 12:47 PM

#

how can i get the guild id, im trying to set costoum prefixes but it isnt working

async def get_prefix(bot, message):
    guild_id = str(message.guild.id)
    user = await bot.pg_con.fetch("SELECT * FROM prefix WHERE guild_id = $1", guild_id)
    if message.guild.id == user['guild_id']:
        prefixes = [user['prefix']]
    else:
        prefixes = []

i tried this but it dosnt do anything, it sends an error if the guild id isnt the same, but prefixes wont work

📎 unknown.png

#

also using PostgreSQL

lime echo May 7, 2020, 12:50 PM

#

I want to make this:
create variable called xp.
check if XP for member_id is already existent in my database
if it is, then on_message() > add +1
if not, then create, give value of 0 and add +1
how can I do that in python?
if(xp != smth(int)):
then...
else
xp++

somber hatch May 7, 2020, 12:55 PM

#

        user = await self.bot.pg_con.fetch("SELECT * FROM users WHERE user_id = $1", author_id)

        if not user:
            await self.bot.pg_con.execute("INSERT INTO users (user_id, lvl, xp) VALUES ($1, 0, 0)",
                                          author_id)
       else:
            await self.bot.pg_con.execute("UPDATE users SET xp = $1 WHERE user_id = $2",
                                          user["xp"] + 1,
                                          author_id)

@lime echo i do this

#

but i have it cross server

#

youd probably want to instert the guild id too

rich trout May 7, 2020, 1:31 PM

#

your get_prefix should return the prefixes?

torn sphinx May 7, 2020, 2:36 PM

#

Hey guys, hope you are all doing well and safe. I got a major problem in a face-recognition algorithm in python. I use face_recognition.
Here is my problem: in the example sketch, we load the image and then encode them. The fact is that it’s a lost of time if you do that every time you start the algorithm. So I decide to send all the encoding in a .csv file and every time we start the program, we load the data that are in the .csv file. The main problem is that those data from the /csv don’t work due to many problem especially with array in numpy.
I don’t want to spam this channel with code but if will be awesome if someone can help me solve this problem. I can send the code by pm.

Thank a lot guys

lilac bolt May 7, 2020, 5:27 PM

#

@rich trout thank you so much!

#

for some reason it still runs into a problem however

#

this is the new code

#

def ifexisting(emp):
    c.execute("""SELECT first FROM employees""")
    results = c.fetchall()
    for i in results:
        print(i)
        print(emp.first)
        if i == emp.first:
            print("yes")
        else:
            print("fail")

#

and this is the output

#

('jason',)
jason
fail

#

apparently "("jason",)" isn't equal to "jason"

willow shard May 7, 2020, 6:14 PM

#

Someone able to tell me how to join tables in SLQALCHEMY so everything yellow is in one table?

📎 join.jpg

rain wagon May 7, 2020, 6:54 PM

#

The standard syntax is py query = db.session.query(ModelClass.value1, ModelClass2.value2).join(ModelClass2, ModelClass.value == ModelClass2.value).all()

#

In the query part, you select the values you need, in the join function you join the values on the join clause

#

This is equivalent to sql SELECT value1, value2 FROM ModelClass JOIN ModelClass2 ON ModelClass.value == ModelClasss2.value

reef hawk May 7, 2020, 6:56 PM

#

I'm making something like "logs" that I'm recording in a db.. is there a way to do so that I can delete the earliest entries and limit the amount of rows to lets say 100?

#

(postgres if that matters)

rain wagon May 7, 2020, 6:56 PM

#

@willow shard

#

@reef hawk With a constraint you can limit the rows to 1000, google it, it is pretty straight forward. Your code will need to delete "old" rows manually

reef hawk May 7, 2020, 6:58 PM

#

ah okay, so there isn't a method that does it automatically?

rain wagon May 7, 2020, 6:58 PM

#

You may be able to do it with a trigger maybe, I'd need to research that though

#

but trigger is a good keyword for looking that up

reef hawk May 7, 2020, 6:59 PM

#

right - thanks!

#

I can definitely do it with triggers, just forgot at the moment

rain wagon May 7, 2020, 6:59 PM

#

https://stackoverflow.com/questions/1743439/how-to-write-a-constraint-concerning-a-max-number-of-rows-in-postgresql

Stack Overflow

How to write a constraint concerning a max number of rows in postgr...

I think this is a pretty common problem.

I've got a table user(id INT ...) and a table photo(id BIGINT, owner INT). owner is a reference on user(id).

I'd like to add a constraint to the table photo

reef hawk May 7, 2020, 7:01 PM

#

👌 ty

willow shard May 7, 2020, 7:08 PM

#

Very helpful to see the SELECT statement. Looks about right, will try ✌️

lime echo May 7, 2020, 7:48 PM

#

@somber hatch oh I only saw your message right now, thanks man!
the self.bot.pg_con... thing seems a bit complicated, I will try to do it a bit easier and I already have my ifs and else set up, I just need to know how to get the largest number in my xp_count

lime echo May 7, 2020, 9:38 PM

#

Can someone help me? I am about to loose my nerves:

@client.event
async def on_message(message):
    db = sqlite3.connect("xp_system.sqlite")
    for member_id in db.execute("""SELECT member_id FROM xpsystemdb"""):
        if member_id == 0:
            xp_count_new=0
            post_count_new=0
            db.execute("""INSERT INTO xpsystemdb(guild_id, member_id, xp_count, post_count) VALUES(?,?,?,?)""", (message.guild.id,message.author.id,xp_count_new, post_count_new))
        else:
            xp_count_old = db.execute("""SELECT MAX(xp_count) AS xp_count FROM xpsystemdb""")
            post_count_old = db.execute("""SELECT MAX(post_count) AS post_count FROM xpsystemdb""")
            xp_count_new=xp_count_old+1
            post_count_new=post_count_old+1
            db.execute("""INSERT INTO xpsystemdb(guild_id, member_id, xp_count, post_count) VALUES(?,?,?,?)""", (message.guild.id,message.author.id, xp_count_new, post_count_new))

#

line 25 Unsupported operand type(s) for +: 'sqlite3.cursor' and 'int'

celest blaze May 7, 2020, 9:39 PM

#

you need to do a fetch thing on your cursor

#

fetch_one I think

runic pilot May 7, 2020, 9:40 PM

#

and after that you have to pull out the data from that item

celest blaze May 7, 2020, 9:40 PM

#

ayup

runic pilot May 7, 2020, 9:40 PM

#

it might just be indexing into it db.execute(...).fetch_one()[0] I think

celest blaze May 7, 2020, 9:41 PM

#

@lime echo I suggest you take the body of that function, and play with it outside of your bot, since that's easier

lime echo May 7, 2020, 9:41 PM

#

Let's me try.

#

It requires me to create another table and such... 😫 I will just fix it real quick on the bot.

keen gorge May 7, 2020, 9:58 PM

#

hey, i use mysql, how can I give my pc acces on my mysql server?
GRANT ALL ON xino.* TO root@ip5f5ae3c8.dynamic.kabel-deutschland.de IDENTIFIED BY "anotherPW"this doesnt work, error is ERROR 1064 (42000): You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near '-deutschland.de IDENTIFIED BY "anotherPW"' at line 1

lime echo May 7, 2020, 10:02 PM

#

it's a lot, I am overwhelmed now.

#

@client.event
async def on_message(message):
    db = sqlite3.connect("xp_system.sqlite")
    for member_id in db.execute("""SELECT member_id FROM xpsystemdb"""):
        if member_id == 0:
            xp_count_new=0
            post_count_new=0
            db.execute("""INSERT INTO xpsystemdb(guild_id, member_id, xp_count, post_count) VALUES(?,?,?,?)""", (message.guild.id,message.author.id,xp_count_new, post_count_new))
        else:
            xp_count_old = db.execute("""SELECT MAX(xp_count) AS xp_count FROM xpsystemdb""").fetchone()[0]
            post_count_old = db.execute("""SELECT MAX(post_count) AS post_count FROM xpsystemdb""").fetchone()[0]
            
            xp_count_new=xp_count_old+1
            post_count_new=post_count_old+1
            db.execute("""INSERT INTO xpsystemdb(guild_id, member_id, xp_count, post_count) VALUES(?,?,?,?)""", (message.guild.id,message.author.id, xp_count_new, post_count_new))

#

Cannot operate on a closed database.

#

It sounds like that I need a cursor.

ripe helm May 7, 2020, 10:24 PM

#

@lime echo do you know how to get one..?

lime echo May 7, 2020, 10:25 PM

#

@ripe helm no.

ripe helm May 7, 2020, 10:26 PM

#

its quite easy

#

here

#

you make it with c = conn.cursor()

#

or in ur case

#

db.cursor()

#

and then just change all ur executes things with c.execute()

#

if you are changing a value, at the end make sure u do db.commit()

#

and then finally put db.close()

deft kelp May 7, 2020, 10:55 PM

#

Ignored Question.........................................

lime echo May 7, 2020, 10:56 PM

#

@ripe helm i will try that right now, thanks.

ripe helm May 7, 2020, 10:56 PM

#

mhm

lime echo May 7, 2020, 10:57 PM

#

@ripe helm should I just use db.commit() once? or each time I change a value?

#

I already have it once at the end of my code.

ripe helm May 7, 2020, 10:58 PM

#

you only need to once

#

@lime echo

reef hawk May 7, 2020, 10:59 PM

#

Don't know if anyone has any rough estimates but how fast would it generally be to query a select for lets say a hundred million rows with 7 columns? I need to select all rows that have the same user_id for example

lime echo May 7, 2020, 10:59 PM

#

Oh oki!

#

cannot operate on a closed database

ripe helm May 7, 2020, 11:02 PM

#

can you send your program

#

@reef hawk which db?

#

sqlite3?

reef hawk May 7, 2020, 11:03 PM

#

postgresql

ripe helm May 7, 2020, 11:03 PM

#

ahh idk

#

@lime echo make sure you have db = sqlite3.connect("xp_system.sqlite"), then c = db.cursor()

lime echo May 7, 2020, 11:04 PM

#

I do have it.

reef hawk May 7, 2020, 11:04 PM

#

do u have a rough estimate for sqlite3 then?

ripe helm May 7, 2020, 11:04 PM

#

strage

#

can you send your program

lime echo May 7, 2020, 11:05 PM

#

import discord
from discord.ext import commands
import datetime
import sys
import sqlite3

client = discord.Client()


@client.event
async def on_ready():
    print("Bot is running")

@client.event
async def on_message(message):
    db = sqlite3.connect("xp_system.sqlite")
    c = db.cursor()
    for member_id in c.execute("""SELECT member_id FROM xpsystemdb"""):
        if member_id == 0:
            xp_count_new=0
            post_count_new=0
            c.execute("""INSERT INTO xpsystemdb(guild_id, member_id, xp_count, post_count) VALUES(?,?,?,?)""", (message.guild.id,message.author.id,xp_count_new, post_count_new))
        else:
            xp_count_old = c.execute("""SELECT MAX(xp_count) AS xp_count FROM xpsystemdb""").fetchone()[0]
            post_count_old = c.execute("""SELECT MAX(post_count) AS post_count FROM xpsystemdb""").fetchone()[0]
            
            xp_count_new=xp_count_old+1
            post_count_new=post_count_old+1
            c.execute("""INSERT INTO xpsystemdb(guild_id, member_id, xp_count, post_count) VALUES(?,?,?,?)""", (message.guild.id,message.author.id, xp_count_new, post_count_new))
            db.commit()
            db.close()
client.run()```
@ripe helm

ripe helm May 7, 2020, 11:05 PM

#

@reef hawk im not sure what you mean, but SELECT * FROM tablename WHERE userid = something, that will select all rows if the userid is something

reef hawk May 7, 2020, 11:06 PM

#

oh yeah I meant the timing for it lol

ripe helm May 7, 2020, 11:06 PM

#

ohh

#

well

#

oof idk

#

im looking at urs rn @lime echo

#

well

lime echo May 7, 2020, 11:07 PM

#

oki

ripe helm May 7, 2020, 11:07 PM

#

im not sure if this is the problem but

#

this is wrong:

#

for member_id in c.execute("""SELECT member_id FROM xpsystemdb""")

#

you gotta use .fetchall()

#

that'll return a tuple

#

@lime echo

lime echo May 7, 2020, 11:10 PM

#

c.execute("""SELECT member_id FROM xpsystemdb""").fetchall()?

ripe helm May 7, 2020, 11:10 PM

#

yeah

#

although im not sure if i've seen anyone do that

lime echo May 7, 2020, 11:11 PM

#

@ripe helm

ripe helm May 7, 2020, 11:11 PM

#

i normally do something like c.execute("""SELECT member_id FROM xpsystemdb""") something = c.fetchall()

lime echo May 7, 2020, 11:25 PM

#

Let's me try it

torn sphinx May 7, 2020, 11:27 PM

#

Hey, i am using pymongo and apparently whenever i try to make around 300+ update requests in less than a minute it lags the discord bot 🤔

#

would there be a fix for it

lime echo May 7, 2020, 11:39 PM

#

@ripe helm you can't imagine how I am happy right now.
It worked.

ripe helm May 7, 2020, 11:41 PM

#

im happy to help 😄

charred void May 8, 2020, 12:30 AM

#

hey guys I need help I need to create a form where people fill it out and when they press send it gets inserted into a database

#

i have the UI finished, and the flask server running, the flask server is connected to the oracle database

#

but I dont know how to actually send the information to the flask server

tight hollow May 8, 2020, 1:04 AM

#

hey frenz. I'm trying to create some models but I'm having some issues conceptualizing. id appreciate some help from anyone willing. im using flask-sqlalchemy. The models are in the image.

Item is a collection of items, they can be harvestable, craftable, and they can appear in recipes.
Recipe is a collection of ingredients that are associated to an Item.
Ingredient is a the amount of a given item in a given recipe

I'm not sure if im saying that right. I'm failing with the relationship aspects

📎 Screen_Shot_2020-05-07_at_6.00.31_PM.png

acoustic silo May 8, 2020, 1:51 AM

#

hey @tight hollow could you change your name to have ascii characters, so it is easily mentionable? (see our nickname policy in #rules)

modern shale May 8, 2020, 2:17 AM

#

hi i'm looking for some beginners info on how to start with databases. i'm a manufacturing engineer at work, and periodically, we need to create some scanning serial numbers and measurement process to sort/filter parts due to time of something else FUBAR. Anyway, long story short, i want to start learning how to get the code take inputs, stash it in a .db and at a separate packing station, give me a red/green flag as to its quality status in the .db

reef hawk May 8, 2020, 2:42 AM

#

📎 unknown.png

#

which operator class should I choose for int4 variables

paper grail May 8, 2020, 4:02 AM

#

Hey if anyone is familiar with sqlalchemy or just databases in general. I have two tables and an association table that joins those two tables to create a many to many relationship. Every once in a while, only in my production setting, i'll get duplicate entries into that join table or if I put a unique constraint i'll get an error where an insertion is attempted that violates the unique constraint. I'm using the SQLAlchemy api to set these fields.

Fake examples below

class User(db.Model):
   id = db.Column(db.Integer, primary_key=True)

class Pet(db.Model):
   id = db.Column(db.Integer, primary_key=True)
   users = db.relationship("User", secondary="pets_to_users")

pets_to_users = db.Table(
    "users_pets",
    db.metadata,
    db.Column("user_id", db.Integer, db.ForeignKey("user.id")),
    db.Column("pet_id", db.Integer, db.ForeignKey("pet.id")),
    db.UniqueConstraint("user_id", "pet_id"),
)

# fake example of how it may be set
pet = Pets.query.first()
pet.users = User.query.all()
db.session.commit()

Any ideas, this one is causing quiet a bit of issues :/

vocal nebula May 8, 2020, 6:11 AM

#

Hey, actually this is more of a general question for database, I've heard somewhere that sqlite 3 is bad mainly because it does not support concurrency, is it true that it's bad because of that? If so, why or why not?

The context of my question is also mainly just on the application of discord bots, by the way. I'm asking because if there is a need to future proof by migrating database in the future if the bot gets bigger, I might as well get all the information I need first. Also because I'm rather comfortable with the library already so I don't really want to change if possible.

torn sphinx May 8, 2020, 6:42 AM

#

does any db truly support concurrency

#

i think you might mean sqlite3 isnt async?

willow shard May 8, 2020, 7:26 AM

#

Im using Flask-SQLALCHEMY with MYSQL db. Problem is i loose connection very often. SO i must reload a page 4-5x until its ready and no error 500. This is the message:

sqlalchemy.exc.OperationalError: (MySQLdb._exceptions.OperationalError) (2013, 'Lost connection to MySQL server during query')

Or

(2006, 'MySQL server has gone away')

I have this in config:

app.config['SQLALCHEMY_ENGINE_OPTIONS'] = {'pool_size' : 100, 'pool_recycle' : 280}

Any idea how to fix or whats the issue in general?

heady elk May 8, 2020, 7:42 AM

#

I just setup postgre and downloaded postgre extension on vsc, connect ECONNREFUSED 127.0.0.1:5432 error popup when at 6/7 connecting, what does it mean?

heady elk May 8, 2020, 8:08 AM

#

psql: error: could not connect to server: could not connect to server: Connection refused (0x0000274D/10061)``` SQL shell says this too

#

This channel have questions more than answers alot

pliant spire May 8, 2020, 8:51 AM

#

Hi guys, hopefully there's someone awake here

#

Hello again, I have an issue writing my pandas dataframe to an sql table in a sqlite db, here's my code:

def sort_and_dump_data_to_db(data_frame):
    sorted_data_frame = data_frame
    sorted_data_frame.sort_values(by=['Popularity'], inplace=True, ascending=False)
    sorted_data_frame.to_sql(f'{artist_name}_Top_Songs',if_exists='replace',con=db_interface.create_connection())
    return db_interface.cursor().execute(f'SELECT * FROM {artist_name}_Top_Songs')

It loops forever at:

sorted_data_frame.to_sql(f'{artist_name}_Top_Songs',if_exists='replace',con=db_interface.create_connection())

@pliant spire

#

I posted this in the help channels but no luck, maybe I can get lucky here?

sudden ocean May 8, 2020, 10:12 AM

#

Just curious what people's thoughts are about testing on the same database you run in production? I use postgres in prod and test on sqlite. And after reading that it's a good idea to test your exotic, implementation-specific queries, I started thinking of using a postgres db in testing too. It wasn't easy to switch over though. Just keeping the DATABASE settings in my Django config the same for test and dev wasn't enough. I have a test suite in pytest and it just errors out when I try to use postgres... Anyone's thoughts?

somber hatch May 8, 2020, 10:51 AM

#

can i install psotgresql + pgadmin so i can have an web ui thing on an ubantu 18.04 vps

clever topaz May 8, 2020, 11:59 AM

#

@pliant spire I assume db_interface is an sqlalchemy Pool? Can you post the code initialising db_interface.

pliant spire May 8, 2020, 12:06 PM

#

oh no, it's just a module that contains functions for interfacing with the db

#

This is the function I call:

def create_connection():
    conn = sqlite3.connect(r'api_calls_and_sorted_data.db', isolation_level=None)
    return conn

#

It actually ended up working, I just truncated the data_frame

#

turns out the to_sql of pandas is notoriously slow

#

def sort_and_dump_data_to_db(data_frame):
    sorted_data_frame = data_frame
    sorted_data_frame.sort_values(by=['Popularity'], inplace=True, ascending=False)
    sorted_data_frame.drop_duplicates(subset='Name', inplace=True)
    sorted_data_frame = sorted_data_frame[:50]
    with db_interface.create_connection() as db_connection:
        sorted_data_frame.to_sql(f'{artist_name}_Top_Songs',if_exists='replace',method='multi',con=db_connection)
    return db_interface.cursor().execute(f'SELECT * FROM {artist_name}_Top_Songs')

#

This is what i'm trying to run now

#

apparently

method='multi'

#

in the to_sql function is supposed to make it faster

safe pagoda May 8, 2020, 12:10 PM

#

Should I learn first Databases, data-science or machine learning?

pliant spire May 8, 2020, 12:10 PM

#

Now I have this error:

Traceback (most recent call last):
  File "C:/Users/Administrator.DELL-26/projects/spotify_playlist_creator/sorter.py", line 39, in <module>
    main()
  File "C:/Users/Administrator.DELL-26/projects/spotify_playlist_creator/sorter.py", line 36, in main
    sort_and_dump_data_to_db(data_frame)
  File "C:/Users/Administrator.DELL-26/projects/spotify_playlist_creator/sorter.py", line 29, in sort_and_dump_data_to_db
    sorted_data_frame.to_sql(f'{artist_name}_Top_Songs',if_exists='replace',method='multi',con=db_connection)
  File "C:\Python37\lib\site-packages\pandas\core\generic.py", line 2663, in to_sql
    method=method,
  File "C:\Python37\lib\site-packages\pandas\io\sql.py", line 521, in to_sql
    method=method,
  File "C:\Python37\lib\site-packages\pandas\io\sql.py", line 1734, in to_sql
    table.insert(chunksize, method)
  File "C:\Python37\lib\site-packages\pandas\io\sql.py", line 755, in insert
    exec_insert(conn, keys, chunk_iter)
  File "C:\Python37\lib\site-packages\pandas\io\sql.py", line 679, in _execute_insert_multi
    conn.execute(self.table.insert(data))
TypeError: insert() takes exactly 2 arguments (1 given)

#

@clever topaz

somber hatch May 8, 2020, 12:11 PM

#

how can i look if an id is in a table

#

i want to take a guild id an look if the guild id is in there

clever topaz May 8, 2020, 12:13 PM

#

@pliant spire I think the issue was that you were passing a function that creates the connection rather than the connection itself.

pliant spire May 8, 2020, 12:14 PM

#

@clever topaz I'm not doing that anymore though

clever topaz May 8, 2020, 12:15 PM

#

So your code is this?

def sort_and_dump_data_to_db(data_frame):
    sorted_data_frame = data_frame
    sorted_data_frame.sort_values(by=['Popularity'], inplace=True, ascending=False)
    sorted_data_frame.drop_duplicates(subset='Name', inplace=True)
    sorted_data_frame = sorted_data_frame[:50]
    with db_interface.create_connection() as db_connection:
    sorted_data_frame.to_sql(f'{artist_name}_Top_Songs',if_exists='replace',method='multi',con=db_connection)

pliant spire May 8, 2020, 12:16 PM

#

You left the return statement out, but yes

clever topaz May 8, 2020, 12:17 PM

#

So the return is to test that the data has been entered? It's incorrect - it should be db_connection.cursor not db_interface (which is the module name, I assume).

#

Try without multi first and see if that works.

pliant spire May 8, 2020, 12:18 PM

#

Actually, i'm either an idiot or a genius lemon_smug because that's just a function, this is what it calls:

def cursor():
    conn = create_connection()
    return conn.cursor()

#

Without multi it runs fine

clever topaz May 8, 2020, 12:18 PM

#

Yeah, that's wrong.

#

I've a feeling that multi doesn't work with SQLite.

#

Actually, i'm either an idiot or a genius because that's just a function, this is what it calls:
def cursor():
    conn = create_connection()
    return conn.cursor()

@pliant spire This creates another connection - which is wrong.

pliant spire May 8, 2020, 12:20 PM

#

It connects to the same db though

clever topaz May 8, 2020, 12:20 PM

#

It's still two connections.

#

Independent of eachother.

#

'multi': Pass multiple values in a single INSERT clause. It uses a special SQL syntax not supported by all backends. This usually provides better performance for analytic databases like Presto and Redshift, but has worse performance for traditional SQL backend if the table contains many columns. For more information check the SQLAlchemy documention.

pliant spire May 8, 2020, 12:21 PM

#

The first connection is closed though...

clever topaz May 8, 2020, 12:22 PM

#

I don't think it's supported by the sqlite3 module but it may be by the sqlalchemy module:

The Insert construct also supports being passed a list of dictionaries or full-table-tuples, which on the server will render the less common SQL syntax of “multiple values” - this syntax is supported on backends such as SQLite, PostgreSQL, MySQL, but not necessarily others:

pliant spire May 8, 2020, 12:22 PM

#

I did make the change to use the same connection though

clever topaz May 8, 2020, 12:22 PM

#

The first connection is closed though...
@pliant spire Why not just use the same connection?

pliant spire May 8, 2020, 12:23 PM

#

I dunno, so many ways to do things in python, never really sure how to choose which way to solve a problem

clever topaz May 8, 2020, 12:23 PM

#

I'd use sqlalchemy as in the docs:
https://pandas.pydata.org/pandas-docs/stable/reference/api/pandas.DataFrame.to_sql.html

pliant spire May 8, 2020, 12:23 PM

#

I'll see what I can do with sqlalchemy and pandas dataframes

#

Thank you man 🙂

clever topaz May 8, 2020, 12:23 PM

#

https://stackoverflow.com/a/14432914 - it also has the added feature of read_sql_table (see near the end of the answer)

#

Last thing - the less connections the better. If you can do it all with a single connection, then why create more?

pliant spire May 8, 2020, 12:25 PM

#

Yeah, you're right, it's local though so...I dunno...maybe readability...i'm not even sure 🤦‍♂️

somber hatch May 8, 2020, 12:30 PM

#

i have my database in my vps, i can connect to it with my browser, bu t can i connect to it with python?

torn sphinx May 8, 2020, 12:46 PM

#

What could be the potential reason for pymongo to be slow?

trim knoll May 8, 2020, 3:07 PM

#

I am using postgres, and I have like 8 columns in my table. In my function, I need to check some data stored in one of the columns. Unfortunately it wont be possible in the where statement so I have to take all the data and then compare it.

My question is what will be more efficient:

select * from table

run my check on one of the attributes

use other attributes if check passes

2) ```
select check_coulum, primarykeycolumn from table
# run check on check_column
select * from table where (check returned true)
# do stuff

brazen charm May 8, 2020, 3:36 PM

#

you could just use SQL to check before

runic pilot May 8, 2020, 3:37 PM

#

can you provide more details? I'm super curious what check you want to do but can't due to a SQL restriction

brazen charm May 8, 2020, 3:38 PM

#

SQL is actually able to do alot of stuff being a complete programming language

#

1 sec

#

https://www.w3schools.com/SQL/sql_check.asp tho this isnt the full docs this does have an example of using the CHECK

runic pilot May 8, 2020, 3:43 PM

#

to answer your question, both are very bad, since you're doing a scan over the whole table. Because of this, your query performance between the two will be about the same (the worst it can be), and your next bottleneck will be transferring all that data over the network. Since option #2 limits the data transfer (by only using 2 columns instead of all 8 of them), that one will appear more performant. But that also assumes that the second query in #2 is almost unnoticable because it's so fast

rain wagon May 8, 2020, 3:43 PM

#

SQL is NOT a programming language

#

SQL is a query language

#

What adds the logic to SQL is TRANSACT, please do not confuse the two

#

@brazen charm

brazen charm May 8, 2020, 3:45 PM

#

that is a good point actually

#

my bad

runic pilot May 8, 2020, 3:45 PM

#

it can be if you use the PL/pgsql extension https://en.wikipedia.org/wiki/PL/pgSQL

rain wagon May 8, 2020, 3:46 PM

#

That is a dialect though

#

I don't mean to be a pain in the ass, but I think it is sometimes important to distinguish

wooden lotus May 8, 2020, 4:52 PM

#

I'm currently trying to make a simple wishlist using django and was currently trying to render something if the wishlist has no items, but it doesn't show up

#

    <h1>Wish List Items</h1>
    {% for item in item_list%} {% if item_list|length == 0 %}
    <h3>No Wish Items Exist</h3>
    <hr />
    {%else %}
    <h3>Hello</h3>
    {% endif %} {% endfor %}
  </body>``` 
is what I have for the html

rain wagon May 8, 2020, 5:02 PM

#

This isn't related to databases

wooden lotus May 8, 2020, 5:12 PM

#

mb

somber hatch May 8, 2020, 5:30 PM

#

so like im getting this error but idk what to do

📎 unknown.png

#

📎 unknown.png

sudden ocean May 8, 2020, 7:26 PM

#

@somber hatch please paste text and not images, also you'll need to show your model / schema and the query which throws this error

boreal steppe May 8, 2020, 7:37 PM

#

Hi guys I am creating a web app which would require me to continiously write updates to a Postgresql table...(Around 4000 updates every minute)...Anything I should watch out for or any best practice tips? I also plan to expose this table as a main part of my web app via a flask rest api

paper grail May 8, 2020, 7:41 PM

#

Hello everyone i'm trying to create a read only user for my tables. I see some documentation does gives access to tables and other to sequences. What does the sequence portion mean? See example below.

CREATE USER readonly WITH PASSWORD 'somepassword';
GRANT SELECT ON ALL TABLES IN SCHEMA public TO readonly;
// sequence on next step?
GRANT USAGE, SELECT ON ALL SEQUENCES IN SCHEMA public TO readonly

torn sphinx May 8, 2020, 9:45 PM

#

hey im using

a = pymongo delete_one
a.acknowledged shows true even if the document didn't exist. what's the way around that?

keen gorge May 8, 2020, 9:46 PM

#

Hey, I created a user in mysql via CREATE USER 'narmy'@'%' IDENTIFIED BY 'Jo***'
Now I wanna connect to it in DBeaver, but there is a Error.
Who can help me?

📎 unknown.png

#

I put in the IP of my Server, the user name narmy and my password

torn sphinx May 9, 2020, 1:44 AM

#

if i am creating a website that people across the USA, and potentially across the world would use, then i should use the timestamp with time zone data type, right? https://www.postgresql.org/docs/current/datatype-datetime.html

#

actually wait.. i think i just use timestamp without timezone and then use UTC in my app

#

hmmmmmm

#

and also USE_TZ = True

celest blaze May 9, 2020, 1:57 AM

#

explicit is better than implicit

#

please use timestamp with time zone

runic pilot May 9, 2020, 1:59 AM

#

always use timestamps and always use UTC

torn sphinx May 9, 2020, 2:41 AM

#

ok so..

in settings.py i should have TIME_ZONE = 'UTC' and USE_TZ = True
in my model, it should be modifydate = models.DateTimeField() (the SQL will translate into "modifydate" timestamp with time zone NOT NULL per sqlmigrate

is this correct? thank you so much @celest blaze and @runic pilot

runic pilot May 9, 2020, 2:45 AM

#

is this flask/sqlalchemy/postgresql?

torn sphinx May 9, 2020, 2:47 AM

#

No it's Django with the stock ORM and PostgreSQL

#

I didn't install sqlalchemy

#

It works fine without it

runic pilot May 9, 2020, 2:49 AM

#

I can't really answer if the django models/settings look good, but the SQL looks good to me

torn sphinx May 9, 2020, 2:54 AM

#

ok cool ty

short flame May 9, 2020, 7:25 AM

#

ok so I'm trying to parse through a json file that starts out something like [ { "id":8100, "key":"Domination", "icon":"perk-images/Styles/7200_Domination.png", "name":"Domination", "slots":[ { "runes":[ { "id":8112, "key":"Electrocute", "icon":"perk-images/Styles/Domination/Electrocute/Electrocute.png", "name":"Electrocute", "shortDesc":"Hitting a champion with 3 separate attacks or abilities in 3s deals bonus <lol-uikit-tooltipped-keyword key='LinkTooltip_Description_AdaptiveDmg'>adaptive damage</lol-uikit-tooltipped-keyword>.", "longDesc":"Hitting a champion with 3 separate attacks or abilities within 3s deals bonus <lol-uikit-tooltipped-keyword key='LinkTooltip_Description_AdaptiveDmg'>adaptive damage</lol-uikit-tooltipped-keyword>. Damage: 30 - 180 (+0.4 bonus AD, +0.25 AP) damage. Cooldown: 25 - 20s <hr>'We called them the Thunderlords, for to speak of their lightning was to invite disaster.'" },

#

when I do ```Python
with open('runeprocessing/runesReforged.json') as f:
data = json.load(f)

keystones = {}
normalrunes = {}
runes = {}
for runes in data:
    print(runes["slots"])```

#

I get this back, but I'm unsure of how to parse any further into the file

📎 unknown.png

#

I'd like to be able to parse slots-runes-id and get the id of some runes

rain wagon May 9, 2020, 7:44 AM

#

@torn sphinx The answer here is ISO8601. UTC timestamps with timezone modifier

frozen forge May 9, 2020, 11:16 AM

#

hello i have a question about Mongoengine

#

i am creating my schemas, and i was wondering if i can make a reference to a object in my fields ?

#

there is the ReferencField for referencing ot other Documents type object

#

but what if i want to store not Document type object but normal objects ?

#

should i store json, and made a json method for the given object or is there a cleaner method ?

safe pagoda May 9, 2020, 12:57 PM

#

What it's a good book to learn databases, ml and data-science?

coarse temple May 9, 2020, 1:02 PM

#

what kind of app do you want to build

#

or program

#

@lgnacio

timber karma May 9, 2020, 1:15 PM

#

Guys

#

@client.command()
async def update(ctx):
    if isAdmin(ctx.author.id):
        start_task = time.time()

        cursor.execute("SELECT user, name FROM users")
        for user, name in cursor:


            current_name = client.get_user(int(user))

            if current_name is None:
                current_name = await client.fetch_user(int(user))

            if name != current_name:
                cursor.execute(f"UPDATE `gamers`.`users` SET `name`= '{current_name}' WHERE (`name` = '{name}');")
                print(name)
                database.commit()

        end_task = time.time()
        
        await ctx.send("Task completed in {} seconds.".format(end_task - start_task))

#

Basically this should update the names of the users in the database

#

but it stops the loop at the first row, idk why

#

📎 unknown.png

rich trout May 9, 2020, 1:16 PM

#

You're reusing the same cursor within your loop

#

I don't think that's a great idea

timber karma May 9, 2020, 1:19 PM

#

You have any solution?

rich trout May 9, 2020, 1:19 PM

#

use two cursors?

#

or save your results into a variable with fetcahall() before looping

#

or just use fetchall in your for statement

timber karma May 9, 2020, 1:21 PM

#

📎 unknown.png

#

the selection works

#

📎 unknown.png

#

the problem comes here

#

but yeah, select is outside the for-loop

rich trout May 9, 2020, 1:29 PM

#

cursor.execute(SELECT) loads the results into the cursor

#

you then begin to loop over the cursor

#

while looping over the cursor (that still has results in it), you perform another execute call

#

does that add to the results? replace them?

#

the docs dont say

#

clearly its not behaving well

timber karma May 9, 2020, 1:31 PM

#

okay i did this

#

📎 unknown.png

#

so in the string update we will have the final command that we have to execute

#

But is it possible to do cursor.execute with multiline strings?

brazen charm May 9, 2020, 1:32 PM

#

executemany() if i remember off the top of my head allows you to execute multiple strings

timber karma May 9, 2020, 1:34 PM

#

📎 unknown.png

#

or maybe argument multi=True?

#

@client.command()
async def update(ctx):
    if isAdmin(ctx.author.id):
        start_task = time.time()

        cursor.execute("SELECT user, name FROM users")
        
        update = ''
        
        for user, name in cursor:


            current_name = client.get_user(int(user))

            if current_name is None:
                current_name = await client.fetch_user(int(user))

            if name != str(current_name):
                update += f"UPDATE `gamers`.`users` SET `name`= '{current_name}' WHERE (`name` = '{name}');"

        cursor.execute(update, multi=True)
        database.commit()

        end_task = time.time()

        await ctx.send("Task completed in {} seconds.".format(end_task - start_task))

#

Everything ok but nothing changing in the database

#

Is there something wrong in my code?

torn sphinx May 9, 2020, 3:07 PM

#

@rain wagon ty

elder heron May 9, 2020, 7:04 PM

#

For some reason I can't import mysql.connector I have run pip install mysql-connector and pip install mysql-connector-python. It keeps giving me the error No module named 'mysql'

pulsar timber May 9, 2020, 7:07 PM

#

hello! I am currently trying to make a discord bot with some rpg functions. My bot has a few commands at the moment and the main currency can be given, earned, used, etc. However, I don't really know how I would make a decent inventory system. I am currently using sqlite.

ripe helm May 9, 2020, 7:34 PM

#

@pulsar timber there are great tutroials out there that you could use, i literally learned databases in 5-10 minutes

#

it aint that hard

#

i suggest codeacademy

#

however after i learned how to use sqlite i realized how much better aiosqlite is for discord bots

#

since if sqlite runs into an error, it'll stop your entire bot

#

and im just too lazy to learn aiosqlite

pulsar timber May 9, 2020, 7:45 PM

#

o ok, will try codeacademy

craggy belfry May 9, 2020, 11:57 PM

#

I am trying to connect Python to a PSQL DB but get the error "OperationalError: FATAL: password authentication failed for user". I've gone into the command line and created a DB in psql as well as a user with password and given that user full privileges. I try logging in via Python but can't authenticate for some reason with the same supplied username

keen gorge May 9, 2020, 11:57 PM

#

How can I install mysql connector on linux ubuntu?

lime echo May 10, 2020, 12:16 AM

#

message_member_id= c.execute("""SELECT xp_count from xpsystemdb WHERE member_id ='message.author.id'""")

#

What's wrong with this line of code?

celest blaze May 10, 2020, 12:19 AM

#

beats me

#

does it emit an error when you run it?

#

Or does it just not do what you'd hoped?

#

I'm guessing the latter

#

I assume message.author.id is a python expression

#

I also assume something like this is what you want: message_member_id= c.execute("""SELECT xp_count from xpsystemdb WHERE member_id = ?""", message.author.id) (the ? might need to be a %s instead; I can never keep those two straight)

#

@lime echo this

lime echo May 10, 2020, 12:23 AM

#

It runs.
Indeed, it doesn't set any value to message.author.id, which is something I don't hope for.
Let's me try it real quick.
It's indeed a python expression.
@celest blaze

#

It didn't work.

lime echo May 10, 2020, 3:00 AM

#

How do I store big int in aeiosqlite3?

#

Member ID shows as 2.5345456e+17 rather than 345345345346456457

lime echo May 10, 2020, 3:36 AM

#

I found a solution, I should just use bigint instead of int as member_id column.

delicate fieldBOT May 10, 2020, 6:56 AM

#

Resources

The Resources page on our website contains a list of hand-selected learning resources that we regularly recommend to both beginners and experts.

full hollow May 10, 2020, 8:04 AM

#

hello which database should i start learning

#

im new to databases btw

rain wagon May 10, 2020, 8:07 AM

#

learn SQL

#

not databases

full hollow May 10, 2020, 8:11 AM

#

k

alpine patio May 10, 2020, 12:26 PM

#

HEY

#

is there a way to run multiple lines at once for mysql in command line ?

#

for example i want to these commands at once run :

for example i want to these commands at once run :

source /expDATABSE2/classroomcollege_db_classroom_classes_classescoadec.sql;
source /expDATABSE2/classroomcollege_db_classroom_classes_classescoadec_groups.sql;
source /expDATABSE2/classroomcollege_db_classroom_classes_classescoadec_students.sql;
source /expDATABSE2/classroomcollege_db_classroom_classes_classescoadec_teacher.sql;

timber karma May 10, 2020, 2:29 PM

#

I have to release a bot that manages a SQL database

#

can someone help me to prevent sql injection?

#

📎 unknown.png

#

I bad formatted all the queries, idk how to fix them

rain wagon May 10, 2020, 2:44 PM

#

That is an sql injection angle right there

#

Did you even bother to google the basics?

#

cursor.execute("SELECT value1, value2 FROM users WHERE user_id LIKE ?", [user.id])

#

also never use *

timber karma May 10, 2020, 2:48 PM

#

in this case will only select the row

rain wagon May 10, 2020, 2:58 PM

#

it always selects the row

#

that is the point

#

If you have data in that row that is not meant to be seen, you can't leak it if you only select what you need

timber karma May 10, 2020, 4:07 PM

#

And what formatting should i use in order to avoid SQL I?

brazen charm May 10, 2020, 4:25 PM

#

he just showed you

#

diffrent libs have diffrent place holders

#

but e.g sqlite uses ? as a place holder

#

like he showed above

lime echo May 10, 2020, 5:59 PM

#

How to check if a record is existent or not, and return a bool?

#

SELECT TOP 1 1 and IF EXIST (SELECT) won't do the job for me because I want a bool return, not a value return.

lime echo May 10, 2020, 6:25 PM

#

I could solve it by using if len(str(member_id_check)) == 0;

lime echo May 10, 2020, 7:00 PM

#

What's wrong with this?

member_id_check = c.execute("""SELECT MAX(xp_count) FROM xpsystemdb WHERE member_id=?""", message.author.id).fetchone()[0]```
Error: ValueError: Parameters are of unsupported type.

hazy mango May 10, 2020, 7:26 PM

#

What type is member_id in the table schema?

#

@lime echo

lime echo May 10, 2020, 7:27 PM

#

bigint @hazy mango

clever topaz May 10, 2020, 7:28 PM

#

@lime echo And message.author.id?

lime echo May 10, 2020, 7:29 PM

#

It's bigint @clever topaz .

clever topaz May 10, 2020, 7:30 PM

#

Can you print it?

lime echo May 10, 2020, 7:30 PM

#

Yes!

clever topaz May 10, 2020, 7:30 PM

#

You mean "bigint" - a literal string?

#

Or type bigint

lime echo May 10, 2020, 7:30 PM

#

wdym

#

type bigint (a big integer)

clever topaz May 10, 2020, 7:31 PM

#

Can you print it?
@clever topaz By this I mean can you paste it here?

hazy mango May 10, 2020, 7:31 PM

#

(It's a discord user ID)

#

(So 18 chars)

lime echo May 10, 2020, 7:31 PM

#

238435934708826112

clever topaz May 10, 2020, 7:32 PM

#

Try with just the number 1 instead of the variable.

#

member_id_check = c.execute("""SELECT MAX(xp_count) FROM xpsystemdb WHERE member_id=?""", 1).fetchone()[0]

#

Actually try this:

member_id_check = c.execute("""SELECT MAX(xp_count) FROM xpsystemdb WHERE member_id=?""", (message.author.id,)).fetchone()[0]

#

That should work.

lime echo May 10, 2020, 7:33 PM

#

Let's me try it out

#

It doesn't work with 1

#

It worked, but then:
if len(str(member_id_check[0])) == 0:
int object is not subscriptable.

#

Let's me try to fix it real quick

clever topaz May 10, 2020, 7:36 PM

#

Nice it seems that you have to pass a tuple

#

This:

member_id_check = c.execute("""SELECT MAX(xp_count) FROM xpsystemdb WHERE member_id=?""", (message.author.id,)).fetchone()[0]

Should be:

member_id_check = c.execute("""SELECT MAX(xp_count) FROM xpsystemdb WHERE member_id=?""", (message.author.id,)).fetchone()

Since fetchone only selects one item anyway (or None if it fails).
Also your if should be:

if member_id_check:

(Which checks if it is None or a returned row)

lime echo May 10, 2020, 7:38 PM

#

Interesting.

#

Well, I want to know if member_id_check is null or not.
Do you think that if member_id_check: will check if it's null or not?

#

@clever topaz

clever topaz May 10, 2020, 7:47 PM

#

Yes.

#

if member_id_check is identical to if member_id_check != False and if member_id_check != None.

#

In this context, the latter.

lime echo May 10, 2020, 7:51 PM

#

Awesome.

#

xp_count_old = c.execute("""SELECT MAX(xp_count) FROM xpsystemdb WHERE member_id=?""", (message.author.id,)).fetchone()
print(xp_count_old)
xp_count=xp_count_old+1```

#

It prints: (14,) instead of 14
Error: Can only concentrate tuple (not "int") to tuple.

clever topaz May 10, 2020, 7:57 PM

#

@lime echo Yes, because it returns a row which usually has more than one field. You do need the [0] to access that item.

#

But it should be accessed inside an if statement which checks that the row is not null.

#

So:

row = c.execute("""SELECT MAX(xp_count) FROM xpsystemdb WHERE member_id=?""", (message.author.id,)).fetchone()
if row:
    xp_count_old = row[0]
    print(xp_count_old)
    xp_count=xp_count_old+1

lime echo May 10, 2020, 7:58 PM

#

I thought that fetchone() would solve it.

#

Oh nice, I have tried this but it didn't work:
xp_count=xp_count_old[0]+1

clever topaz May 10, 2020, 7:59 PM

#

I thought that fetchone() would solve it.
@lime echo
No, because with this:

c.execute("""SELECT MAX(xp_count) FROM xpsystemdb WHERE member_id=?""", (message.author.id,)).fetchone()[0]

If fetchone returns None then you get:

None[0]

Which will obviously give an error.

lime echo May 10, 2020, 8:00 PM

#

I get it.

#

Name row isn't defined.

#

For this code:

row = c.execute("""SELECT MAX(xp_count) FROM xpsystemdb WHERE member_id=?""", (message.author.id,)).fetchone()
if row:
    xp_count_old = row[0]
    print(xp_count_old)
    xp_count=xp_count_old+1```

#

@clever topaz

brisk crest May 10, 2020, 8:16 PM

#

Hello everybody

#

I know Pure sqlalchemy and flask_sqlalchemy which one is better to use with flask? Or no difference?

torn sphinx May 10, 2020, 8:40 PM

#

What do you call a database that is hosted in the directory? I dont wnat to have to set up a whole sql database, i just want the database to be stored in one file that it loads

brazen charm May 10, 2020, 8:47 PM

#

thats not how it works

#

you have sqlite that only has a single sqlite file but its not good for scale

torn sphinx May 10, 2020, 8:50 PM

#

im just making a small discord bot so im not woring about something crazy

#

and also i dont really like using raw sql. Is there good wrapeprs around it?

#

i think this is what i am looking for

#

https://github.com/coleifer/peewee

GitHub

coleifer/peewee

a small, expressive orm -- supports postgresql, mysql and sqlite - coleifer/peewee

clever topaz May 10, 2020, 9:01 PM

#

For this code:

row = c.execute("""SELECT MAX(xp_count) FROM xpsystemdb WHERE member_id=?""", (message.author.id,)).fetchone()
if row:
    xp_count_old = row[0]
    print(xp_count_old)
    xp_count=xp_count_old+1```

@lime echo Did you fix it?

lime echo May 10, 2020, 9:05 PM

#

@clever topaz no, I haven't fixed it yet.
It says that row is undefined.

clever topaz May 10, 2020, 9:05 PM

#

Can you paste the full error?

lime echo May 10, 2020, 9:06 PM

#

Sure.

#

if row:
 NameError: name 'row' is not defined.```

#

Line 37-38:

if row:
      xp_count_old=row[0]```

clever topaz May 10, 2020, 9:13 PM

#

Did you put row = c.execute("""SELECT MAX(xp_count) FROM xpsystemdb WHERE member_id=?""", (message.author.id,)).fetchone() before that line?

#

@lime echo

#

Can you paste your lines 30-40

lime echo May 10, 2020, 9:13 PM

#

No!

clever topaz May 10, 2020, 9:13 PM

#

I think you've got some flow control which is causing it to work differently to how you expect.

lime echo May 10, 2020, 9:13 PM

#

xp_count_old = c.execute("""SELECT MAX(xp_count) FROM xpsystemdb WHERE member_id=?""", (message.author.id,)).fetchone()

clever topaz May 10, 2020, 9:13 PM

#

Oh right, that's why!

lime echo May 10, 2020, 9:14 PM

#

if message.content =="!xp":
                await message.channel.send(f"Your experience is {xp_count_new}")
        else:

            
            xp_count_old = c.execute("""SELECT MAX(xp_count) FROM xpsystemdb WHERE member_id=?""", (message.author.id,)).fetchone()
            if row:
                xp_count_old=row[0]
                print(xp_count_old)
                xp_count=xp_count_old[0]+1
            
            post_count_old= c.execute("""SELECT max(post_count) FROM xpsystemdb WHERE member_id=?""", (message.author.id,)).fetchone()```

clever topaz May 10, 2020, 9:17 PM

#

@lime echo Try:

row = c.execute("""SELECT MAX(xp_count) FROM xpsystemdb WHERE member_id=?""", (message.author.id,)).fetchone()
            if row:
                xp_count_old=row[0]
                print(xp_count_old)
                xp_count=xp_count_old[0]+1

That should work. Note row is the name we store the fetchone in.

wicked fog May 10, 2020, 9:17 PM

#

Use sqlalchemy if you can @torn sphinx

#

Takes care of sanitization automatically

#

And turns your db rows into python objects

#

Pain in the ass to set up

#

But super slick when you get it working

lime echo May 10, 2020, 9:19 PM

#

@clever topaz I tried that.
So, it works, the bot launches and all, but it doesn't store any data in my database anymore.

clever topaz May 10, 2020, 9:19 PM

#

You don't seem to have any INSERTS @lime echo .

lime echo May 10, 2020, 9:24 PM

#

@clever topaz I do.
print(xp_count_old) prints None.

clever topaz May 10, 2020, 9:24 PM

#

Where are your inserts?

#

Also try print(row) before print(xp_count_old)

#

@lime echo

lime echo May 10, 2020, 9:36 PM

#

``` Line 48.

#

For insert.

clever topaz May 10, 2020, 9:42 PM

#

Do you ever commit the cursor @lime echo ?

lime echo May 10, 2020, 9:42 PM

#

@clever topaz

clever topaz May 10, 2020, 9:42 PM

#

Can you post your whole code?

lime echo May 10, 2020, 9:43 PM

#

I do.
Yes I can.

#

import discord
from discord.ext import commands
import datetime
import sys
import sqlite3

class XpSystemdbcog(commands.Cog):
    def __int(self, bot):
            self.bot=bot

    @commands.Cog.listener()
    async def on_message(self, message):
        db = sqlite3.connect("xp_system.sqlite")
        c = db.cursor()
        member_id_check = c.execute("""SELECT MAX(xp_count) FROM xpsystemdb WHERE member_id=?""", (message.author.id,)).fetchone()
      
        if len(str(member_id_check))==0:
            xp_count_new=0
            post_count_new=0
            c.execute("""INSERT INTO xpsystemdb(guild_id, member_id, xp_count, post_count) VALUES(?,?,?,?)""", (message.guild.id,message.author.id,xp_count_new, post_count_new))

            if message.content =="!xp":
                await message.channel.send(f"Your experience is {xp_count_new}")
        else:

            
            row = c.execute("""SELECT MAX(xp_count) FROM xpsystemdb WHERE member_id=?""", (message.author.id,)).fetchone()
            if row:
                xp_count_old=row[0]
                print(row[0])
                print(xp_count_old)
                xp_count=xp_count_old+1
            
                post_count_old= c.execute("""SELECT max(post_count) FROM xpsystemdb WHERE member_id=?""", (message.author.id,)).fetchone()

                author_id = message.author.id
                print(author_id)
                post_count=post_count_old[0]+1
                print(post_count_old)
                c.execute("""INSERT INTO xpsystemdb(guild_id, member_id, xp_count, post_count) VALUES(?,?,?,?)""", (message.guild.id,message.author.id, xp_count, post_count,))
                db.commit()
                if message.content =="!xp":
                    await message.channel.send(f"Your experience is {xp_count}")  
            db.close()
        
def setup(bot):
    bot.add_cog(XpSystemdbcog(bot))

#

🤔 even print(row) returns None

#

@clever topaz

clever topaz May 10, 2020, 9:48 PM

#

@lime echo

import discord
from discord.ext import commands
import datetime
import sys
import sqlite3

class XpSystemdbcog(commands.Cog):
    def __int(self, bot):
            self.bot=bot

    @commands.Cog.listener()
    async def on_message(self, message):
        db = sqlite3.connect("xp_system.sqlite")
        c = db.cursor()
        member_id_check = c.execute("""SELECT MAX(xp_count) FROM xpsystemdb WHERE member_id=?""", (message.author.id,)).fetchone()

        # Member doesn't exist - add
        if not member_id_check:
            xp_count_new=0
            post_count_new=0
            c.execute("""INSERT INTO xpsystemdb(guild_id, member_id, xp_count, post_count) VALUES(?,?,?,?)""", (message.guild.id,message.author.id,xp_count_new, post_count_new))

            if message.content == "!xp":
                await message.channel.send(f"Your experience is {xp_count_new}")
        else:
            row = c.execute("""SELECT MAX(xp_count) FROM xpsystemdb WHERE member_id=?""", (message.author.id,)).fetchone()
            if row:
                xp_count_old=row[0]
                print(row[0])
                print(xp_count_old)
                xp_count=xp_count_old+1

                post_count_old= c.execute("""SELECT max(post_count) FROM xpsystemdb WHERE member_id=?""", (message.author.id,)).fetchone()

                author_id = message.author.id
                print(author_id)
                post_count=post_count_old[0]+1
                print(post_count_old)
                c.execute("""INSERT INTO xpsystemdb(guild_id, member_id, xp_count, post_count) VALUES(?,?,?,?)""", (message.guild.id,message.author.id, xp_count, post_count,))
                db.commit()
                if message.content =="!xp":
                    await message.channel.send(f"Your experience is {xp_count}")
            db.close()

def setup(bot):
    bot.add_cog(XpSystemdbcog(bot))

if len(str(member_id_check))==0: was wrong - so I changed it.

lime echo May 10, 2020, 9:48 PM

#

Which is what I want

#

Thanks.

clever topaz May 10, 2020, 9:51 PM

#

Did it work? @lime echo

lime echo May 10, 2020, 9:52 PM

#

I think that this is a tautology:

 else:
            row = c.execute("""SELECT MAX(xp_count) FROM xpsystemdb WHERE member_id=?""", (message.author.id,)).fetchone()
            if row:
                xp_count_old=row[0]
                print(row[0])``` to
```else:```

#

Well it's better but it doesn't store data in my database.

#

@clever topaz

#

I also have an issue with this
xp_count=xp_count_old+1 since xp_count is None

clever topaz May 10, 2020, 9:56 PM

#

@lime echo

import discord
from discord.ext import commands
import datetime
import sys
import sqlite3

class XpSystemdbcog(commands.Cog):
    def __int(self, bot):
            self.bot=bot

    @commands.Cog.listener()
    async def on_message(self, message):
        db = sqlite3.connect("xp_system.sqlite")
        c = db.cursor()
        member_id_check = c.execute("""SELECT MAX(xp_count) FROM xpsystemdb WHERE member_id=?""", (message.author.id,)).fetchone()

        # Member doesn't exist - add
        if not member_id_check:
            xp_count_new=0
            post_count_new=0
            c.execute("""INSERT INTO xpsystemdb(guild_id, member_id, xp_count, post_count) VALUES(?,?,?,?)""", (message.guild.id,message.author.id,xp_count_new, post_count_new))
            db.commit()

            if message.content == "!xp":
                await message.channel.send(f"Your experience is {xp_count_new}")
        else:
            row = c.execute("""SELECT MAX(xp_count) FROM xpsystemdb WHERE member_id=?""", (message.author.id,)).fetchone()
            xp_count_old=row[0]
            print(row[0])
            print(xp_count_old)
            xp_count=xp_count_old+1

            post_count_old= c.execute("""SELECT max(post_count) FROM xpsystemdb WHERE member_id=?""", (message.author.id,)).fetchone()

            author_id = message.author.id
            print(author_id)
            post_count=post_count_old[0]+1
            print(post_count_old)
            c.execute("""INSERT INTO xpsystemdb(guild_id, member_id, xp_count, post_count) VALUES(?,?,?,?)""", (message.guild.id,message.author.id, xp_count, post_count,))
            db.commit()
            if message.content =="!xp":
                await message.channel.send(f"Your experience is {xp_count}")

        db.close()

def setup(bot):
    bot.add_cog(XpSystemdbcog(bot))

Not exactly a tautology as row may be null (as in this case though it isn't being detected as such).

#

However I found the mistake.

#

You didn't have a commit for the first insert.

lime echo May 10, 2020, 10:09 PM

#

I still do have this problem with
print(row[0])
print(xp_count_old) being None
Hence
xp_count=xp_count_old+1 doesn't work.

clever topaz May 10, 2020, 10:10 PM

#

Does data get entered into the DB?

#

@lime echo

lime echo May 10, 2020, 10:11 PM

#

No.

clever topaz May 10, 2020, 10:12 PM

#

@lime echo Rather than INSERT INTO xpsystemdb(guild_id, member_id, xp_count, post_count) VALUES(?,?,?,?) try INSERT INTO xpsystemdb VALUES(?,?,?,?)

lime echo May 10, 2020, 10:13 PM

#

Oki

clever topaz May 10, 2020, 10:13 PM

#

I still do have this problem with
print(row[0])
print(xp_count_old) being None
Hence
xp_count=xp_count_old+1 doesn't work.
@lime echo This issue won't fix until data is being saved. Also can you print member_id_check?

lime echo May 10, 2020, 10:14 PM

#

I will try.

clever topaz May 10, 2020, 10:14 PM

#

Ah I see why.

#

Even when null, it returns (None,) which is not none, thus it always goes to the else.

lime echo May 10, 2020, 10:16 PM

#

Oh!

#

How can it can be "not none" if it's None lmao

clever topaz May 10, 2020, 10:16 PM

#

@lime echo Try this:

import discord
from discord.ext import commands
import datetime
import sys
import sqlite3

class XpSystemdbcog(commands.Cog):
    def __int(self, bot):
            self.bot=bot

    @commands.Cog.listener()
    async def on_message(self, message):
        db = sqlite3.connect("xp_system.sqlite")
        c = db.cursor()
        member_id_check = c.execute("""SELECT MAX(xp_count) FROM xpsystemdb WHERE member_id=?""", (message.author.id,))

        # Member doesn't exist - add
        if member_id_check.rowcount == -1:
            xp_count_new=0
            post_count_new=0
            c.execute("""INSERT INTO xpsystemdb(guild_id, member_id, xp_count, post_count) VALUES(?,?,?,?)""", (message.guild.id,message.author.id,xp_count_new, post_count_new))
            db.commit()

            if message.content == "!xp":
                await message.channel.send(f"Your experience is {xp_count_new}")
        else:
            row = member_id_check.fetchone()
            xp_count_old=row[0]
            print(row[0])
            print(xp_count_old)
            xp_count=xp_count_old+1

            post_count_old= c.execute("""SELECT max(post_count) FROM xpsystemdb WHERE member_id=?""", (message.author.id,)).fetchone()

            author_id = message.author.id
            print(author_id)
            post_count=post_count_old[0]+1
            print(post_count_old)
            c.execute("""INSERT INTO xpsystemdb(guild_id, member_id, xp_count, post_count) VALUES(?,?,?,?)""", (message.guild.id,message.author.id, xp_count, post_count,))
            db.commit()
            if message.content =="!xp":
                await message.channel.send(f"Your experience is {xp_count}")

        db.close()

def setup(bot):
    bot.add_cog(XpSystemdbcog(bot))

#

The None is inside the tuple which is an object thus not None.

lime echo May 10, 2020, 10:17 PM

#

Oh I get it.

#

By the way, you're right because print(member_id_check) doesn't execute.

#

if member_id_check.rowcount == -1:
'tuple' object has no attribute 'rowcount'

clever topaz May 10, 2020, 10:21 PM

#

member_id_check = c.execute("""SELECT MAX(xp_count) FROM xpsystemdb WHERE member_id=?""", (message.author.id,)) without the .fetchone()

lime echo May 10, 2020, 10:22 PM

#

I am trying:
if member_id_check.rowcount == 0: now.

#

Oki

clever topaz May 10, 2020, 10:22 PM

#

No should be -1

#

It returns -1 if nothing found.

lime echo May 10, 2020, 10:23 PM

#

Oh oki

#

Oh my God, I can't believe. It stores data in my database now!

#

It doesn't increment anything but at least, it store data.

clever topaz May 10, 2020, 10:24 PM

#

Nice!

lime echo May 10, 2020, 10:25 PM

#

Thank you so much.

clever topaz May 10, 2020, 10:25 PM

#

No worries.

lime echo May 10, 2020, 10:40 PM

#

@clever topaz can you still help me figure out a solution for another problem?

#

If yes, then here is the code

      
        if member_id_check.rowcount == -1:```

#

oh my God I can see why it doesn't work

#

let's me try smth real quick 😛

lime echo May 11, 2020, 12:47 AM

#

@clever topaz the bot is fixed, and it works completely well ✅
thank you for your help!

uncut egret May 11, 2020, 1:31 AM

#

hi

#

"SELECT id, elo FROM stats WHERE elo = 2000"```

#

how do i do a range from 2000-2200

#

i want everyone with between 2000 and 2200

celest blaze May 11, 2020, 1:39 AM

#

probably something like WHERE 2000 <= elo <= 2200

#

depends on the database

#

not really a python question

torn sphinx May 11, 2020, 7:51 AM

#

idk if this is the wrong channel for this but how do I get rid of this popup when there is no installation visibly running? I tried twice to download and install MySQL but the first few times, it crashed or paused and wouldn't pick up again. I've since deleted the installers and tried again. I also checked the task manager to see if it was running there but nothing showed up besides the apps I intentionally had open.

📎 unknown.png

brazen charm May 11, 2020, 8:53 AM

#

save yourself the hassle and dont use mysql

#

if you're gonna go through the effort of setting up the server and all that you might aswell use postgres

wicked fog May 11, 2020, 8:56 AM

#

PostgreSQL best sql

peak willow May 11, 2020, 10:14 AM

#

how can I create rows like that

📎 unknown.png

#

model

📎 unknown.png

#

actually :/

📎 unknown.png

#

(django, sqlite3)

upbeat lily May 11, 2020, 11:06 AM

#

The Django ORM/Migration tool should create the tables for you. If you've defined your modules, then you need to use manage.py and migrations/makemigrations to let the tool create the tables for you

#

The django tutorial docs should explain that

peak willow May 11, 2020, 11:08 AM

#

But I defined modules, maked migrations

upbeat lily May 11, 2020, 11:40 AM

#

did you run python manage.py makemigrations followed by python manage.py migrate?

peak willow May 11, 2020, 11:54 AM

#

yes

#

My database normally works, but I want to have columns

#

like firstname - lastname - email

somber hatch May 11, 2020, 3:23 PM

#

in mongoDB you must have the _id, how can i get the last id so the new id can be +1 to the last one

#

i jsut tried this xd

📎 unknown.png

somber hatch May 11, 2020, 4:25 PM

#

dont know why i thought that would work xd

clever topaz May 11, 2020, 4:46 PM

#

@somber hatch Apparently $inc can be used - https://stackoverflow.com/a/17054663

Stack Overflow

PyMongo $inc having issues

When trying to use $inc I get a Syntax error stating that $set is wrong. My query is as follow:

mongo_db.campaign.update({'_id': str(campaign_id)}, { $inc: { 'item': 1 } }):
I've looked at the mo...

somber hatch May 11, 2020, 5:00 PM

#

yeah but how could i do it lol

small gust May 11, 2020, 7:31 PM

#

Is there a way to add a function to a sqlalchemy query, in place?

db.session.query(AssessmentPlan, func.timezone("US/Pacific", AssessmentPlan.updated_at))

In this example, I get two different results (<AssessmentPlan 1>, datetime.datetime(2020, 5, 11, 12, 0, 59, 913401)), but I really just want the (<AssessmentPlan 1>), with the updated_at column adjusted in place

pulsar valley May 11, 2020, 7:33 PM

#

hey guys what dbms would you recommend for a beginner like me. I've heard of SQLite but I think it's not as secure and doesn't have that many datatypes as mysql, I've also heard of mongodb. Those are pretty much the only three I know, pls can anyone give me some direction

small gust May 11, 2020, 7:34 PM

#

SQlite >>>>>> MongoDB

#

You're in a Python channel, so the obv candidate is Postgres. But SQLite is no slouch. It depends on what you are doing.

solemn ridge May 11, 2020, 7:35 PM

#

dude

#

You cant compare NoSQL to SQL

#

lmao

#

They are used for different things

small gust May 11, 2020, 7:35 PM

#

A lot of tutorial unfortunately use MongoDB

#

Beginner tutorials

solemn ridge May 11, 2020, 7:36 PM

#

Why unfortunately

#

Their complexity are not the same

#

and as far as how easy one is, NoSQL is close to the json format

thorn nymph May 11, 2020, 8:12 PM

#

Does anyone knows how i can create multiple databases, iam doing it in flask with sqlalchemy, one should contain users, 2nd posts and 3rd comments

pulsar valley May 11, 2020, 8:13 PM

#

So is mongo that bad?

runic pilot May 11, 2020, 8:15 PM

#

mongo is great

#

so is SQL

#

they're different tools

pulsar valley May 11, 2020, 8:34 PM

#

oh so mongo is not an rdbms

runic pilot May 11, 2020, 8:35 PM

#

it's not relational, no

runic pilot May 11, 2020, 8:51 PM

#

nice, they're usually useful for things like that

ripe helm May 11, 2020, 9:44 PM

#

why is this:

    conn = sqlite3.connect('hierarchy.db')
    c = conn.cursor()
    c.execute('SELECT * FROM polls')```
giving me:
```sqlite3.OperationalError: no such table: polls```
It was working fine before, but i changed up a couple lines and now it doesnt work..??

gloomy bone May 11, 2020, 9:50 PM

#

maybe there is no table such table as "polls" @ripe helm

ripe helm May 11, 2020, 9:50 PM

#

there clearly is

#

i see it with db browser

gloomy bone May 11, 2020, 9:52 PM

#

maybe your python script doesn't run in the destination working dir and sqlite3 creating a new "hierarchy.db" file

ripe helm May 11, 2020, 9:55 PM

#

im 100% sure it exists

#

ive been using it

#

i just changed a couple lines that has nothing related with it

#

and now it doesnt work

#

in fact, none of my databases work

#

strange.. i did literally nothing and it works now

glad bobcat May 12, 2020, 2:23 AM

#

Small sqlalchemy session question I’m not finding the answer too online.
Are sqlalchemy session automatically closed when they’re part of a function that has returned?

uncut egret May 12, 2020, 4:09 AM

#

hi]

#

sqlite3.OperationalError: database is locked

#

😦

lime echo May 12, 2020, 4:41 AM

#

@uncut egret quit the database and launch it again.

#

And the bot, if you're using a Discord bot.

toxic rune May 12, 2020, 7:48 AM

#

I have quite a dumb (tired) question I have the following schema: https://mystb.in/qimuzodiju.sql
Basically every continent has multiple chapters and every chapter has multiple stages. Would it be fine to have a relationship and a column with the continent id on Game.ContinentChapterStage?

storm quest May 12, 2020, 8:20 AM

#

@toxic rune It is fine to have FK to both the tables if that'd result in any performance improvement. But this is usually considered a bad design, given that you can get the Continent from ContinentChapter

toxic rune May 12, 2020, 10:42 AM

#

Ah I see that makes sense thanks

storm condor May 12, 2020, 11:22 AM

#

did the sqlite can be run as cluster scheme?

pale crest May 12, 2020, 12:41 PM

#

@chilly creek Where do you get the data from?

chilly creek May 12, 2020, 12:50 PM

#

@pale crest REST GET an api

pale crest May 12, 2020, 12:50 PM

#

And it sends you a 15GB response?

chilly creek May 12, 2020, 12:50 PM

#

no

#

1 answer is about 3k

pale crest May 12, 2020, 12:50 PM

#

So it's paged?

chilly creek May 12, 2020, 12:50 PM

#

yep

pale crest May 12, 2020, 12:50 PM

#

Ok, and what are you gonna do with it?

chilly creek May 12, 2020, 12:50 PM

#

indexing till 5 million

pale crest May 12, 2020, 12:51 PM

#

The data, I mean

chilly creek May 12, 2020, 12:52 PM

#

one key of the json is a string of tags, seperated by a space
I want to be able to get all indexes with a specific tag

#

some tags have up to 500k entries

pale crest May 12, 2020, 12:53 PM

#

What do you mean by "indexes"?

chilly creek May 12, 2020, 12:53 PM

#

and I want to not completely overload the api (which is also limited to 50 results/page)

#

so that I get back 1, 45, 32, ...

#

or whatever I want, the json of all found entries

#databases

run my check on one of the attributes

use other attributes if check passes