#databases

1 messages · Page 67 of 1

void flower
#

and not the migration part

#

The system checks are being run during migrations and that's why they're failing

pure cypress
#

yup true

void flower
pure cypress
#

@void flower I fixed it by creating a virtualenv in the container

#

Don't know why that fixes it though

#

wait what the fuck

#

no it didnät

#

I got the check to pass...

void flower
#

lol

#

Yeah.

#

Within the container it will migrate and work fine

#

The docker image build is where it blows up

#

I tried using docker inspect to look into the runtime more of the container vs the image

#

but I have no idea what I'm looking at

pure cypress
#

Something even weirder happened. I installed the venv but didn't activate it properly yet somehow the check passed. But then switching back to makemigration failed

void flower
#

😩

#

I've been reading a bit on this

#

And my understanding is that virtual env configurations don't persist between layers

pure cypress
#

Well probably activation wouldn't

#

I'm trying RUN /bin/bash -c "source .venv/bin/activate && pip3 install -r requirements.txt && pip3 install -r requirements.txt"

void flower
#

ah yes

#

I have tried that too with no success

pure cypress
#

Ok so get this

#

I did that

#

then

#

docker run -it imagename /bin/bash

#

So now I activate the venv

#

I run the check and it fails

void flower
#

🤔

pure cypress
#

I get E300 though

#

no E307

void flower
#

Yeah

#

It's hit or miss which ones are provided

#

Sometimes it's both

#

Sometimes it's one

#

It's never neither 😩

pure cypress
#

where the hell are the migrations even going?

#

do they never get created?

#

cause of the check error?

void flower
#

That is my understanding

#

it short circuits

#

/opt/piano_gym_back_end/piano_gym_api/migrations is where they typically end up

#

If I hop in the container and do the migrations, which will work, that is where they would generate

pure cypress
#

Well I managed to break migrations within the container too

#

by using a venv 🤷

void flower
#

We're a power combination

pure cypress
#

That may be a good thing

#

I was curious what this will return

#

AppConfig.get_models()

#

Yeah, LearnerEnrolledSchool and Course legitimately do not show up in the models list

void flower
#

That's so crazy

#

because when I do migrations it shows up as a created model locally

pure cypress
#

Do you know how Django is even discovering your models?

void flower
#

no

pure cypress
#

It says it just attempts to import the "models" submodule of each app

#

And all you have is an __init__ which imports user and usermanager

#

Maybe you just need to import them there?

void flower
#

Sorry, I'm a little slow on the models discovery part you're talking about

void flower
#

Are you talking about in the path piano_gym_back_end/piano_gym_api/__init__.py

pure cypress
#

/models/__init__.py

void flower
#

I see

#

This is a bit of a tangent question

#

I moved my models down into the versions directory

#

and kept the models directory because I had to for the sake of django's default user configurations

#

If I'm supposed to provide them in the models/__init__.py but they live in the versions/v1/* does that mean I should try versions/v1/__init__.py for discovery?

#

I am reading the link you sent right now

pure cypress
#

I don't think so since you never specified "versions" as an app

#

I

#

am not well versed in how it works, that's just my understanding of it

#

I have no clue how any of your models are discovered

#

Doesn't make sense based on what I am reading

void flower
#

still reading

pure cypress
#

So I fixed it in my venv-ised docker image that was confirmed broken before

#

All I did was import the two models into the piano_gym_api/models/__init__.py file

void flower
#

That....

#

is maddening

pure cypress
#

But that feels like kind of a hack; you should probably be importing all your models

void flower
#

Can you paste the import you used?

pure cypress
#

or just specify the app as the version.v1 thing but idk if that'd break other shit

#
from piano_gym_api.versions.v1.models.learner_enrolled_school_model import *
from piano_gym_api.versions.v1.models.learner_enrolled_course_model import *
void flower
#

When you say specify the app

#

What does that look like

pure cypress
#

piano_gym_api.versions.v1 I guess

#

in INSTALLED_APPS

void flower
#

ah

#

From your experience does my models hiearchy look silly?

#

Am I doing something crazy that I'm not aware of?

pure cypress
#

Well to be honest I don't have much django experience

void flower
#

lol

#

I am definitely not a django expert

pure cypress
#

I just did some PRs for some back end DRF stuff once

void flower
#

You sound like the holiday inn commercial lol

pure cypress
#

If you want to get an idea for hierarchy you should take inspiration from open-source django projects

void flower
#

good idea

#

I don't know of any prominent ones, but I'll search it out

pure cypress
#

Well our own site is one

void flower
#

For clarity

#

Your suggested solution works in building the image

#

Also for clarity

pure cypress
void flower
#

I am dying inside

#

lol

pure cypress
#

So all you did to fix was add those two imports?

void flower
#

Yes

#

And the build is working

#

The tests are running

#

And everything is gravy

pure cypress
#

Great but like I said don't rely on it

void flower
#

agreed

pure cypress
#

I think it's best to import all models

#

Right now I am guessing the rest of the models weaselled their way through in some imports somewhere else

void flower
#

Bleh

#

I thought I had solved this when I did the initial configuration of the project

#

clearly I did not

#

@pure cypress I cannot articulate enough how much I appreciate you taking the time to sit down and work through this with me

#

Is there anything I can do to show that gratitude?

pure cypress
#

You're welcome

void flower
#

Also I'll be removing you from the gitlab project 😉

pure cypress
#

Just a thanks is enough so don't worry

void flower
#

Well thank you, and thank you, and thank you!

tropic folio
#

is there an sql help discord?

void otter
#

its this one

gilded narwhal
#
Exception in thread Thread-5:
Traceback (most recent call last):
  File "C:\Users\admin\AppData\Local\Programs\Python\Python37\lib\threading.py", line 917, in _bootstrap_inner
    self.run()
  File "C:\Users\admin\AppData\Local\Programs\Python\Python37\lib\threading.py", line 865, in run
    self._target(*self._args, **self._kwargs)
  File "C:/Users/admin/Documents/Projects/sqlite-file-index/test_2.py", line 27, in add_items
    (f'data {i}',)
  File "C:\Users\admin\Documents\Projects\sqlite-file-index\sqlite_file_index\threadsafe_db.py", line 56, in execute
    retry_while_locked(cursor.execute, query, params)
  File "C:\Users\admin\Documents\Projects\sqlite-file-index\sqlite_file_index\retry.py", line 26, in __call__
    return callee(*args, **kwargs)
sqlite3.DatabaseError: not an error
#

wat

#

schroedinger's error

hazy crystal
#

Guys with sqlalchemy is there any way to have two linked columns from different table.
For example If I insert something on table1 I want table2 colx to use the data from table1 colx

#

wait. This is the same as relationship. Got confused 😄

pure scroll
#

What is the actual issue you want to solve with this?

torn sphinx
#

@void otter late late reply, never used docker containers and the likes, first time trying a database, so I want it simple. SQLite but with json essentially.

#

is there an alternative?

void otter
#

for an sqlite with json?

#

mongoDB

#

its json postgres pretty much

#

you can run your local mongo server and interact with it through docker networs

#

idk if python has ORM for neDB

#

its same as sqlite, db is in a filesystem

#

and json

#

@torn sphinx

torn sphinx
#

hmm

#

but I need to figure out docker containers and stuff

#

if I use mongo

void otter
#

yes

torn sphinx
#

is there an alternative?

void otter
#

neDB but idk if it can run with python

#

i know it can with js

#

google it really

#

3 of big databases with python are sqlite, mysql/postgres and mongo

torn sphinx
#

ok

dull scarab
#

Any pointers as to how i can install rethinkdb on manjaro? I've tried most of the alternatives on their install guide but it all fails to build

pliant oxide
#

Someone know SQLAlchemy? I asked in #help-apple, please ping me if you know it

pure cypress
#

@dull scarab Don't use Manjaro but post errors and what guides you've followed maybe I can help

dull scarab
#

2 sec, lemme boot up

#

is a chunk of the last logs

#

using the package manager

#

I tried building manually and through pacman, but it all failed while building

pure cypress
#

does manjaro support AUR?

dull scarab
#

It should?

pure cypress
#

Did you try installing from there?

dull scarab
#

this is the one i tried

#

All others more or less fail instantly

pure cypress
#

Do you have the build log?

dull scarab
pure cypress
#

Someone also left a comment on AUR FYI, package fails to build with boost 1.69.0-1, but successfully builds with 1.68.0-2.

#

And given the issue does seem to be related to boost, that may be a good lead

dull scarab
#

Any ideas how i specify version?

pure cypress
#

No actually. I've never had to do that before

#

It may be especially weird to do since this is a dependency and not directly the package you want to install

#

Cause normally with arch you upgrade everything whenever you wanna install a new package

dull scarab
#

would i run that before trying to build then?

pure cypress
#

Yeah, before installing rethink

dull scarab
#

Let's have a go at it then

#

Takes forever to build ...

pure cypress
#

oof yeah I somehow ended up building qt4 and that took like 20 minutes fuck me

dull scarab
#

Yeah this is suuuper slow

#

Theres even a comment in the buildfiles saying "this makes it super slow" or something

dull scarab
#

@pure cypress Holy shit, i believe it worked. Only 1 hour and 20 minutes. Thanks alot dabward

pure cypress
#

wow that is really fucking long

#

np mate

dull scarab
#

Very long

#

I feel rejuvenated not having to sit through that again

pure cypress
#

Being that long, I would have felt bad if it didn't work

dull scarab
#

Failed it like 3-4 times so far

#

reaching like 200/400 or something

gleaming frost
#

@dull scarab Use docker?

#

If your desperate to get it up and running

#

Also good to know I'm not the only one still using it! haha

dull scarab
#

¯_(ツ)_/¯

#

It seemed like a good fit for a project im starting with the university this summer

gleaming frost
#

Why not MongoDB?

#

I mean: Rethinkdb support is slowing dying

dull scarab
#

not quite sure

#

Just figured it was a bit more "scale-y"

gleaming frost
#

I like the changes feature or rethinkcb

#

a few lines of code, and you have full database logging

dull scarab
#

but tbh, I don't know the best fit for out project, its a first draft of the system structure so figured we'd give it a go

gleaming frost
#

Use it in docker, it's all I use.

#

The base-installers are very sketchy

dull scarab
#

I don't know docker yet grumpchib

gleaming frost
#

I could never get them to work

#

Ouch.

#

Lemme go get you the single command install if you want?

dull scarab
#

We seemed to get it installed on all 3 machines, mac, win and manjaro

gleaming frost
#

windows it's easy

pure cypress
#

I don't thionk he wants to wait for it to build again 😄

gleaming frost
#

Hahaha

dull scarab
pure cypress
#

Or can yo install it without building from source in docker?

#

Is there a pebuilt binary somewhere?

gleaming frost
#

Yep

#

There's an official docker image

#

Just download docker and run....

pure cypress
#

Oh of course

#

Didn't think of that

gleaming frost
#

sudo docker run -p 28015:28015 -d -v chat:/data rethinkdb:latest rethinkdb -d /data --initial-password password --bind all

#

This will expose port 28015, and stop data getting deleted when your server restarts

#

And boom it's running!

pure cypress
#

Don't you need a volume set up for that?

#

That looks like a named volume

gleaming frost
#

Nope.

pure cypress
#

not a bind mount

gleaming frost
#

There's a command in there

#

That's a docker-volume

#

It keeps the data

#

or at least I think it does

#

Right?

pure cypress
#

Yes that's what volumes do

#

I just thought for named volumes you had to create the volume beforehand

gleaming frost
#

Nope, it's automatic

pure cypress
#

Which isn't the case for bind-mounts for like a local directory

gleaming frost
#

:D

pure cypress
#

oh ok 🤷

gleaming frost
#

It creates the volume for you.

#

Or at least, in the latest docker version

dull scarab
#

Might consider that if we have problems deploying, thanks sharp

pliant pendant
#

anyone here who could assist me

hazy mango
#

!ask

delicate fieldBOT
#
ask

Asking good questions will yield a much higher chance of a quick response:

• Don't ask to ask your question, just go ahead and tell us your problem.
• Try to solve the problem on your own first, we're not going to write code for you.
• Show us the code you've tried and any errors or unexpected results it's giving
• Keep your patience while we're helping you.

You can find a much more detailed explanation on our website.

lime whale
#

select ID, FIRST_NAME, LAST_NAME from CUSTOMER where (length(FIRST_NAME) + length(LAST_NAME)) < 12
order by length(FIRST_NAME) + length(LAST_NAME), (FIRST_NAME + LAST_NAME) asc, ID asc;

#

how do i get this right? sorting by full name?

torn sphinx
#

can anybody explain the term s3 compliant used commonly these days?

#

does it mean it's using s3 as the backend storage only? or is it something that complies with communication based on s3 like api?

charred thorn
#

Has anyone used SQL Machine Learning Services, specifically writing Python inside of it? I’m trying to set up a machine learning model and serialize it in a database but I’m having trouble getting the code sorted out.

Without going into too much detail I’m trying to import a CSV file with the training data and features (3 columns total of 5000 rows). I used pandas.read_csv to import the file to a data frame, would putting the file into the database make the path a bit shorter?

torn sphinx
#

@charred thorn what do you mean shorter path? isn't pandas.read_csv already the simplest read op you can do?

#

but it sounds like SQL service lets you execute python code at DB side so you might not need pandas at all? they provide package..

charred thorn
#

I meant like the path to the CSV file. I might ask my mentor for info on how accessing files from SQL works tomorrow

torn sphinx
#

i mean you can do direct pandas to SQL call

#

but if they provide package, pretty sure there'd be io related ones

charred thorn
#

Got it, I'll give that a shot when I get back to work on it tomorrow

torn sphinx
#

what's the benefit of this way of running btw? just to save data travel time?

#

interesting service

charred thorn
#

Ideally the model would be stored in the database for future access. And like you said it would cut down on data travel time.

I don't think there's any major benefits that come to mind, but it's a pretty new tool at the place I'm interning at so successfully implementing it in a project as an intern would be pretty impressive :) @torn sphinx

torn sphinx
#

what is 'model' that's stored in db like?

charred thorn
#

it would be an XGBoost binary classification model to take a few inputs and determine if the output is "true" or "false" based on them

torn sphinx
#

oh i mean, in what data form are you storing it? in storing in regular SQL table?

charred thorn
#

It would be dumped using the pickle library and serialized, then stored as a value in a regular SQL table until it's needed afaik

torn sphinx
#

so as binary blob like then

#

interesting use of relational table.

#

so you'd have model parsing done on app layer

pliant oxide
#

Can someone give me a general knowledge about creating views in MySQL?
I got offers table:

#

I got flights table:

#

How to "join" those two, for example to show:
outbound_flight_flightNumber
inbound_flight_from_aiport
inbound_flight_from_date (with some function which would parse unix timestamp to human readable)

#

I want a general concept, then I would follow to add more informations. That's why I'm asking about 3 cases.

#

Airports table (would be need):

wintry aspen
#

Hello, quick question, what is the difference between pgadmin 4 python wheel and pgadmin for macOS ?

#

I don't see why python is compare to an OS

#

oh I see, that is because it's run as a web app for the version 4

pliant oxide
#

In relation with last question:

CREATE 
    ALGORITHM = UNDEFINED 
    DEFINER = `root`@`%` 
    SQL SECURITY DEFINER
VIEW `offers_viewer` AS
    SELECT 
        `offers`.`price` AS `price`,
        `offers`.`outbound_flight_id` AS `outbound_flight_id`,
        `out_flights`.`flight_number` AS `outbound_flight_number`,
        `offers`.`inbound_flight_id` AS `inbound_flight_id`,
        `in_flights`.`flight_number` AS `inbound_flight_number`
    FROM
        ((`offers`
        JOIN `flights` `out_flights` ON ((`offers`.`outbound_flight_id` = `out_flights`.`id`))
        JOIN `flights` `in_flights` ON ((`offers`.`inbound_flight_id` = `in_flights`.`id`)))
        JOIN `airports` `out_airports` ON (`out_flights`.`from_aiport_id` = `out_airports`.`id`))
#

ERROR 1054: Unknown column 'out_flights.from_aiport_id' in 'on clause'

torn sphinx
#

seems like you've misspelled airport

#

@pliant oxide

#

Does anyone know why blob storage is called so.... I understand it's binary large object storage.. but what's the background/context

torn sphinx
#

WHERE Country = 'Germany' AND 'France'

#

I am getting a syntax error for AND

#

does anyone know why this is wrong?

carmine heart
#

You probably need a complete conditional statement on both sides of the AND

#

WHERE Country = 'Germany' AND Country = 'France'

#

However, can the Country be Germany and France simultaneously?

#

Maybe you want OR

torn sphinx
#

yo

#

it was WHERE Country = 'Germany' OR Country = 'France'

#

cheers mate

prisma swift
#

Hi!

#

can i get a help with pandas?

fallen lava
#

hey 😄

#

I need help for Pandas as well 😃

#

I'm currently using Pandas - and I used this line of code

d = data.groupby(['Category'])['Price'].sum()

(read from a csv file)
how can I then separate the results into 2 different lists ?
if I use d.tolist() it only creates a list of values
but I also need the names
not sure if I did a good job at explaining

light ether
#

When creating models in sqlalchemy based on my classes, if I make a modification to a class --> say I add another column to class after already running the app for the first time...wil sqlalchemy autoupdate the database schemea and add the new column

void otter
#

Nope

#

You have to delete and recreate the database

plucky cloak
#

or you could migrate the class table with something like alembic

wary merlin
#

Hi, I have some issue with mariaDB insert into where not exists

#

Traceback (most recent call last):
  File "C:\Users\DEVEAUX\Desktop\parse adresse.py", line 251, in <module>
    (line["numéro"], rue_id, line["latitude"], line["longitude"], line["numéro"], rue_id),
  File "C:\Users\DEVEAUX\AppData\Local\Programs\Python\Python37\lib\site-packages\mysql\connector\cursor_cext.py", line 266, in execute
    raw_as_string=self._raw_as_string)
  File "C:\Users\DEVEAUX\AppData\Local\Programs\Python\Python37\lib\site-packages\mysql\connector\connection_cext.py", line 398, in cmd_query
    sqlstate=exc.sqlstate)
mysql.connector.errors.ProgrammingError: 1064 (42000): You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near 'WHERE NOT EXISTS(
                                SELECT *
                                FROM `addresse`
                                WHERE `label` = '1'
                                A' at line 3```
#
        cursor = conn.cursor()
        cursor.execute(
            """
            INSERT INTO `adresse`(`label`, `id_rue`, `latitude`, `longitude`)
            VALUES(%s, %s, %s, %s)
            WHERE NOT EXISTS(
                SELECT *
                FROM `addresse`
                WHERE `label` = %s
                AND `id_rue` = %s
            )
            LIMIT 1;
            """,
            (line["numéro"], rue_id, line["latitude"], line["longitude"], line["numéro"], rue_id),
        )
        cursor.close()```
#

the where not exists is for prevent duplicate

#
+-----------+--------------+------+-----+---------+----------------+
| Field     | Type         | Null | Key | Default | Extra          |
+-----------+--------------+------+-----+---------+----------------+
| id        | int(11)      | NO   | PRI | NULL    | auto_increment |
| label     | varchar(255) | YES  | MUL | NULL    |                |
| id_rue    | int(11)      | NO   | MUL | NULL    |                |
| latitude  | float        | YES  |     | NULL    |                |
| longitude | float        | YES  |     | NULL    |                |
+-----------+--------------+------+-----+---------+----------------+
5 rows in set (0.01 sec)```
#

and adresse table is created with sql CREATE TABLE IF NOT EXISTS `adresse`( `id` INTEGER PRIMARY KEY NOT NULL AUTO_INCREMENT, `label` VARCHAR(255), `id_rue` INTEGER NOT NULL, `latitude` FLOAT, `longitude` FLOAT, INDEX (`label`, `id_rue`), CONSTRAINT FK_address_id FOREIGN KEY (id_rue) REFERENCES rue(id) );

torn sphinx
#

Having a table with id,date time,number value
is it possible to group the values where the date is similar? (because it's like "2019-06-10 12:00")
to select them and have the total per day as result.*
or would it be simpler to do this through python?

dire dragon
#

why would I not be able to put an int into a blob field of a sqlite database? Aren't you supposed to be able to put anything in there?

dire dragon
#

I would just use an INT field but it's a discord user ID so I need 64 bit unsigned, not signed

#

This statement complains about the paramaters being of unsupported type

SELECT * FROM Users WHERE ID = ?;
#

and then ? is being swapped out with the ID of a discord user, which is int

#

also happens if the column is UNSIGNED BIG INT

pure cypress
#

Are you sure you put the ID in a tuple?

dire dragon
#

why would I do that?

#

do I need to?

#

no I didn't because I didn't know I had to

pure cypress
#

Yeah, for parameter substitution it needs tuples

dire dragon
#

oh I was passing them as seperate paramaters

#

so the ID needs to be in its own tuple or just the whole thing?

pure cypress
#

Like this: py execute("SELECT * FROM Users WHERE ID = ?", (user_id,))

dire dragon
#

but if I'm passing multiple things would it be like (foo, (user_id)) or (foo, user_id)?

pure cypress
#

the latter

dire dragon
#

alright

#

the line is this and it still says unsupported type

c.execute("SELECT * FROM Users WHERE ID = ?;", (ID))
pure cypress
#

The comma in the tuple is important

dire dragon
#

sorry, I've never done anything with sqlite or databases in general before

#

oh

pure cypress
#

(ID) is not a tuple. (ID,) is a tuple

dire dragon
#

I saw that ones with multiple values didn't need it so I figured single values were the same

pure cypress
#

Tuples with multiple elements don't need a trailing comma cause it isn't ambiguous whether it's a tuple

dire dragon
#

well that part is working, thanks

hexed wren
#

hello guys i new at programing where should i learn it

wide jolt
#

!resources is a good place to start

delicate fieldBOT
#
Resources

The Resources page on our website contains a list of hand-selected goodies that we regularly recommend to both beginners and experts.

quiet ermine
#

Does kubernetes support distributing an sql or nosql database over the entire network

#

So you have 5 database pods and each one has ~1/5th of the database in a special docker volume (not inluding backups)

#

I think this would be something like redis but hdd based? Never looked into redis

indigo mason
#

Things like Postgres support connection pools so everything isn't on one connection

#

Or well, that's up for the lib to implement

#

Idk about psycopg2 but asyncpg does it well

#

If you're worried about database performance, it's better to look at the database first before things like that

#

Postgres will always outperform SQLite because SQLite is file based

#

But of course this difference only matters at scale

#

And is there anywhere you could save yourself a query or cache results etc.

frozen osprey
#

Hey. When I try to put sth in db I got this error (I'm on windows).

raise CommandInvokeError(exc) from exc
discord.ext.commands.errors.CommandInvokeError: Command raised an exception: ServerSelectionTimeoutError: connection closed,connection closed,connection closed
#

And how to reboot mongodb server?

wintry aspen
#

Hello, anyone knows where may I find, a WebSite to download free data file, to play with ?

buoyant breach
#

Depends on what format data file are you interested in. @wintry aspen

wintry aspen
#

I'm playing with SQL, so I think that would be .tar or .db, If I'm not wrong

buoyant breach
#

SQL is a general purpose query language. It does not refer to any particular database storage format or engine.

#

It can be used with a variety of technologies.

#

Yet if you want something simple to start with - try using sqlite.

#

It should be pretty straightforward to make a file to play with yourself.

wintry aspen
#

Yep, I'm using it with python

#

and I'm looking for free data to play with

buoyant breach
wintry aspen
#

thank you !

buoyant breach
#

Welcome. It's just one google away. =]

wintry aspen
#

When you work on data and SQL, what is the best way to display it on chart ?

buoyant breach
#

You mean some kind of visual aggregation?

frozen osprey
#

Hey. How to use mongodb?

buoyant breach
#

You may want to express your question a but more detailed.

#

What exactly are you struggling with? Connection to the DB? Running queries?

wintry aspen
#

@buoyant breach Right, visual aggregation 😃

buoyant breach
#

@wintry aspen scipy is what comes to mind immediately. But it may be overkill, there must be some more straightforward libs for that.

quiet ether
#

@noble pelican you are using a cursor, you need to just do a normal select or bulk collect of some sort if you are fetching large amounts of records

#

if it;s indexed properly 12000 should be nothing for mysql to handle

#

check the db execution plan as well

wintry aspen
#

Thank you for that @buoyant breach . Do you think I can use QT chart as well?

buoyant breach
#

I did not wark with those all that much, so there is not much I can advice here, sorry.

#

You can try as long as there is python wrapper.

wintry aspen
#

Okay, thank for your return at all 😃

buoyant breach
frozen osprey
#

Hey. When I use a cmd to add data to MongoDB it shows this error. What might be wrong?

raise CommandInvokeError(exc) from exc
discord.ext.commands.errors.CommandInvokeError: Command raised an exception: ServerSelectionTimeoutError: localhost:27017: [WinError 10061] The ability to connect, the target computer actively refuses it

I'm using atlas version on windows

atomic plank
#

Hi i'm using asyncpg with postgresql what my question is how can I query the text from a column? like say (this that) and not <Record = etc everytime I try i get an error that it's an unserializable json etc

#

{o.class.name} '
TypeError: Object of type Record is not JSON serializable

#

what i'm trying to do is make welcome and leave messages customizable

indigo mason
#

A record is only a container for what you want

#

There's docs that'll help you extract what you need from it

#

Also, there are shortcut functions that bypass Record in situations

#

You might want fetchval

#

Or fetchrow

atomic plank
#

ah ty! I'll try ose and will check out some other docs the doc i was reading didn't have much as to specific things I was looking for ty 😃

onyx seal
#

Hey guys. Im using asyncpg for my mod bot and im just wondering: what error does the lib return when an entry is submitted that already exists and thus is a duplicate primary key?

#

Been searching but can’t find much info about it on their documentation or github source

#

Disregard above. I ran a test and found it’s uniqueviolationerror

atomic plank
#

Hey question what would be the best way to format said string in python for postgresql? like say if I put "Welcome {member} to {guild}" how could I set it to format the member and guild? I'm unsure as to if i could just use the f/format() method on it as this is a public thing and many could use it so i'm unsure about what to do exactly

#

like as in the return value not when inserting to postgres

wintry aspen
#

Hello, I don't see what is exactly the difference between 'serial' and 'integer', someone could give me a quick explanation ? 😃

#

It is about SQL by the way ^^

quiet ether
#

serial is just autoincrement for postgres. SQL Server would be "autoincrement"

#

upon each insert it autoincrements by 1 for the ID

#

just a sysid really

#

if you have good normalization it is what you would normally use as an ID for your records

#

otherwise youd have to use some type of GUID

#

or multi-key identifier

feral falcon
#

I will be doing an analysis of imdb database actor and their collaboration with other actors for a school project. I'm interested in showing how some actor's connections changed over the years and for example how different it was comparing an actor from 60' or 70' to some actor today. Anyone know if something similar to this has been done before? I tried searching a bit but had no luck.

quiet ermine
#

Is redis good for kubernetes or should you use something like mongodb or postgres for databases

pulsar timber
#

When running https://hastebin.com/sufekixuhu.sql this code, this error shows:

  File "C:\Users\markc\Anaconda3\envs\toast\lib\site-packages\asyncpg\pool.py", line 558, in fetchrow
    return await con.fetchrow(query, *args, timeout=timeout)
  File "C:\Users\markc\Anaconda3\envs\toast\lib\site-packages\asyncpg\connection.py", line 455, in fetchrow
    data = await self._execute(query, args, 1, timeout)
  File "C:\Users\markc\Anaconda3\envs\toast\lib\site-packages\asyncpg\connection.py", line 1414, in _execute
    query, args, limit, timeout, return_status=return_status)
  File "C:\Users\markc\Anaconda3\envs\toast\lib\site-packages\asyncpg\connection.py", line 1422, in __execute
    return await self._do_execute(query, executor, timeout)
  File "C:\Users\markc\Anaconda3\envs\toast\lib\site-packages\asyncpg\connection.py", line 1434, in _do_execute
    stmt = await self._get_statement(query, None)
  File "C:\Users\markc\Anaconda3\envs\toast\lib\site-packages\asyncpg\connection.py", line 329, in _get_statement
    statement = await self._protocol.prepare(stmt_name, query, timeout)
  File "asyncpg\protocol\protocol.pyx", line 163, in prepare
asyncpg.exceptions.PostgresSyntaxError: syntax error at or near ","```
quiet ether
#

pretty sure that probably expects a tuple or some type of dict, not comma separated @pulsar timber

#

@quiet ermine Kubernetes is a way to have distributed containers, whether you use Redis or Postgres is up to you, usually you will have some type of compose file (for ex. maybe docker compose, then whatever database or database(s) your application uses you would define these in your startup file to be connected)

#

Redis is usually more for temporary type storage for user type information or information that is lightweight, whereas Postgres is a full RDBMS

quiet ermine
#

More talking about horozontal scalability

#

I think i'll go with nosql like mongo

#

& use redis if I ever need it in the future; maybe something for ml

open hill
#

Is it possible to connect to a database that's on hosting server I bought if I'm programming on local machine?

quiet ermine
#

Yes

#

Instead of connecting through it on localhost, just portforward your postgres port on the database's server (I think in your case the vps/hosted one) and use the [ip]:port instead of [localhost]:port

#

@open hill

open hill
#

@quiet ermine I tried but I'm getting ER_ACCESS_DENIED_ERROR. I used password I set

dawn pulsar
#

I'm using a MariaDB and mysql.connector, anyway I can get it to return a list of items, rather then a list of single item tuples?

#

Please ping me

onyx seal
#

@dawn pulsar you’re probably better off returning the results as a dictionary as mysql.connector only has support to return a tuple, namedtuple or dict for queries afaik 😄

dawn pulsar
#

Why don't they add the ability to return a list?

void otter
#

@atomic plank .format() would be an easier option

cloud valley
#

@void otter could you help me ?

void otter
#

Im on bus

#

But i can try

#

What's up

cloud valley
#

Im doing a space invaders game and I need to know how to get new levels and how to do enemies shooting @void otter

void otter
#

Including db?

cloud valley
#

sorry but whats db

void otter
#

Database

cloud valley
#

uhh i dont think i need that

void otter
#

Then i think you should ask in some #help channel

cloud valley
#

OH ok

void otter
atomic plank
#

ty Saki and I figured out what the issue was had to have a sort of array with things like {"user": author.name} and the issue i was having was if a user did anything other than what was in the array it would result in a key error I found a way around that by using .format_map(defaultdict(str, **items))

#

just for others who have a similar question or issue ^

velvet escarp
#

I’m just gonna bump this into here as well since it’s db related: I am using curl to write json values to mongodb via bottle and everything I write returns the key:value as key:null, does anyone have any suggestions on where I’ve gone wrong https://paste.pythondiscord.com/izajebiqeb.py

torn sphinx
#

anyone?

quiet ermine
#

How would you distribute a database like mongodb over a kubernetes cluster

#

So if a disk dies you 1. have a backup somewhere else and 2. it makes a container somewhere else and gets backups sent to it

#

So your entire database is distributed across the entire cluster

ancient warren
#

Can i ask a quick question? I am doing some database work with Python. It is actually strictly more of a MySQL question rather than a Python question. Can i ask anyway?

tender hollow
#

Yes of course

ancient warren
#

I am trying to load data from a .csv file into my database.

Lets say my database has the fields name, surname, dob.
My csv is structured like this: Donald, Trump, 14/6/2019.

#

Now some of my csv rows look like this : Barack, Obama,

#

Or even Barack,,

#

So missing some values. When i try to load this .csv i get an error that these rows have different date values for those fields for example.

#

How can i fix this?

#

Also sorry for the horrid example. I am really tired.

tender hollow
#

Uhh, it's a lil unclear what the issue actually is here. Are your fields with missing data corresponding to columns in your table that are marked as NON NULL or something of the sort?

#

Or if you had the precise error you get that might be helpful

ancient warren
#

None of the tables are marked as NON NULL. I thought if i have an empty entry in my .csv like ",," it would just count as NULL. That is what i do not understand.

#

Error Code: 1292. Incorrect datetime value: '' for column 'dateofDeath' at row 18

#

That is my exact error.

#

Thanks for helping.

tender hollow
#

Uhh.. What is the actual line of the CSV at row 18 or whatever that prompts this error?

ancient warren
#

None of the columns are markes as NON NULL is what i meant so say ofc my bad.

#

Wait i will look for the exact line

#

http://www.wikidata.org/entity/Q317232,Carausius,0300-01-01 00:00:00,0293-01-01 00:00:00,carausius
This is the CSV row 18.

#

And i am inserting into this table:

#
    item varchar(50),
    itemLabel varchar(50),
    dateofBirth datetime,
    dateofDeath datetime,
    nomisma varchar(50)
    );```
tender hollow
#

That's.. strange? Because the error seems to imply that at this row there is an empty string passed as a datetime value

ancient warren
#

Oh i got an idea. With LOAD DATA INFILE i am ignoring the csv header.

#

maybe it is actually referring row 19?

tender hollow
#

That is possible

#

What's row 19 like

ancient warren
#

row 19
http://www.wikidata.org/entity/Q61088565,Q61088565,0250-01-01 00:00:00,,

#

ah yea

tender hollow
#

errrrrrrrr

ancient warren
#

I really do not get why it does not just set the Field as NULL though.

#

Or am i a complete moron?

tender hollow
#

I don't get it either (although I have no experience importing CSV to MySQL)

#

I guess you could just try explicitly setting the missing ones to NULL? Though that might require a bit of work.

ancient warren
#

Even if i change the row to have some artificial datetime and varchar i get the same error.

tender hollow
#

I'm quickly googling it but not finding much

#

Uhhh

ancient warren
#

Setting it to NULL also did not do anything.

tender hollow
#

Can you perhaps copy paste that row and also the couple rows before/after?

ancient warren
#

I will in a second. I just deleted the row but it gives the same error when the missing datetime occurs again.

#

So it is also not some weird character chain that is getting misinterpreted

#

Row 17 through 21:

#
http://www.wikidata.org/entity/Q317232,Carausius,0300-01-01 00:00:00,0293-01-01 00:00:00,carausius
http://www.wikidata.org/entity/Q61088565,Q61088565,0250-01-01 00:00:00,,
http://www.wikidata.org/entity/Q885870,Victorinus II.,0300-01-01 00:00:00,0271-01-01 00:00:00,
http://www.wikidata.org/entity/Q550622,Maximus,0400-01-01 00:00:00,0422-01-01 00:00:00,maximus_barcelona```
tender hollow
#

I'm currently reading up on something about strict mode

#

What version of Mysql are you using?

ancient warren
#

ah fuck

#

need to look

#

how i even check that

tender hollow
#

errr mysql -v I assume?

#

haha

#

i lied

#

mysql --version

#

Anyways, if you're running 5.7, they added something that makes the rules of the database a lot stricter by default, including something called "NO_ZERO_DATE" which might have something to do with it

#

You can try changing the mode of the database by using the command, in the mysql cli,

mysql> set global SQL_MODE="NO_ENGINE_SUBSTITUTION";

Info: https://www.percona.com/blog/2016/10/18/upgrading-to-mysql-5-7-beware-of-the-new-strict-mode/

ancient warren
#

Sorry took a while: 8.0.16 is my version.

#

😉

tender hollow
#

uhh, what's the number after Distrib

#

anyways, doesn't matte,r it's very likely this is your issue

#

I recommend trying to change the SQL_MODE as above and try again

ancient warren
#

yes thanks i will

tender hollow
#

Let me know if it helped.

ancient warren
#

thanks a lot i will in a sec

#

I am so confused right now.

#

My my.ini file does not have a sql-mode variable

#

and neither can i change it via the command line client

tender hollow
#

err, what happens when you try to do it thru the cli

ancient warren
#

uhhh never used that much

#

i get an error saying it is incorrect sql syntax

#

so i guess i am doing something wrong there from teh getgo?

#

Okay i set the sql-mode to NO_ENGINE_SUBSTITUTION

tender hollow
#

Sorry, I was afk

ancient warren
#

But it didnt fix the issue

tender hollow
#

:/

ancient warren
#

okay great timing actually haha

tender hollow
#

I'm pretty out of ideas regarding this issue lol

ancient warren
#

153 row(s) affected, 2 warning(s): 1265 Data truncated for column 'dateofDeath' at row 18 1265 Data truncated for column 'dateofDeath' at row 44 Records: 153 Deleted: 0 Skipped: 0 Warnings: 2

#

Okay i had to reconnect to the server for the sql_mode change to take effect. My bad.

#

With that i now get the above warning.

#

And the fields are not NULL but rather '0000-00-00 00:00:00'

#

I guess i can work with that. I really for the life of me cant understand why this is so difficult to get around or why it does not work at all. Thanks a bunch for your help though.

#

Really appreciated.

#

If you still care how it can be done differently i found a link @tender hollow

#

First answer is a really nice example.

ruby jackal
#

hey guys, I know this isnt directly python related (yet), but I'm trying to set up mysql for the first time.

I can't seem to figure out how this works, but MySQL isn't allowing me to connect to localhost.

If anyone knows, do I need to change any of my computers host files or anything?

Have been looking online but can't seem to find any resolutions
https://gyazo.com/6614879fa2a5a764990d48f45227ef6d

#

I don't have MySQL running in services or anything either...

tender hollow
#

Well, it would appear that the Mysql server itself isn't running..

#

You shouldn't have to do anything with your host files, 127.0.0.1 should point to localhost already

ruby jackal
#

It doesn't :/

#

I have no idea what to do

#

I just uninstalled and doing a new reinstall

#

hoping it fixes

ruby jackal
#

Alright

#

Got my database working now

#

Getting this error:
ModuleNotFoundError: No module named 'mysql.connector'; 'mysql' is not a package

#

I installed the mysql-connector, got that error
then installed mysql-connector-python-rf

#

ah ignore im blind

ancient warren
#

Can i ask another question? My brain really is not working today!

craggy shore
#

sure just ask 😛

ancient warren
#

My brain is absolutely fried here at 4 am and i am kind of new to SQL. Can you help me out?

#

Table 1: fatherName, dob, etc.

Table 2: fatherName, childName

ancient warren
#

Now it does not make much sense in relation to my example above. But lets assume that two different fathers can have the same child.

#

How could i SELECT the name of the fathers and the name of their "common" child?

#

Okay i rephrased my question. It is much more clearer now (i guess!).

pure cypress
#

I think you set up your schema poorly

#

In the second table, the child should be the primary key

#

and it should have a column for each parent

#

cause each person has 2 parents right 🤔

#

Well that's actually an interesting case, if it's possible to have more than 2 parents (obviously not biologically)

#

Not sure how you'd account for that...

#

A parent can have multiple children too, so this would actually be a many-many relationship I believe

#

In that case you could use what's typically called a "junction table"

torn sphinx
#

can anyone help me with this?
Command raised an exception: TypeError: 'asyncpg.Record' object is not callable

#

@pure cypress?

pure cypress
#

Please don't ping helpers arbitrarily. I am not familiar with asyncpg but I could try to help if you provided more details

#

Post the full traceback for the error

#

and include the relevant bits of your code

ancient warren
#

Alright @pure cypress i have no controll over how my schema is set up though. That is why i had the question on how to do this in the first place. I guess it is some kind of many to many realtion yes!

#

Thanks anyway!

#

It is not actually parents though in my database that was just an example i used previously.

#

a poor example i admit

quiet ermine
knotty parcel
#
 @welcome.command()
  async def channel(self, ctx, channel:discord.TextChannel):
    if ctx.message.author.guild_permissions.manage_messages or ctx.message.author.id == (553614184735047712) :
      global client
      db = client.bot
      channelid = channel.id
      posts = db.serversettings
      for x in posts.find({"serverid":f"{ctx.guild.id}"},{ "_id": 0, "serverid": 0, "leveling": 0,"swearing":0,"automod":0,"economy":0 ,"welcome":0,"prefix":0}):
        achannel = x["wchannel"]
        print(achannel," ",x)
      myquery = { "wchannel": f"{achannel}" }
      newvalues = { "$set": { "wchannel": f"{channelid}" } }
      print(achannel)
      a = posts.update_one(myquery, newvalues)
      print(a,"\n",channelid)
      await ctx.send(f"Channel has been set to {channel.mention}")
#

Doesn't save to the database

#

this is mongodb

knotty parcel
#

@plain radish

carmine heart
#

@knotty parcel please don't mention and remove multiple (staff) members until your question gets some attention.

knotty parcel
#

Okay

#

I got attention

#

Is there a way to fix it?

carmine heart
#

I'm not available right now.

ruby jackal
#

Anyone know a solid tutorial online for discord + mysql?

ancient warren
#

I think you have more luck asking that in the discord-py channel. But i can not help you myself.

copper echo
#

is it possible to delete everything without 1 thing?

#

something like DELETE * FROM guild_settings WITHOUT lang

copper echo
#

ping me if you can help me

#

DELETE FROM guild_settings WHERE guild_id = $1 AND row_name != 'lang' maybe something like that but how

atomic plank
#

@copper echo I don't technically understand what you mean are you referring to deleting just certain items or the whole entry?

copper echo
#

i want to delete whole data from table without one column

atomic plank
#

Okay so I don't necessarily know how to do delete everything from one system but leave 1 column

#

but you "could" try

#

DELETE x FROM table where guild_id = $1

#

Similar to regular selecting but change X to what you want to delete

#

not tested this and i'm just getting into sql so i don't know if that would work

#

or optional just use DELETE FROM table WHERE guild_id = $1

#

which would delete everything including the column you wish to save

graceful nimbus
#

How would I create a table using a variable to name it? Can't seem to find any information online. (sqlite3)

copper echo
#

delete from table is deleteing everyting from it

#

delete x from is not working for me idk why

#

i meazn

#

i tried DELETE something, something1, something2 FROM table

pure cypress
#

You want to delete certain columns for certain rows?

#

If I understand correctly then this doesn't seem possible cause all rows must have the same columns

#

You could set the columns to null but you couldn't just straight up delete them selectively

#

You could also drop a column but that would affect the entire table

copper echo
#

i want to delete all columns except one - "lang"

pure cypress
#

You want to delete them for the entire table?

velvet flume
#

Any online db

#

where I can hit a URL and the params get saved

#

so database saves Vraj

#

I think firebase can do it

#

any ideas

copper echo
#

yes Mark

hearty iron
#

my table:

CREATE TABLE IF NOT EXISTS players (
  user_id BIGINT PRIMARY KEY,
  ign VARCHAR(32) UNIQUE NOT NULL
)```

now when i try to insert a row(i.e. add a player), i want to know if the player already exists or not.
What i am doing rn is:
```sql
INSERT INTO players (user_id, ign)
VALUES
($1, $2)
ON CONFLICT (user_id, ign)
DO NOTHING```
what would be the best way here to know on what column the conflict was(was it that the player already is registered or the ign is taken by another user) or if there was no conflict and everything went on smoothly
#

using PostgreSQL + asyncpg

knotty parcel
#

@carmine heart Are you available

carmine heart
#

Just ask your question and someone may help you. No need to ping specific people.

knotty parcel
#

In pymongo How do I get a request amount so like see how many times that query pops up and a detail from every query

unborn sentinel
#

What have you tried

knotty parcel
#

@unborn sentinel I haven't tried anything since I'm not even sure what to do the closest is

for x in posts.find({"guildid":f"{ctx.guild.id}","_id":f"{user.id}"},{ "_id": 0, "serverid": 0}):
      reason[x] = x["reason"]
      issuer[x] = x["issuer"]
      a += 1
    print(f"{reason1}\n{reason2} and {issuer2} in {a}")
unborn sentinel
#

So you haven't tried to run that?

knotty parcel
#

I did syntax error

#

works for you?

unborn sentinel
#

I don't have access to the database, so there's no real way I can test it.

knotty parcel
#

Would you like access?

unborn sentinel
#

Honestly not really. We can try to help you work out the logic, but the main work has to be done by you. We won't write the code for you.

knotty parcel
#

@unborn sentinel So could you help me?

unborn sentinel
#

I currently cannot, as I'm swamped at work. Hopefully someone will be around to help you at some point, but until then just keep trying.

knotty parcel
#

I was hoping you would since I'm at work too

graceful nimbus
#

So I found out how to use a var to name a table:

sql_cmd = '''CREATE TABLE {}(test, test2, test3, test4)'''.format(
       tableName)
    dataBaseConnection.execute(sql_cmd)

But now i'm trying to also name the row names with variables.

rotund cove
#

It may be worth noting if you take user input that executing sql with string formatting could leave you open to some nasty sql injection 😃 just a heads up

knotty parcel
#

@rotund cove Can you sql inject into a non sql database such as mongodb

knotty parcel
#

How do I take things from a database and put them in 1 output:

 @commands.command()
  async def warnings(self,ctx,user:discord.Member):
    global client
    db = client.bot
    posts = db.warnings
    a = 0
    for x in posts.find({"guildid":f"{ctx.guild.id}"},{ "_id": 0, "serverid": 0}):
      reason = x["reason"]
      issuer = x["issuer"]
      print(f"{reason} and {issuer} in {a}")
      a += 1
arctic anvil
#

I could use some advice on using Alembic and organizing code into modules:
I am setting up autogeneration with the sqlalchemy declarative base.

In my_project folder, I used alembic init data so I can keep all my data stuff there.
In there I made a folder called models with a basic.py file inside that.
In that basic.py file I have a simple setup:

from sqlalchemy import Column, Integer, String
from sqlalchemy.ext.declarative import declarative_base

Base = declarative_base()

class SomeClass(Base):
    __tablename__ = 'some_table'
    id = Column(Integer, primary_key=True)
    name = Column(String(50))

So that in alembic's env.py file I can import the Base and assign it’s metadata to the target_metadata value.

However:

When typing my import statement PyCharm wants to autocomplete as if I was in the project directory, even though the env.py file is in the data directory. So I do from data.models import basic but on that alembic fails ModuleNotFoundError: No module named 'data’

If ignore PyCharm and do the seemingly obvious from models import basic alembic fails ModuleNotFoundError: No module named 'models’.

Alembic also fails if I try from my_project.data.models import basic

Adding __init__.py files doesn't help either. I think they are unecessary as of 3.3/4?. I am running on 3.7.1

#
├── my_project
│   └── data
│       ├── models
│         └── basic.py
│       └── env.py
│       └── versions
│       └── ...

well I figured out that since alembic isn't local nothing related to the project is in sys.path so using PYTHONPATH helps, but I think I am going to add what I want to sys.path in the env.py file.... hopefully that's not a bad idea.

torn sphinx
#

I know this isn't really for python but if someone could help, it'd be appreciated

level = await self.bot.pool.fetch("SELECT * FROM leveling WHERE guildid = $1 ORDER BY lvl DESC LIMIT 5", ctx.guild.id)
xp = await self.bot.pool.fetch("SELECT * FROM leveling WHERE guildid = $1  ORDER BY xp DESC LIMIT 5", ctx.guild.id)
``` im trying to get the xp and lvl and have it organized, but i still get it where a person that the same lvl but less xp as number 1
calm knoll
#

Do you advice some tutorials to get started with databases?

cobalt yew
#

SQL on codecademy

#

is super good

meager leaf
#

I'm using postgresql and asyncpg. I have a two row table, one containing an id and the other a string. I need to find the string by id a lot of times. Would it be faster to cache them all into a dict instead of querying the database every time I need to get the string?

echo turret
#

yh it would be efficient if you would query them once and then store them in like a dict for example , at least I think so, or maybe a pd dataframe @meager leaf

ionic pecan
#

define „a lot of times“

meager leaf
#

Several times a minute for different ids

harsh pulsar
#

for a two-row table? definitely more efficient to cache

pure scroll
#

two column I guess

#

at least from his description it looks like two column

quasi holly
#

Hello! I'm using PostgreSQL and I want to upload the layout on my database for everyone to use. How would I do that?

meager leaf
#

yeah sorry, column

#

@quasi holly look for software that can reverse engineer the database and export it into sql commands

#

I know pgmodeler can do it, but you have to pay for it or compile it yourself for free

#

theres also pg_dump command if you are familiar with terminals

#

pg_dump dbname > outfile

#

psql dbname < infile

quasi holly
#

I will look into it, thank you!

ionic pecan
#

make a —schema-only dump with pg_dump

torn sphinx
#
level = await self.bot.pool.fetch("SELECT * FROM leveling WHERE guildid = $1 ORDER BY lvl DESC LIMIT 5", ctx.guild.id)
xp = await self.bot.pool.fetch("SELECT * FROM leveling WHERE guildid = $1  ORDER BY xp DESC LIMIT 5", ctx.guild.id)
``` im trying to get the xp and lvl and have it organized, but i still get it where a person that the same lvl but less xp as number 1
cerulean pendant
#

use a single query

#
SELECT * FROM leveling WHERE guildid = $1 ORDER BY lvl DESC, xp DESC LIMIT 5
torn sphinx
#

doesn't work

#

gives me Command raised an exception: TypeError: list indices must be integers or slices, not str

cerulean pendant
#

Seems to be unrelated

#

Show the code and traceback

torn sphinx
#

i used the order by on both lines of the original code and it seemed to work

hazy crystal
#

Guys how do I construct the following scheme with sqlalchemy.

A user can have a relationship to messages but messages will have senderID and reciverID (which will be two different users but still poiting to user messages colume)

#

for example

#
class Message(db.Model):
    __tablename__ = "message"

    id = db.Column('id', db.Integer(), primary_key=True, nullable=False)
    senderID = db.Column(db.Integer(), db.ForeignKey('users.id'))
    reciverID = db.Column(db.Integer(), db.ForeignKey('users.id'))


class User(db.Model, UserMixin):
    __tablename__ = "users"

    id = db.Column('id', db.Integer(), primary_key=True, nullable=False)
    messages = db.relationship('Message', backref='user')
dull scarab
#

How scaleable is mongodb for a webapp backend?

quiet ether
#

@dull scarab as scalable as your code and infrastructure

#

I would argue the data 99% people use do not need to worry about scale but more about how their code is interacting with the database and following best practices outlined by their respective DB

#

unless you are a large government entity or a bank

dull scarab
#

It's for a smaller project for uni so i guess ill be fine for a long time

#

¯_(ツ)_/¯

quiet ether
#

just check the official documentation, also make sure you use "lean()" or serialize your data before retreiving it from your API

#

otherwise it will return the entire document

#

which will cause performance issues

dull scarab
#

Will keep in mind

quiet ether
#

also just look at indexes/etc if you are going to be searching through documents

#

im not a huge mongodb expert, but their docs are pretty solid

gleaming frost
#

Can I just quickly check the output of that statement would be

#

A 6.99
B 2.99

pure cypress
#

Yeah looks like it

arctic spear
#

How can i use the pubmed api to access data on structures?

arctic spear
#

i got it nvm

torn sphinx
#

I need some help

#

with sqlite3

#
cursor.execute("UPDATE servers SET rankadmin = ? WHERE serverid = ?;",(role.id,ctx.guild.id))
novel vigil
#

how do i insert binary data into my table?

cur.execute("INSERT INTO post (binVal) VALUES (%?)", (bytes(something)))
#

what specifier would i put in place of the ?

umbral basin
#

Hello there.
I need help
I have import mysql-connector with the pip command but it's not working on this PC.
I have this error when I run it:

void otter
#

try mysql-connector

valid hare
#

HI guys may i know any way to use python connect mysql database with ssh ?

umbral basin
tender basalt
#

custom aiopg thingy is this:

#
import os


class aiopg_commands:
    async def connect(self):
        self.conn = await aiopg.connect(database=os.getenv('DATABASE'),
                                        user=os.getenv('USER'),
                                        password=os.getenv('PASSWORD'),
                                        host=os.getenv('HOST'))
        self.cursor = await self.conn.cursor()
        # self.pool = await aiopg.create_pool(self.dsn)

    async def execute(self, statement, args: tuple = None):
        if args is None:
            await self.cursor.execute(statement)
        else:
            await self.cursor.execute(statement, args)
#

error on last line

#

@void otter sorry for ping sir. can u help?

valid hare
#
import paramiko

ip = "ipaddress"  
port =  port123
user = "aaa"
password = "password"
ssh = paramiko.SSHClient()
ssh.set_missing_host_key_policy(paramiko.AutoAddPolicy())

ssh.connect(ip,port,user,password,timeout = 10)

stdin,stdout,stderr = ssh.exec_command("mysql")

result = stdout.read()
print(result)

ssh.close()

HI all with this code i success to login the ssh but when input any command it will only print b" , any wrong i did ?

glacial sparrow
#

I have one row [Lith, A2, intact, Demokdawa, 4]
and another [Lith, A2, intact, kenexey, 5]

And i want to agregate row in a select to get something like

[Lith, A2, intact, demokdawa (4) kenexey (5)]

void otter
#

port = port123

#

try changing that line

ionic pecan
#

@valid hare b'' just means the program printed nothing on standard output

#

try printing standard error

torn sphinx
#

@hazy crystal : This relationship worked?

wintry aspen
#

Hello, I'm facing an issue with SQL, an error message : sqlite3.InterfaceError: Error binding parameter :Title - probably unsupported type.

And don't figure out why... There is the script about the table creation :

 Postal_code text, City VARCHAR(55), Phone_number text, E_mail text)""")

Ans there about the insertion :


    c = conn.cursor()


    c.execute(("INSERT INTO formation_data VALUES"
               "(:Trainer, "
               ":Date_formation_start, "
               ":Title, "
               ":Street_address, "
               ":Postal_code, "
               ":City, "
               ":Phone_number, "
               ":E_mail)"),

            {'Trainer': a,
             'Date_formation_start': b,
             'Title': c,
             'Street_address': d,
             'Postal_code': e,
             'City': f,
             'Phone_number': g,
             'E_mail': h})

    conn.commit()
#

The title is a simple string

#

The Error comes up when I try to insert the information

wintry aspen
#

Okay that was because of "c" already used for the cursor ... -_-'

pure cypress
#

Should definitely name your variables better

wintry aspen
#

Yes ^^

valid hare
#

@ionic pecan i just notice that have a odbc group selection, is this posible do with python ?

ionic pecan
#

im not quite sure what that is

#

in general, computers can do anything if you work at it hard enough, so yes

glacial sparrow
#

Quick question :

I use this query

'''SELECT Relic.Name, Relic.Era, User.Pseudo, Relic.Quality, Relic.Quantity FROM Relic INNER JOIN User ON Relic.IDOwner = User.IDUser GROUP BY Relic.Name, Relic.Era, Relic.Quality'''

To "fuse" two lines from db

Basically these :

Lith, A2, 4, Demokdawa, Intacte
Lith, A2, 4, kenexey,Intacte

And it outputs this line

Lith, A2, 4, kenexey, Intacte

#

Can i make it output
Lith, A2, 4, kenexey(4) demokdawa(4), Intacte

Instead ?

Thx in advance !

#

Okay for future reference, the solution was using 'GROUP_CONCAT'

frozen osprey
#
mongo_credentials = {
    "host": "mongo",
    "port": 27017,
    "password": "censored"
}

mongo_client = AsyncIOMotorClient(**mongo_credentials)

db = mongo_client.main
#
@bot.command()
async def restart(ctx):
    await ctx.send("The restart command is starting")
    await db.contests.insert_one(
        {
            "id": ctx.guild.id
    }
#

I tried to use MongoDB but I keep getting those errors (from the code shown above)

#
raise CommandInvokeError(exc) from exc
discord.ext.commands.errors.CommandInvokeError: Command raised an exception: ServerSelectionTimeoutError: mongo:27017: [Errno 11004] getaddrinfo failed
heady pollen
#

The above error is normaly thrown when no MongoDB server is available.

#

check that mongo db is available and listening would be first port of call then check connection info again.

hazy peak
#

Hello, what is the best way to check if multiple columns combined are unique? For example, a column can have multiple rows of same data, but 2 columns cannot have same rows.

#

Im using flask_sqlalchemy btw

frozen lantern
#

is this any help?

#

@hazy peak

hazy peak
#

I tried adding table_args = (UniqueConstraint('customer_id', 'location_code', name='_customer_location_uc'),) but it didnt help.

#

I solved it by querying the data first before adding it to the table

frozen lantern
#

shouldn't you be checking uniquecontstraint with parent_id and child_id instead

#

but good to know it's been solved

hazy peak
#

I believe that my solution is a hack :/

#

but it works

frozen lantern
#

oh it's a hack alright

#

in django we have a UniqueTogether

hazy peak
#

that seems pretty easy

frozen lantern
#

Sets of field names that, taken together, must be unique:

unique_together = [['driver', 'restaurant']]

This is a list of lists that must be unique when considered together.

#

from docs

frozen osprey
#

How to delete the data from MongoDB database?

silent tapir
#

using aiomysql I want to catch errors, but there doesn't seem to be any documentation about it. Does someone know how I can do this?

frozen lantern
#

@frozen osprey are you using a wrapper or calling mongo api directly?

#

@silent tapir are you looking to catch database errors or library errors or all errors?

silent tapir
#

database errors

frozen lantern
#

because afaik aiomysql uses pymysql internally

#

so it should throw the same errors

thick urchin
#

does anyone know how to use update_one() with pymongo for objects

errant phoenix
#

guys anyone using sequelize?

include: [{
            model: gControl.Posts,
            where: {
              status: {
                [sql.Op.or]: ['approved', 'announce']
              }
            },
            attributes: ['author', 'permlink', 'url', 'status'],
            as: 'posts'
          },
          {
            model: gControl.Polls,
            attributes: { exclude: ['createdAt', 'updatedAt', 'author', 'memberPollsID'] },
            as: 'polls'
          },```
escape to cancel • enter to save
this won't work if im using additional where in the model gcontrol.polls
ionic pecan
#

this is for python database discussion, use the off-topic channels for other languages

wintry aspen
#

Hi guys, I have a issue with psycopq2 since few weeks now, I don't arrive to install it, and don't find any solutions ...

#

There is the error message :

#

Any ideas ?

#

So I cannot import it from VS :/

#

I tried with :
```sudo pip install psycopg2````

#

I don't see why :/

pure cypress
#

try that

wintry aspen
#

I did

#

It installed

#

but I still cannot use psycopg2

#

I mean I can't import

pure cypress
#

Why not?

#

What's the error?

#

The errors you showed relate to installing the package, not importing it

wintry aspen
#

No error PyPi is installed

pure cypress
#

So why can't you import?

#

What happens if you try?

wintry aspen
#

The same

#

I mean first and second error message

#

I showed you

pure cypress
#

That doesn't make sense. Are you saying that when you import it, it tries to install itself??

wintry aspen
#

Just a second

#

So If I understand

#

in first :
pip install PyPi```` and then : pip install psycopg2```

#

right ?

pure cypress
#

What's this for pip install PyPi?

wintry aspen
#

the psycopg2-binary not ?

pure cypress
#

No. That first command doesn't seem to do anything

#

Anyway, if you are trying to install from source (which is what pip install psycopg2 does), there are some prerequisites.

#

They are described here

wintry aspen
#

Thank you @pure cypress , I'm going to take a look

#

I was on the link ^^, It seems I didn't understand well ^^

#

The file postgresql is not found on my path /usr/lib

pure cypress
#

Did you install libpq-dev?

wintry aspen
#

I have installed postgreSQL from the website

#

is it the same ?

pure cypress
#

Probably not

wintry aspen
#

okay ^^

#

What is it ?

#

Error message as well when I install it from the terminal ...

#

ERROR: Could not find a version that satisfies the requirement libpq-dev (from versions: none)
ERROR: No matching distribution found for libpq-dev

pure cypress
#

Are you trying to install it with pip?

#

It's not a Python package

wintry aspen
#

yes

pure cypress
#

You need to use your system's package manager

#

Like apt-get, pacman, etc

#

What's your distro?

wintry aspen
#

I don't manage the terminal very well yet

#

I'm using Mac OS

#

is it Unix ?

pure cypress
#

oh macOs

#

hmm

#

Not familiar with it

wintry aspen
#

oh ^^

pure cypress
#

Apparently you can install it with homebrew

#

brew install libpq

wintry aspen
#

Hum ... The command is not found

pure cypress
#

Ah ok

#

So start here

#

Brew is a really big part of development on macOs

#

You'll probably be using it more in the future

wintry aspen
#

Okay thank you for your help @pure cypress

wintry aspen
#

OMG

#

I downloaded it

#

what a great feeling

#

Thank you @pure cypress

#

Now I'm going to try to import it

pure cypress
#

You're welcome

#

Hope it works

wintry aspen
#

Okay, so psycopg2 is installed but can't still import it

#

^^

pure cypress
#

You never answered what happens when you import it...

wintry aspen
#

I used brew to install "postgresql" then I "pip install "spycopg2", Done successfully. Then now when I Try to import it :
``ìmport psyco...```
The IDE doesn't propose psycopg2

pure cypress
#

Let's ignore what the IDE proposes...

#

Did you try actually running any code?

wintry aspen
#

not yet, just a second

#

import psycopg2
ModuleNotFoundError: No module named 'psycopg2'

#

I think that is coming from the path

#

/Users/anaconda3/lib/python3.7/site-packages

#

Visual cannot get access to because it's in anaconda3 ?

#

you think ?

pure cypress
#

Doesn't pip install install it for python2?

wintry aspen
#

-_-'

#

Right xD

#

Thank you again @pure cypress

#

It's working

#

🙌🏻

pure cypress
#

You're welcome

pearl hornet
#

I'm using pyodbc to talk to MSSQL

#

i suppose this is the answer from pyodbc docs:

Microsoft SQL Server
SQL Server's recent drivers match the specification, so no configuration is necessary. Using the pyodbc defaults is recommended.
wintry aspen
#

Hi @pure cypress are you there ?

wintry aspen
#

Hi guys, I'm still stuck with this psycopg2 ... That drive me crazy. Now I have this issue :

#

The cursor is not coming up. Then I take a look on the library to get information, and then :

#

A new Error message, about an image not found...

#

psycopg2 is so bad to install and to get set to use ...

#

Any Ideas ?

#

To get this error message, I called the "help()" section

#

I reinstalled openssl and I installed libffi

#

It's still not working

ionic pecan
#

install the precompiled package

#

pip uninstall psycopg2
pip install psycopg2-binary

wintry aspen
#

Thank you @ionic pecan for your return

#

I did it

#

I'm going to try again

ionic pecan
#

psycopg2 is not „bad to install“, mac‘s „package management“ is just awful (you need to export 20 env vars for each package you want to link to), on linux systems its super simple

wintry aspen
#

pip install psycopg2-binary already satisfied

#

And when I run again it, it's the same error message

#

:(:(:(

ionic pecan
#

Did you uninstall psycopg2 first?

wintry aspen
#

Yes

#

I did

ionic pecan
#

can you show the full output of pip install

wintry aspen
#

Yes jus a sec

#

Okay actually my command didn't word

#

I did again and it's responding

#

just a second, I test now

#

Okay

#

so now the help section is responding, but I can't set the cursor :/

#

Like it's not found in the library

#

c = conn.cursor()

conn = pg2.connect(database='dvdrental', user='postgres', password=password)

r = """
--sql
SELECT * FROM customer
;
"""

c.execute(r)

#

There is my error message :

#

c = conn.cusor()
AttributeError: 'psycopg2.extensions.connection' object has no attribute 'cusor'

#

Weird right ?

#

I set the "connect" in the variable "conn" and then the cursor to the connect in the variable "c" :/

#

Okay it's working now

#

But why visual studio doesn't propose me the library ?

#

I mean when I start to write "cur", "cursor()" doesn't show up in the combox

#

Thank @ionic pecan for your help

ionic pecan
#

the code you posted above doesnt work because it uses conn before its set

#

No idea about visual studio, sorry

wintry aspen
#

Okay ^^thank you

frozen osprey
#

Hey. How to update documents in MongoDB? When I use await db.contests_coll.update_one(...) it sends this error

raise CommandInvokeError(exc) from exc
discord.ext.commands.errors.CommandInvokeError: Command raised an exception: ValueError: update only works with $ operators
craggy coyote
#

Hey guys using a flask/sqlalchemy setup. So there are a few things in my database that change incredibly rarely (once every 5-6 months) but are accessed frequently The total amount of records in this tables are about 1k or so. Is there a way / point to basically cache these sections of the database?

I've looked into both redis and dogpile but not sure if they meet my needs or are overkill

pure scroll
#

having 1k records and DB won't make much of the difference if you store it to redis or not

#

you can consider using local in-memory cache if that is possible (e.g if you have a single instance of app running) or you could also do caching with expiring within few minutes

reef skiff
#

i would say that if you are not sure about it then you dont need it

sweet wolf
#

I just use a cache decorator for stuff like that

reef skiff
#

it depends on the way of accessing data

#

if you are using something like opengrpah it becomes complicatated

lime whale
#

SELECT ID, FIRST_NAME, LAST_NAME FROM CUSTOMER WHERE (LENGTH(FIRST_NAME) + LENGTH(LAST_NAME)) <12 ORDER BY (lenGTH(FIRST_NAME) + lenGTH(LAST_NAME)) ASC, CONCAT(FIRST_NAME, ' ', LAST_NAME) ASC, ID ASC

im doing leetcode test ATM what am i ddoing wrong???

glad lagoon
#

capitals

#

for a start

frozen osprey
#

Hey. Does anybody use MongoDB? What is the problem here?

await db.contest_coll.update_many({"guild": message.guild.id},
                                                  {"$set": {f"ready{save}": "True"}},
                                                  {"$push": {f"answers{save}": answer}}
                                                  )
Traceback (most recent call last):
  File "C:\Users\PC\PycharmProjects\Amino\venv\lib\site-packages\discord\ext\commands\bot.py", line 859, in invoke
    await ctx.command.invoke(ctx)
  File "C:\Users\PC\PycharmProjects\Amino\venv\lib\site-packages\discord\ext\commands\core.py", line 725, in invoke
    await injected(*ctx.args, **ctx.kwargs)
  File "C:\Users\PC\PycharmProjects\Amino\venv\lib\site-packages\discord\ext\commands\core.py", line 88, in wrapped
    raise CommandInvokeError(exc) from exc
discord.ext.commands.errors.CommandInvokeError: Command raised an exception: TypeError: upsert must be True or False
#

On MongoDB website it says that upsert is default

#

and it's False then

echo turret
#
import bitfinex
import pandas as pd
import numpy as np
import datetime
import time
import os
import MySQLdb


class Bitcoin:

    def __init__(self):
        self.limit = 1000
        self.time_step = 1000 * 60 * self.limit
        self.t_start = datetime.datetime(2019, 6, 19, 0, 0)
        self.t_start = time.mktime(self.t_start.timetuple()) * 1000
        self.t_stop = datetime.datetime(2019, 6, 27, 23, 0)
        self.t_stop = time.mktime(self.t_stop.timetuple()) * 1000
        self.bin_size = '1m'
        self.data = []
        self.api_v2 = bitfinex.bitfinex_v2.api_v2()
        self.start = self.t_start
        self.stop = self.t_stop
        self.symbol = 'btcusd',
        self.interval = self.bin_size
        self.tick_limit = self.limit
        self.step = self.time_step
        try:
            self.client = MySQLdb.connect(host='host', port=3306, user='user', passwd='passwd', db="btcusd")
            self.cursor = self.client.cursor()
        except Exception:
            print("ERROR")
            raise RuntimeError
        finally:
            print("Successfully connected to DB!")

    def insert(self):
        try:
            main_query = "INSERT INTO data(time, open, close, high, low, volume) values(pair_data)"
            self.cursor.execute(main_query)
            self.client.commit()
        except Exception:
            print("ERROR")
            self.client.rollback()
            raise RuntimeError

#
    def fetch_data(self):
        start = self.start - self.step
        while start < self.stop:
            start = start + self.step
            end = start + self.step
            res = self.api_v2.candles(symbol=self.symbol, interval=self.interval, limit=self.tick_limit, start=start,
                                      end=end)
            self.data.extend(res)
            print('Retrieving data from {} to {} for {}'.format(pd.to_datetime(start, unit='ms'),
                                                                pd.to_datetime(end, unit='ms'), self.symbol))
            time.sleep(2)
        return self.data

#

so guys how can I run this code, when I try to do, print(Bitcoin.fetch_data()) it give an error unfilled parameter self

#

so any ideas on running this thing>

nova hawk
#

You'll need to instantiate a bitcoin object.

oak pebble
#

im confused on how salting works

#

the way i understand it, passwords are hashed and the hashed password is stored, and when the user tries to log in, the password they enter is hashed and that hashed is compared to the stored hash

torn sphinx
#

same.

oak pebble
#

but when the hash is salted i dont see how its possible to compare it to the stored hash anymore

#

if the salt changes each time i think it would be impossible to compare two salted hashes

#

unless the salt used for the stored hash is stored itself, in which case i dont see the point of a salt

torn sphinx
#

Apes like me don't understand.

#

I

#

t

sweet wolf
#

That stuff is normally hidden for you anyway. Most web frameworks do that for you.

#

You just interact with User object the system created for you

#

If making a custom system for that you are probably in a bigger company and you would probably not be the person making it

#

In 2019 I would say it's irresponsible to customise that in most cases

oak pebble
#

@sweet wolf do you know how it works though

frozen osprey
#

Hey. Why it's not working? I'm 99% sure it's good but it doesn't add anything to colours3: [] in my database.

save = 3

color = (await bot.wait_for('message', check=check)).content

color = {
                    'red': 0xff3300,
                    'green': 0x66ff66,
                    'yellow': 0xffff00,
                }.get(color.lower(), 0x2c2f33)

            await db.conest_coll.update_one({"guild": message.guild.id},
                                                {"$push": {f"colours{save}": color}}
sweet wolf
#

It was to support some very insecure password hashing from a a very old system made in php

#

Then django will upgrade the hashing type to the most secure one when the user logs on (The hashing method is store on the user)

#

This was coverting a phpbb forum to a Misago forum (hobby stuff)

#

For work/professional settings I guess most people deal with stuff like oauth2 (at least in larger organisations)

lucid fulcrum
#
  • oauth2
sweet wolf
#

Totally never happened 😄

placid flicker
#

What's the best way to connect to a remote MYSQL database and insert queries via Python 3?

oak pebble
#

I meant do you know how the salting works

lucid fulcrum
#

In cryptography, a salt is random data that is used as an additional input to a one-way function that "hashes" data, a password or passphrase. Salts are used to safeguard passwords in storage. Historically a password was stored in plaintext on a system, but over time addition...

torn sphinx
#
data = json.load(open("file users.json", "r"))```
#

How do i link to one data base?

#

as it does not fine that file

lucid fulcrum
torn sphinx
#
Traceback (most recent call last):
  File "C:/Users/conno/Desktop/Nite Smells/idlebot.py", line 6, in <module>
    data =  json.load("file users.json", "r")
TypeError: load() takes 1 positional argument but 2 were given```
lucid fulcrum
#

Sorry, my fault

#

You were right

torn sphinx
#

😮

#
Traceback (most recent call last):
  File "C:/Users/conno/Desktop/Nite Smells/idlebot.py", line 6, in <module>
    data = data = json.load(open("file users.json", "r"))
FileNotFoundError: [Errno 2] No such file or directory: 'file users.json'
>>> 
sudden solstice
#

is file users.json in the same folder as your script?

lucid fulcrum
#

There is no such file, simple.

torn sphinx
#

yes it is

lucid fulcrum
#

Try using the absolute path for now

sudden solstice
#

how are you running your script?

torn sphinx
#

idle pyth

lucid fulcrum
#

CWD needs to be the directory your file is in, I think

torn sphinx
#

??

lucid fulcrum
#

Current working directory

torn sphinx
lucid fulcrum
#

but try C:/Users/conno/Desktop/Nite Smells/users.json

torn sphinx
#

what insted of just users.json?

lucid fulcrum
#

^

#

Why should you write file in front of it if your actual file was named users.json ?

#

open("users.json", "r"), for the record

torn sphinx
#

??

#

man

#

what r u talking about

#

LOL

lucid fulcrum
#

Does it work now ?

torn sphinx
#

igvve not changed nothing

#

xd

#

like this?

sudden solstice
#

no

#

why is the path there twice?

lucid fulcrum
#

^

sudden solstice
#

connor, look at it this way:

#

you have to tell python where to find your file

torn sphinx
#

oh my

#

it was there twice buc of my keyboard

#

it always double pasters

#

ok now it is a single

oak pebble
#

@lucid fulcrum salting is used to store things in databases

torn sphinx
lucid fulcrum
#

Yes

torn sphinx
sudden solstice
#

wait, does your code say

#

data = data = json.load?

torn sphinx
#

my code is above

sudden solstice
#

it should only be data = json.load

you can't do that haha

oak pebble
#

i believe you can have multiple comparisons in the same line

sudden solstice
#

really?

oak pebble
#

yes

#
>>> a = b = 5
>>> a
5
>>> b
5
#

its not that data = data = json.load doesnt work, its just redundant

torn sphinx
#

deniz im 100% sure

#

your the one that told me to put data=data

#

same error

#
>>> 
=========== RESTART: C:/Users/conno/Desktop/Nite Smells/idlebot.py ===========
Traceback (most recent call last):
  File "C:/Users/conno/Desktop/Nite Smells/idlebot.py", line 6, in <module>
    data = json.load(open(" C:/Users/conno/Desktop/Nite Smells/users.json", "r"))
OSError: [Errno 22] Invalid argument: ' C:/Users/conno/Desktop/Nite Smells/users.json'
>>> 
#

@sudden solstice

gilded narwhal
#

there's a space at the start of the file path

#

@torn sphinx

torn sphinx
#

were?

gilded narwhal
#

right at the start

#

the first character

#

' C:/Users/conno/Desktop/Nite Smells/users.json'

#

before C

torn sphinx
#

yes