#databases
1 messages · Page 67 of 1
yup true
Relevant link in case it's helpful
https://docs.djangoproject.com/en/2.2/ref/checks/#related-fields
@void flower I fixed it by creating a virtualenv in the container
Don't know why that fixes it though
wait what the fuck
no it didnät
I got the check to pass...
lol
Yeah.
Within the container it will migrate and work fine
The docker image build is where it blows up
I tried using docker inspect to look into the runtime more of the container vs the image
but I have no idea what I'm looking at
Something even weirder happened. I installed the venv but didn't activate it properly yet somehow the check passed. But then switching back to makemigration failed
😩
I've been reading a bit on this
And my understanding is that virtual env configurations don't persist between layers
Well probably activation wouldn't
I'm trying RUN /bin/bash -c "source .venv/bin/activate && pip3 install -r requirements.txt && pip3 install -r requirements.txt"
Ok so get this
I did that
then
docker run -it imagename /bin/bash
So now I activate the venv
I run the check and it fails
🤔
Yeah
It's hit or miss which ones are provided
Sometimes it's both
Sometimes it's one
It's never neither 😩
where the hell are the migrations even going?
do they never get created?
cause of the check error?
That is my understanding
it short circuits
/opt/piano_gym_back_end/piano_gym_api/migrations is where they typically end up
If I hop in the container and do the migrations, which will work, that is where they would generate
We're a power combination
That may be a good thing
I was curious what this will return
AppConfig.get_models()
Yeah, LearnerEnrolledSchool and Course legitimately do not show up in the models list
Do you know how Django is even discovering your models?
no
It says it just attempts to import the "models" submodule of each app
And all you have is an __init__ which imports user and usermanager
Maybe you just need to import them there?
Sorry, I'm a little slow on the models discovery part you're talking about
Are you talking about in the path piano_gym_back_end/piano_gym_api/__init__.py
/models/__init__.py
I see
This is a bit of a tangent question
I moved my models down into the versions directory
and kept the models directory because I had to for the sake of django's default user configurations
If I'm supposed to provide them in the models/__init__.py but they live in the versions/v1/* does that mean I should try versions/v1/__init__.py for discovery?
I am reading the link you sent right now
I don't think so since you never specified "versions" as an app
I
am not well versed in how it works, that's just my understanding of it
I have no clue how any of your models are discovered
Doesn't make sense based on what I am reading
still reading
So I fixed it in my venv-ised docker image that was confirmed broken before
All I did was import the two models into the piano_gym_api/models/__init__.py file
But that feels like kind of a hack; you should probably be importing all your models
Can you paste the import you used?
or just specify the app as the version.v1 thing but idk if that'd break other shit
from piano_gym_api.versions.v1.models.learner_enrolled_school_model import *
from piano_gym_api.versions.v1.models.learner_enrolled_course_model import *
ah
From your experience does my models hiearchy look silly?
Am I doing something crazy that I'm not aware of?
Well to be honest I don't have much django experience
I just did some PRs for some back end DRF stuff once
You sound like the holiday inn commercial lol
If you want to get an idea for hierarchy you should take inspiration from open-source django projects
Well our own site is one
Not exactly prominent but... https://github.com/python-discord/site/tree/django
So all you did to fix was add those two imports?
Great but like I said don't rely on it
agreed
I think it's best to import all models
Right now I am guessing the rest of the models weaselled their way through in some imports somewhere else
Bleh
I thought I had solved this when I did the initial configuration of the project
clearly I did not
@pure cypress I cannot articulate enough how much I appreciate you taking the time to sit down and work through this with me
Is there anything I can do to show that gratitude?
You're welcome
Also I'll be removing you from the gitlab project 😉
Just a thanks is enough so don't worry
Well thank you, and thank you, and thank you!
is there an sql help discord?
its this one
Exception in thread Thread-5:
Traceback (most recent call last):
File "C:\Users\admin\AppData\Local\Programs\Python\Python37\lib\threading.py", line 917, in _bootstrap_inner
self.run()
File "C:\Users\admin\AppData\Local\Programs\Python\Python37\lib\threading.py", line 865, in run
self._target(*self._args, **self._kwargs)
File "C:/Users/admin/Documents/Projects/sqlite-file-index/test_2.py", line 27, in add_items
(f'data {i}',)
File "C:\Users\admin\Documents\Projects\sqlite-file-index\sqlite_file_index\threadsafe_db.py", line 56, in execute
retry_while_locked(cursor.execute, query, params)
File "C:\Users\admin\Documents\Projects\sqlite-file-index\sqlite_file_index\retry.py", line 26, in __call__
return callee(*args, **kwargs)
sqlite3.DatabaseError: not an error
wat
schroedinger's error
Guys with sqlalchemy is there any way to have two linked columns from different table.
For example If I insert something on table1 I want table2 colx to use the data from table1 colx
wait. This is the same as relationship. Got confused 😄
What is the actual issue you want to solve with this?
@void otter late late reply, never used docker containers and the likes, first time trying a database, so I want it simple. SQLite but with json essentially.
is there an alternative?
for an sqlite with json?
mongoDB
its json postgres pretty much
you can run your local mongo server and interact with it through docker networs
idk if python has ORM for neDB
its same as sqlite, db is in a filesystem
and json
@torn sphinx
yes
is there an alternative?
neDB but idk if it can run with python
i know it can with js
google it really
3 of big databases with python are sqlite, mysql/postgres and mongo
ok
Any pointers as to how i can install rethinkdb on manjaro? I've tried most of the alternatives on their install guide but it all fails to build
Someone know SQLAlchemy? I asked in #help-apple, please ping me if you know it
@dull scarab Don't use Manjaro but post errors and what guides you've followed maybe I can help
2 sec, lemme boot up
https://rethinkdb.com/docs/install/arch/ is what i followed
is a chunk of the last logs
using the package manager
I tried building manually and through pacman, but it all failed while building
does manjaro support AUR?
It should?
Did you try installing from there?
Do you have the build log?
Someone also left a comment on AUR FYI, package fails to build with boost 1.69.0-1, but successfully builds with 1.68.0-2.
And given the issue does seem to be related to boost, that may be a good lead
Any ideas how i specify version?
No actually. I've never had to do that before
It may be especially weird to do since this is a dependency and not directly the package you want to install
Cause normally with arch you upgrade everything whenever you wanna install a new package
Trying to build the AUR package from here: https://aur.archlinux.org/packages/rethinkdb/ src/clustering/administration/auth/user.cc: In member function ‘bool auth::user_t::has_connect_permission() ...
would i run that before trying to build then?
Yeah, before installing rethink
oof yeah I somehow ended up building qt4 and that took like 20 minutes fuck me
Yeah this is suuuper slow
Theres even a comment in the buildfiles saying "this makes it super slow" or something
@pure cypress Holy shit, i believe it worked. Only 1 hour and 20 minutes. Thanks alot 
Being that long, I would have felt bad if it didn't work
@dull scarab Use docker?
If your desperate to get it up and running
Also good to know I'm not the only one still using it! haha
¯_(ツ)_/¯
It seemed like a good fit for a project im starting with the university this summer
I like the changes feature or rethinkcb
a few lines of code, and you have full database logging
but tbh, I don't know the best fit for out project, its a first draft of the system structure so figured we'd give it a go
I don't know docker yet 
I could never get them to work
Ouch.
Lemme go get you the single command install if you want?
We seemed to get it installed on all 3 machines, mac, win and manjaro
windows it's easy
I don't thionk he wants to wait for it to build again 😄
Hahaha

Or can yo install it without building from source in docker?
Is there a pebuilt binary somewhere?
sudo docker run -p 28015:28015 -d -v chat:/data rethinkdb:latest rethinkdb -d /data --initial-password password --bind all
This will expose port 28015, and stop data getting deleted when your server restarts
And boom it's running!
Nope.
not a bind mount
There's a command in there
That's a docker-volume
It keeps the data
or at least I think it does
Right?
Yes that's what volumes do
I just thought for named volumes you had to create the volume beforehand
Nope, it's automatic
Which isn't the case for bind-mounts for like a local directory
:D
oh ok 🤷
Might consider that if we have problems deploying, thanks sharp
anyone here who could assist me
!ask
Asking good questions will yield a much higher chance of a quick response:
• Don't ask to ask your question, just go ahead and tell us your problem.
• Try to solve the problem on your own first, we're not going to write code for you.
• Show us the code you've tried and any errors or unexpected results it's giving
• Keep your patience while we're helping you.
You can find a much more detailed explanation on our website.
select ID, FIRST_NAME, LAST_NAME from CUSTOMER where (length(FIRST_NAME) + length(LAST_NAME)) < 12
order by length(FIRST_NAME) + length(LAST_NAME), (FIRST_NAME + LAST_NAME) asc, ID asc;
how do i get this right? sorting by full name?
can anybody explain the term s3 compliant used commonly these days?
does it mean it's using s3 as the backend storage only? or is it something that complies with communication based on s3 like api?
Has anyone used SQL Machine Learning Services, specifically writing Python inside of it? I’m trying to set up a machine learning model and serialize it in a database but I’m having trouble getting the code sorted out.
Without going into too much detail I’m trying to import a CSV file with the training data and features (3 columns total of 5000 rows). I used pandas.read_csv to import the file to a data frame, would putting the file into the database make the path a bit shorter?
@charred thorn what do you mean shorter path? isn't pandas.read_csv already the simplest read op you can do?
but it sounds like SQL service lets you execute python code at DB side so you might not need pandas at all? they provide package..
I meant like the path to the CSV file. I might ask my mentor for info on how accessing files from SQL works tomorrow
i mean you can do direct pandas to SQL call
but if they provide package, pretty sure there'd be io related ones
Got it, I'll give that a shot when I get back to work on it tomorrow
what's the benefit of this way of running btw? just to save data travel time?
interesting service
Ideally the model would be stored in the database for future access. And like you said it would cut down on data travel time.
I don't think there's any major benefits that come to mind, but it's a pretty new tool at the place I'm interning at so successfully implementing it in a project as an intern would be pretty impressive :) @torn sphinx
what is 'model' that's stored in db like?
it would be an XGBoost binary classification model to take a few inputs and determine if the output is "true" or "false" based on them
oh i mean, in what data form are you storing it? in storing in regular SQL table?
It would be dumped using the pickle library and serialized, then stored as a value in a regular SQL table until it's needed afaik
so as binary blob like then
interesting use of relational table.
so you'd have model parsing done on app layer
Can someone give me a general knowledge about creating views in MySQL?
I got offers table:
I got flights table:
How to "join" those two, for example to show:
outbound_flight_flightNumber
inbound_flight_from_aiport
inbound_flight_from_date (with some function which would parse unix timestamp to human readable)
I want a general concept, then I would follow to add more informations. That's why I'm asking about 3 cases.
Airports table (would be need):
Hello, quick question, what is the difference between pgadmin 4 python wheel and pgadmin for macOS ?
I don't see why python is compare to an OS
oh I see, that is because it's run as a web app for the version 4
In relation with last question:
CREATE
ALGORITHM = UNDEFINED
DEFINER = `root`@`%`
SQL SECURITY DEFINER
VIEW `offers_viewer` AS
SELECT
`offers`.`price` AS `price`,
`offers`.`outbound_flight_id` AS `outbound_flight_id`,
`out_flights`.`flight_number` AS `outbound_flight_number`,
`offers`.`inbound_flight_id` AS `inbound_flight_id`,
`in_flights`.`flight_number` AS `inbound_flight_number`
FROM
((`offers`
JOIN `flights` `out_flights` ON ((`offers`.`outbound_flight_id` = `out_flights`.`id`))
JOIN `flights` `in_flights` ON ((`offers`.`inbound_flight_id` = `in_flights`.`id`)))
JOIN `airports` `out_airports` ON (`out_flights`.`from_aiport_id` = `out_airports`.`id`))
ERROR 1054: Unknown column 'out_flights.from_aiport_id' in 'on clause'
seems like you've misspelled airport
@pliant oxide
Does anyone know why blob storage is called so.... I understand it's binary large object storage.. but what's the background/context
WHERE Country = 'Germany' AND 'France'
I am getting a syntax error for AND
does anyone know why this is wrong?
You probably need a complete conditional statement on both sides of the AND
WHERE Country = 'Germany' AND Country = 'France'
However, can the Country be Germany and France simultaneously?
Maybe you want OR
hey 😄
I need help for Pandas as well 😃
I'm currently using Pandas - and I used this line of code
d = data.groupby(['Category'])['Price'].sum()
(read from a csv file)
how can I then separate the results into 2 different lists ?
if I use d.tolist() it only creates a list of values
but I also need the names
not sure if I did a good job at explaining
When creating models in sqlalchemy based on my classes, if I make a modification to a class --> say I add another column to class after already running the app for the first time...wil sqlalchemy autoupdate the database schemea and add the new column
or you could migrate the class table with something like alembic
Hi, I have some issue with mariaDB insert into where not exists
Traceback (most recent call last):
File "C:\Users\DEVEAUX\Desktop\parse adresse.py", line 251, in <module>
(line["numéro"], rue_id, line["latitude"], line["longitude"], line["numéro"], rue_id),
File "C:\Users\DEVEAUX\AppData\Local\Programs\Python\Python37\lib\site-packages\mysql\connector\cursor_cext.py", line 266, in execute
raw_as_string=self._raw_as_string)
File "C:\Users\DEVEAUX\AppData\Local\Programs\Python\Python37\lib\site-packages\mysql\connector\connection_cext.py", line 398, in cmd_query
sqlstate=exc.sqlstate)
mysql.connector.errors.ProgrammingError: 1064 (42000): You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near 'WHERE NOT EXISTS(
SELECT *
FROM `addresse`
WHERE `label` = '1'
A' at line 3```
cursor = conn.cursor()
cursor.execute(
"""
INSERT INTO `adresse`(`label`, `id_rue`, `latitude`, `longitude`)
VALUES(%s, %s, %s, %s)
WHERE NOT EXISTS(
SELECT *
FROM `addresse`
WHERE `label` = %s
AND `id_rue` = %s
)
LIMIT 1;
""",
(line["numéro"], rue_id, line["latitude"], line["longitude"], line["numéro"], rue_id),
)
cursor.close()```
the where not exists is for prevent duplicate
+-----------+--------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+-----------+--------------+------+-----+---------+----------------+
| id | int(11) | NO | PRI | NULL | auto_increment |
| label | varchar(255) | YES | MUL | NULL | |
| id_rue | int(11) | NO | MUL | NULL | |
| latitude | float | YES | | NULL | |
| longitude | float | YES | | NULL | |
+-----------+--------------+------+-----+---------+----------------+
5 rows in set (0.01 sec)```
and adresse table is created with sql CREATE TABLE IF NOT EXISTS `adresse`( `id` INTEGER PRIMARY KEY NOT NULL AUTO_INCREMENT, `label` VARCHAR(255), `id_rue` INTEGER NOT NULL, `latitude` FLOAT, `longitude` FLOAT, INDEX (`label`, `id_rue`), CONSTRAINT FK_address_id FOREIGN KEY (id_rue) REFERENCES rue(id) );
Having a table with id,date time,number value
is it possible to group the values where the date is similar? (because it's like "2019-06-10 12:00")
to select them and have the total per day as result.*
or would it be simpler to do this through python?
why would I not be able to put an int into a blob field of a sqlite database? Aren't you supposed to be able to put anything in there?
I would just use an INT field but it's a discord user ID so I need 64 bit unsigned, not signed
This statement complains about the paramaters being of unsupported type
SELECT * FROM Users WHERE ID = ?;
and then ? is being swapped out with the ID of a discord user, which is int
also happens if the column is UNSIGNED BIG INT
Are you sure you put the ID in a tuple?
Yeah, for parameter substitution it needs tuples
oh I was passing them as seperate paramaters
so the ID needs to be in its own tuple or just the whole thing?
Like this: py execute("SELECT * FROM Users WHERE ID = ?", (user_id,))
but if I'm passing multiple things would it be like (foo, (user_id)) or (foo, user_id)?
the latter
alright
the line is this and it still says unsupported type
c.execute("SELECT * FROM Users WHERE ID = ?;", (ID))
The comma in the tuple is important
(ID) is not a tuple. (ID,) is a tuple
I saw that ones with multiple values didn't need it so I figured single values were the same
Tuples with multiple elements don't need a trailing comma cause it isn't ambiguous whether it's a tuple
well that part is working, thanks
hello guys i new at programing where should i learn it
!resources is a good place to start
The Resources page on our website contains a list of hand-selected goodies that we regularly recommend to both beginners and experts.
Does kubernetes support distributing an sql or nosql database over the entire network
So you have 5 database pods and each one has ~1/5th of the database in a special docker volume (not inluding backups)
I think this would be something like redis but hdd based? Never looked into redis
Things like Postgres support connection pools so everything isn't on one connection
Or well, that's up for the lib to implement
Idk about psycopg2 but asyncpg does it well
If you're worried about database performance, it's better to look at the database first before things like that
Postgres will always outperform SQLite because SQLite is file based
But of course this difference only matters at scale
And is there anywhere you could save yourself a query or cache results etc.
Hey. When I try to put sth in db I got this error (I'm on windows).
raise CommandInvokeError(exc) from exc
discord.ext.commands.errors.CommandInvokeError: Command raised an exception: ServerSelectionTimeoutError: connection closed,connection closed,connection closed
And how to reboot mongodb server?
Hello, anyone knows where may I find, a WebSite to download free data file, to play with ?
Depends on what format data file are you interested in. @wintry aspen
I'm playing with SQL, so I think that would be .tar or .db, If I'm not wrong
SQL is a general purpose query language. It does not refer to any particular database storage format or engine.
It can be used with a variety of technologies.
Yet if you want something simple to start with - try using sqlite.
It should be pretty straightforward to make a file to play with yourself.
thank you !
Welcome. It's just one google away. =]
When you work on data and SQL, what is the best way to display it on chart ?
You mean some kind of visual aggregation?
Hey. How to use mongodb?
You may want to express your question a but more detailed.
What exactly are you struggling with? Connection to the DB? Running queries?
@buoyant breach Right, visual aggregation 😃
@wintry aspen scipy is what comes to mind immediately. But it may be overkill, there must be some more straightforward libs for that.
@noble pelican you are using a cursor, you need to just do a normal select or bulk collect of some sort if you are fetching large amounts of records
if it;s indexed properly 12000 should be nothing for mysql to handle
check the db execution plan as well
Thank you for that @buoyant breach . Do you think I can use QT chart as well?
I did not wark with those all that much, so there is not much I can advice here, sorry.
You can try as long as there is python wrapper.
Okay, thank for your return at all 😃
Like https://pypi.org/project/PyQtChart/ may be.
Hey. When I use a cmd to add data to MongoDB it shows this error. What might be wrong?
raise CommandInvokeError(exc) from exc
discord.ext.commands.errors.CommandInvokeError: Command raised an exception: ServerSelectionTimeoutError: localhost:27017: [WinError 10061] The ability to connect, the target computer actively refuses it
I'm using atlas version on windows
Hi i'm using asyncpg with postgresql what my question is how can I query the text from a column? like say (this that) and not <Record = etc everytime I try i get an error that it's an unserializable json etc
{o.class.name} '
TypeError: Object of type Record is not JSON serializable
what i'm trying to do is make welcome and leave messages customizable
A record is only a container for what you want
There's docs that'll help you extract what you need from it
Also, there are shortcut functions that bypass Record in situations
You might want fetchval
Or fetchrow
ah ty! I'll try ose and will check out some other docs the doc i was reading didn't have much as to specific things I was looking for ty 😃
Hey guys. Im using asyncpg for my mod bot and im just wondering: what error does the lib return when an entry is submitted that already exists and thus is a duplicate primary key?
Been searching but can’t find much info about it on their documentation or github source
Disregard above. I ran a test and found it’s uniqueviolationerror
Hey question what would be the best way to format said string in python for postgresql? like say if I put "Welcome {member} to {guild}" how could I set it to format the member and guild? I'm unsure as to if i could just use the f/format() method on it as this is a public thing and many could use it so i'm unsure about what to do exactly
like as in the return value not when inserting to postgres
Hello, I don't see what is exactly the difference between 'serial' and 'integer', someone could give me a quick explanation ? 😃
It is about SQL by the way ^^
serial is just autoincrement for postgres. SQL Server would be "autoincrement"
upon each insert it autoincrements by 1 for the ID
just a sysid really
if you have good normalization it is what you would normally use as an ID for your records
otherwise youd have to use some type of GUID
or multi-key identifier
I will be doing an analysis of imdb database actor and their collaboration with other actors for a school project. I'm interested in showing how some actor's connections changed over the years and for example how different it was comparing an actor from 60' or 70' to some actor today. Anyone know if something similar to this has been done before? I tried searching a bit but had no luck.
Is redis good for kubernetes or should you use something like mongodb or postgres for databases
When running https://hastebin.com/sufekixuhu.sql this code, this error shows:
File "C:\Users\markc\Anaconda3\envs\toast\lib\site-packages\asyncpg\pool.py", line 558, in fetchrow
return await con.fetchrow(query, *args, timeout=timeout)
File "C:\Users\markc\Anaconda3\envs\toast\lib\site-packages\asyncpg\connection.py", line 455, in fetchrow
data = await self._execute(query, args, 1, timeout)
File "C:\Users\markc\Anaconda3\envs\toast\lib\site-packages\asyncpg\connection.py", line 1414, in _execute
query, args, limit, timeout, return_status=return_status)
File "C:\Users\markc\Anaconda3\envs\toast\lib\site-packages\asyncpg\connection.py", line 1422, in __execute
return await self._do_execute(query, executor, timeout)
File "C:\Users\markc\Anaconda3\envs\toast\lib\site-packages\asyncpg\connection.py", line 1434, in _do_execute
stmt = await self._get_statement(query, None)
File "C:\Users\markc\Anaconda3\envs\toast\lib\site-packages\asyncpg\connection.py", line 329, in _get_statement
statement = await self._protocol.prepare(stmt_name, query, timeout)
File "asyncpg\protocol\protocol.pyx", line 163, in prepare
asyncpg.exceptions.PostgresSyntaxError: syntax error at or near ","```
pretty sure that probably expects a tuple or some type of dict, not comma separated @pulsar timber
@quiet ermine Kubernetes is a way to have distributed containers, whether you use Redis or Postgres is up to you, usually you will have some type of compose file (for ex. maybe docker compose, then whatever database or database(s) your application uses you would define these in your startup file to be connected)
Redis is usually more for temporary type storage for user type information or information that is lightweight, whereas Postgres is a full RDBMS
More talking about horozontal scalability
I think i'll go with nosql like mongo
& use redis if I ever need it in the future; maybe something for ml
Is it possible to connect to a database that's on hosting server I bought if I'm programming on local machine?
Yes
Instead of connecting through it on localhost, just portforward your postgres port on the database's server (I think in your case the vps/hosted one) and use the [ip]:port instead of [localhost]:port
@open hill
@quiet ermine I tried but I'm getting ER_ACCESS_DENIED_ERROR. I used password I set
I'm using a MariaDB and mysql.connector, anyway I can get it to return a list of items, rather then a list of single item tuples?
Please ping me
@dawn pulsar you’re probably better off returning the results as a dictionary as mysql.connector only has support to return a tuple, namedtuple or dict for queries afaik 😄
Why don't they add the ability to return a list?
@atomic plank .format() would be an easier option
@void otter could you help me ?
Im doing a space invaders game and I need to know how to get new levels and how to do enemies shooting @void otter
Including db?
sorry but whats db
Database
uhh i dont think i need that
Then i think you should ask in some #help channel
OH ok
Or maybe #user-interfaces
ty Saki and I figured out what the issue was had to have a sort of array with things like {"user": author.name} and the issue i was having was if a user did anything other than what was in the array it would result in a key error I found a way around that by using .format_map(defaultdict(str, **items))
just for others who have a similar question or issue ^
I’m just gonna bump this into here as well since it’s db related: I am using curl to write json values to mongodb via bottle and everything I write returns the key:value as key:null, does anyone have any suggestions on where I’ve gone wrong https://paste.pythondiscord.com/izajebiqeb.py
anyone?
How would you distribute a database like mongodb over a kubernetes cluster
So if a disk dies you 1. have a backup somewhere else and 2. it makes a container somewhere else and gets backups sent to it
So your entire database is distributed across the entire cluster
Can i ask a quick question? I am doing some database work with Python. It is actually strictly more of a MySQL question rather than a Python question. Can i ask anyway?
Yes of course
I am trying to load data from a .csv file into my database.
Lets say my database has the fields name, surname, dob.
My csv is structured like this: Donald, Trump, 14/6/2019.
Now some of my csv rows look like this : Barack, Obama,
Or even Barack,,
So missing some values. When i try to load this .csv i get an error that these rows have different date values for those fields for example.
How can i fix this?
Also sorry for the horrid example. I am really tired.
Uhh, it's a lil unclear what the issue actually is here. Are your fields with missing data corresponding to columns in your table that are marked as NON NULL or something of the sort?
Or if you had the precise error you get that might be helpful
None of the tables are marked as NON NULL. I thought if i have an empty entry in my .csv like ",," it would just count as NULL. That is what i do not understand.
Error Code: 1292. Incorrect datetime value: '' for column 'dateofDeath' at row 18
That is my exact error.
Thanks for helping.
Uhh.. What is the actual line of the CSV at row 18 or whatever that prompts this error?
None of the columns are markes as NON NULL is what i meant so say ofc my bad.
Wait i will look for the exact line
http://www.wikidata.org/entity/Q317232,Carausius,0300-01-01 00:00:00,0293-01-01 00:00:00,carausius
This is the CSV row 18.
And i am inserting into this table:
item varchar(50),
itemLabel varchar(50),
dateofBirth datetime,
dateofDeath datetime,
nomisma varchar(50)
);```
That's.. strange? Because the error seems to imply that at this row there is an empty string passed as a datetime value
Oh i got an idea. With LOAD DATA INFILE i am ignoring the csv header.
maybe it is actually referring row 19?
row 19
http://www.wikidata.org/entity/Q61088565,Q61088565,0250-01-01 00:00:00,,
ah yea
errrrrrrrr
I really do not get why it does not just set the Field as NULL though.
Or am i a complete moron?
I don't get it either (although I have no experience importing CSV to MySQL)
I guess you could just try explicitly setting the missing ones to NULL? Though that might require a bit of work.
Even if i change the row to have some artificial datetime and varchar i get the same error.
Setting it to NULL also did not do anything.
Can you perhaps copy paste that row and also the couple rows before/after?
I will in a second. I just deleted the row but it gives the same error when the missing datetime occurs again.
So it is also not some weird character chain that is getting misinterpreted
Row 17 through 21:
http://www.wikidata.org/entity/Q317232,Carausius,0300-01-01 00:00:00,0293-01-01 00:00:00,carausius
http://www.wikidata.org/entity/Q61088565,Q61088565,0250-01-01 00:00:00,,
http://www.wikidata.org/entity/Q885870,Victorinus II.,0300-01-01 00:00:00,0271-01-01 00:00:00,
http://www.wikidata.org/entity/Q550622,Maximus,0400-01-01 00:00:00,0422-01-01 00:00:00,maximus_barcelona```
I'm currently reading up on something about strict mode
What version of Mysql are you using?
errr mysql -v I assume?
haha
i lied
mysql --version
Anyways, if you're running 5.7, they added something that makes the rules of the database a lot stricter by default, including something called "NO_ZERO_DATE" which might have something to do with it
You can try changing the mode of the database by using the command, in the mysql cli,
mysql> set global SQL_MODE="NO_ENGINE_SUBSTITUTION";
Info: https://www.percona.com/blog/2016/10/18/upgrading-to-mysql-5-7-beware-of-the-new-strict-mode/
uhh, what's the number after Distrib
anyways, doesn't matte,r it's very likely this is your issue
I recommend trying to change the SQL_MODE as above and try again
yes thanks i will
Let me know if it helped.
thanks a lot i will in a sec
I am so confused right now.
My my.ini file does not have a sql-mode variable
and neither can i change it via the command line client
err, what happens when you try to do it thru the cli
uhhh never used that much
i get an error saying it is incorrect sql syntax
so i guess i am doing something wrong there from teh getgo?
Okay i set the sql-mode to NO_ENGINE_SUBSTITUTION
Sorry, I was afk
But it didnt fix the issue
:/
okay great timing actually haha
I'm pretty out of ideas regarding this issue lol
153 row(s) affected, 2 warning(s): 1265 Data truncated for column 'dateofDeath' at row 18 1265 Data truncated for column 'dateofDeath' at row 44 Records: 153 Deleted: 0 Skipped: 0 Warnings: 2
Okay i had to reconnect to the server for the sql_mode change to take effect. My bad.
With that i now get the above warning.
And the fields are not NULL but rather '0000-00-00 00:00:00'
I guess i can work with that. I really for the life of me cant understand why this is so difficult to get around or why it does not work at all. Thanks a bunch for your help though.
Really appreciated.
If you still care how it can be done differently i found a link @tender hollow
First answer is a really nice example.
hey guys, I know this isnt directly python related (yet), but I'm trying to set up mysql for the first time.
I can't seem to figure out how this works, but MySQL isn't allowing me to connect to localhost.
If anyone knows, do I need to change any of my computers host files or anything?
Have been looking online but can't seem to find any resolutions
https://gyazo.com/6614879fa2a5a764990d48f45227ef6d
I don't have MySQL running in services or anything either...
Well, it would appear that the Mysql server itself isn't running..
You shouldn't have to do anything with your host files, 127.0.0.1 should point to localhost already
It doesn't :/
I have no idea what to do
I just uninstalled and doing a new reinstall
hoping it fixes
Alright
Got my database working now
Getting this error:
ModuleNotFoundError: No module named 'mysql.connector'; 'mysql' is not a package
I installed the mysql-connector, got that error
then installed mysql-connector-python-rf
ah ignore im blind
Can i ask another question? My brain really is not working today!
sure just ask 😛
My brain is absolutely fried here at 4 am and i am kind of new to SQL. Can you help me out?
Table 1: fatherName, dob, etc.
Table 2: fatherName, childName
Now it does not make much sense in relation to my example above. But lets assume that two different fathers can have the same child.
How could i SELECT the name of the fathers and the name of their "common" child?
Okay i rephrased my question. It is much more clearer now (i guess!).
I think you set up your schema poorly
In the second table, the child should be the primary key
and it should have a column for each parent
cause each person has 2 parents right 🤔
Well that's actually an interesting case, if it's possible to have more than 2 parents (obviously not biologically)
Not sure how you'd account for that...
A parent can have multiple children too, so this would actually be a many-many relationship I believe
In that case you could use what's typically called a "junction table"
can anyone help me with this?
Command raised an exception: TypeError: 'asyncpg.Record' object is not callable
@pure cypress?
Please don't ping helpers arbitrarily. I am not familiar with asyncpg but I could try to help if you provided more details
Post the full traceback for the error
and include the relevant bits of your code
Alright @pure cypress i have no controll over how my schema is set up though. That is why i had the question on how to do this in the first place. I guess it is some kind of many to many realtion yes!
Thanks anyway!
It is not actually parents though in my database that was just an example i used previously.
a poor example i admit
@welcome.command()
async def channel(self, ctx, channel:discord.TextChannel):
if ctx.message.author.guild_permissions.manage_messages or ctx.message.author.id == (553614184735047712) :
global client
db = client.bot
channelid = channel.id
posts = db.serversettings
for x in posts.find({"serverid":f"{ctx.guild.id}"},{ "_id": 0, "serverid": 0, "leveling": 0,"swearing":0,"automod":0,"economy":0 ,"welcome":0,"prefix":0}):
achannel = x["wchannel"]
print(achannel," ",x)
myquery = { "wchannel": f"{achannel}" }
newvalues = { "$set": { "wchannel": f"{channelid}" } }
print(achannel)
a = posts.update_one(myquery, newvalues)
print(a,"\n",channelid)
await ctx.send(f"Channel has been set to {channel.mention}")
Doesn't save to the database
this is mongodb
@plain radish
@knotty parcel please don't mention and remove multiple (staff) members until your question gets some attention.
I'm not available right now.
Anyone know a solid tutorial online for discord + mysql?
I think you have more luck asking that in the discord-py channel. But i can not help you myself.
is it possible to delete everything without 1 thing?
something like DELETE * FROM guild_settings WITHOUT lang
ping me if you can help me
DELETE FROM guild_settings WHERE guild_id = $1 AND row_name != 'lang' maybe something like that but how
@copper echo I don't technically understand what you mean are you referring to deleting just certain items or the whole entry?
i want to delete whole data from table without one column
Okay so I don't necessarily know how to do delete everything from one system but leave 1 column
but you "could" try
DELETE x FROM table where guild_id = $1
Similar to regular selecting but change X to what you want to delete
not tested this and i'm just getting into sql so i don't know if that would work
or optional just use DELETE FROM table WHERE guild_id = $1
which would delete everything including the column you wish to save
How would I create a table using a variable to name it? Can't seem to find any information online. (sqlite3)
delete from table is deleteing everyting from it
delete x from is not working for me idk why
i meazn
i tried DELETE something, something1, something2 FROM table
You want to delete certain columns for certain rows?
If I understand correctly then this doesn't seem possible cause all rows must have the same columns
You could set the columns to null but you couldn't just straight up delete them selectively
You could also drop a column but that would affect the entire table
i want to delete all columns except one - "lang"
You want to delete them for the entire table?
Any online db
where I can hit a URL and the params get saved
so database saves Vraj
I think firebase can do it
any ideas
yes Mark
my table:
CREATE TABLE IF NOT EXISTS players (
user_id BIGINT PRIMARY KEY,
ign VARCHAR(32) UNIQUE NOT NULL
)```
now when i try to insert a row(i.e. add a player), i want to know if the player already exists or not.
What i am doing rn is:
```sql
INSERT INTO players (user_id, ign)
VALUES
($1, $2)
ON CONFLICT (user_id, ign)
DO NOTHING```
what would be the best way here to know on what column the conflict was(was it that the player already is registered or the ign is taken by another user) or if there was no conflict and everything went on smoothly
using PostgreSQL + asyncpg
@carmine heart Are you available
Just ask your question and someone may help you. No need to ping specific people.
In pymongo How do I get a request amount so like see how many times that query pops up and a detail from every query
What have you tried
@unborn sentinel I haven't tried anything since I'm not even sure what to do the closest is
for x in posts.find({"guildid":f"{ctx.guild.id}","_id":f"{user.id}"},{ "_id": 0, "serverid": 0}):
reason[x] = x["reason"]
issuer[x] = x["issuer"]
a += 1
print(f"{reason1}\n{reason2} and {issuer2} in {a}")
So you haven't tried to run that?
I don't have access to the database, so there's no real way I can test it.
Would you like access?
Honestly not really. We can try to help you work out the logic, but the main work has to be done by you. We won't write the code for you.
@unborn sentinel So could you help me?
I currently cannot, as I'm swamped at work. Hopefully someone will be around to help you at some point, but until then just keep trying.
I was hoping you would since I'm at work too
Need to change this without taking up that much space,
So I found out how to use a var to name a table:
sql_cmd = '''CREATE TABLE {}(test, test2, test3, test4)'''.format(
tableName)
dataBaseConnection.execute(sql_cmd)
But now i'm trying to also name the row names with variables.
It may be worth noting if you take user input that executing sql with string formatting could leave you open to some nasty sql injection 😃 just a heads up
@rotund cove Can you sql inject into a non sql database such as mongodb
How do I take things from a database and put them in 1 output:
@commands.command()
async def warnings(self,ctx,user:discord.Member):
global client
db = client.bot
posts = db.warnings
a = 0
for x in posts.find({"guildid":f"{ctx.guild.id}"},{ "_id": 0, "serverid": 0}):
reason = x["reason"]
issuer = x["issuer"]
print(f"{reason} and {issuer} in {a}")
a += 1
I could use some advice on using Alembic and organizing code into modules:
I am setting up autogeneration with the sqlalchemy declarative base.
In my_project folder, I used alembic init data so I can keep all my data stuff there.
In there I made a folder called models with a basic.py file inside that.
In that basic.py file I have a simple setup:
from sqlalchemy import Column, Integer, String
from sqlalchemy.ext.declarative import declarative_base
Base = declarative_base()
class SomeClass(Base):
__tablename__ = 'some_table'
id = Column(Integer, primary_key=True)
name = Column(String(50))
So that in alembic's env.py file I can import the Base and assign it’s metadata to the target_metadata value.
However:
When typing my import statement PyCharm wants to autocomplete as if I was in the project directory, even though the env.py file is in the data directory. So I do from data.models import basic but on that alembic fails ModuleNotFoundError: No module named 'data’
If ignore PyCharm and do the seemingly obvious from models import basic alembic fails ModuleNotFoundError: No module named 'models’.
Alembic also fails if I try from my_project.data.models import basic
Adding __init__.py files doesn't help either. I think they are unecessary as of 3.3/4?. I am running on 3.7.1
├── my_project
│ └── data
│ ├── models
│ └── basic.py
│ └── env.py
│ └── versions
│ └── ...
well I figured out that since alembic isn't local nothing related to the project is in sys.path so using PYTHONPATH helps, but I think I am going to add what I want to sys.path in the env.py file.... hopefully that's not a bad idea.
I know this isn't really for python but if someone could help, it'd be appreciated
level = await self.bot.pool.fetch("SELECT * FROM leveling WHERE guildid = $1 ORDER BY lvl DESC LIMIT 5", ctx.guild.id)
xp = await self.bot.pool.fetch("SELECT * FROM leveling WHERE guildid = $1 ORDER BY xp DESC LIMIT 5", ctx.guild.id)
``` im trying to get the xp and lvl and have it organized, but i still get it where a person that the same lvl but less xp as number 1
Do you advice some tutorials to get started with databases?
I'm using postgresql and asyncpg. I have a two row table, one containing an id and the other a string. I need to find the string by id a lot of times. Would it be faster to cache them all into a dict instead of querying the database every time I need to get the string?
yh it would be efficient if you would query them once and then store them in like a dict for example , at least I think so, or maybe a pd dataframe @meager leaf
define „a lot of times“
Several times a minute for different ids
for a two-row table? definitely more efficient to cache
Hello! I'm using PostgreSQL and I want to upload the layout on my database for everyone to use. How would I do that?
yeah sorry, column
@quasi holly look for software that can reverse engineer the database and export it into sql commands
I know pgmodeler can do it, but you have to pay for it or compile it yourself for free
theres also pg_dump command if you are familiar with terminals
pg_dump dbname > outfile
psql dbname < infile
I will look into it, thank you!
make a —schema-only dump with pg_dump
level = await self.bot.pool.fetch("SELECT * FROM leveling WHERE guildid = $1 ORDER BY lvl DESC LIMIT 5", ctx.guild.id)
xp = await self.bot.pool.fetch("SELECT * FROM leveling WHERE guildid = $1 ORDER BY xp DESC LIMIT 5", ctx.guild.id)
``` im trying to get the xp and lvl and have it organized, but i still get it where a person that the same lvl but less xp as number 1
use a single query
SELECT * FROM leveling WHERE guildid = $1 ORDER BY lvl DESC, xp DESC LIMIT 5
doesn't work
gives me Command raised an exception: TypeError: list indices must be integers or slices, not str
i used the order by on both lines of the original code and it seemed to work
Guys how do I construct the following scheme with sqlalchemy.
A user can have a relationship to messages but messages will have senderID and reciverID (which will be two different users but still poiting to user messages colume)
for example
class Message(db.Model):
__tablename__ = "message"
id = db.Column('id', db.Integer(), primary_key=True, nullable=False)
senderID = db.Column(db.Integer(), db.ForeignKey('users.id'))
reciverID = db.Column(db.Integer(), db.ForeignKey('users.id'))
class User(db.Model, UserMixin):
__tablename__ = "users"
id = db.Column('id', db.Integer(), primary_key=True, nullable=False)
messages = db.relationship('Message', backref='user')
How scaleable is mongodb for a webapp backend?
@dull scarab as scalable as your code and infrastructure
I would argue the data 99% people use do not need to worry about scale but more about how their code is interacting with the database and following best practices outlined by their respective DB
unless you are a large government entity or a bank
just check the official documentation, also make sure you use "lean()" or serialize your data before retreiving it from your API
otherwise it will return the entire document
which will cause performance issues
also just look at indexes/etc if you are going to be searching through documents
im not a huge mongodb expert, but their docs are pretty solid
Yeah looks like it
How can i use the pubmed api to access data on structures?
i got it nvm
I need some help
with sqlite3
cursor.execute("UPDATE servers SET rankadmin = ? WHERE serverid = ?;",(role.id,ctx.guild.id))
how do i insert binary data into my table?
cur.execute("INSERT INTO post (binVal) VALUES (%?)", (bytes(something)))
what specifier would i put in place of the ?
Hello there.
I need help
I have import mysql-connector with the pip command but it's not working on this PC.
I have this error when I run it:
try mysql-connector
HI guys may i know any way to use python connect mysql database with ssh ?
No it doesn't work
bot code : https://github.com/Ekansh-kashyap/Bupfer
custom aiopg thingy is this:
import os
class aiopg_commands:
async def connect(self):
self.conn = await aiopg.connect(database=os.getenv('DATABASE'),
user=os.getenv('USER'),
password=os.getenv('PASSWORD'),
host=os.getenv('HOST'))
self.cursor = await self.conn.cursor()
# self.pool = await aiopg.create_pool(self.dsn)
async def execute(self, statement, args: tuple = None):
if args is None:
await self.cursor.execute(statement)
else:
await self.cursor.execute(statement, args)
error on last line
@void otter sorry for ping sir. can u help?
import paramiko
ip = "ipaddress"
port = port123
user = "aaa"
password = "password"
ssh = paramiko.SSHClient()
ssh.set_missing_host_key_policy(paramiko.AutoAddPolicy())
ssh.connect(ip,port,user,password,timeout = 10)
stdin,stdout,stderr = ssh.exec_command("mysql")
result = stdout.read()
print(result)
ssh.close()
HI all with this code i success to login the ssh but when input any command it will only print b" , any wrong i did ?
I have one row [Lith, A2, intact, Demokdawa, 4]
and another [Lith, A2, intact, kenexey, 5]
And i want to agregate row in a select to get something like
[Lith, A2, intact, demokdawa (4) kenexey (5)]
@valid hare b'' just means the program printed nothing on standard output
try printing standard error
@hazy crystal : This relationship worked?
Hello, I'm facing an issue with SQL, an error message : sqlite3.InterfaceError: Error binding parameter :Title - probably unsupported type.
And don't figure out why... There is the script about the table creation :
Postal_code text, City VARCHAR(55), Phone_number text, E_mail text)""")
Ans there about the insertion :
c = conn.cursor()
c.execute(("INSERT INTO formation_data VALUES"
"(:Trainer, "
":Date_formation_start, "
":Title, "
":Street_address, "
":Postal_code, "
":City, "
":Phone_number, "
":E_mail)"),
{'Trainer': a,
'Date_formation_start': b,
'Title': c,
'Street_address': d,
'Postal_code': e,
'City': f,
'Phone_number': g,
'E_mail': h})
conn.commit()
The title is a simple string
The Error comes up when I try to insert the information
Okay that was because of "c" already used for the cursor ... -_-'
Should definitely name your variables better
Yes ^^
@ionic pecan i just notice that have a odbc group selection, is this posible do with python ?
im not quite sure what that is
in general, computers can do anything if you work at it hard enough, so yes
Quick question :
I use this query
'''SELECT Relic.Name, Relic.Era, User.Pseudo, Relic.Quality, Relic.Quantity FROM Relic INNER JOIN User ON Relic.IDOwner = User.IDUser GROUP BY Relic.Name, Relic.Era, Relic.Quality'''
To "fuse" two lines from db
Basically these :
Lith, A2, 4, Demokdawa, Intacte
Lith, A2, 4, kenexey,Intacte
And it outputs this line
Lith, A2, 4, kenexey, Intacte
Can i make it output
Lith, A2, 4, kenexey(4) demokdawa(4), Intacte
Instead ?
Thx in advance !
Okay for future reference, the solution was using 'GROUP_CONCAT'
mongo_credentials = {
"host": "mongo",
"port": 27017,
"password": "censored"
}
mongo_client = AsyncIOMotorClient(**mongo_credentials)
db = mongo_client.main
@bot.command()
async def restart(ctx):
await ctx.send("The restart command is starting")
await db.contests.insert_one(
{
"id": ctx.guild.id
}
I tried to use MongoDB but I keep getting those errors (from the code shown above)
raise CommandInvokeError(exc) from exc
discord.ext.commands.errors.CommandInvokeError: Command raised an exception: ServerSelectionTimeoutError: mongo:27017: [Errno 11004] getaddrinfo failed
The above error is normaly thrown when no MongoDB server is available.
check that mongo db is available and listening would be first port of call then check connection info again.
Hello, what is the best way to check if multiple columns combined are unique? For example, a column can have multiple rows of same data, but 2 columns cannot have same rows.
I want to avoid rows with ids: 4,5,6,7
Im using flask_sqlalchemy btw
is this any help?
@hazy peak
I tried adding table_args = (UniqueConstraint('customer_id', 'location_code', name='_customer_location_uc'),) but it didnt help.
I solved it by querying the data first before adding it to the table
shouldn't you be checking uniquecontstraint with parent_id and child_id instead
but good to know it's been solved
that seems pretty easy
Sets of field names that, taken together, must be unique:
unique_together = [['driver', 'restaurant']]
This is a list of lists that must be unique when considered together.
from docs
How to delete the data from MongoDB database?
using aiomysql I want to catch errors, but there doesn't seem to be any documentation about it. Does someone know how I can do this?
@frozen osprey are you using a wrapper or calling mongo api directly?
@silent tapir are you looking to catch database errors or library errors or all errors?
database errors
https://stackoverflow.com/a/25026333 check this
because afaik aiomysql uses pymysql internally
so it should throw the same errors
does anyone know how to use update_one() with pymongo for objects
guys anyone using sequelize?
include: [{
model: gControl.Posts,
where: {
status: {
[sql.Op.or]: ['approved', 'announce']
}
},
attributes: ['author', 'permlink', 'url', 'status'],
as: 'posts'
},
{
model: gControl.Polls,
attributes: { exclude: ['createdAt', 'updatedAt', 'author', 'memberPollsID'] },
as: 'polls'
},```
escape to cancel • enter to save
this won't work if im using additional where in the model gcontrol.polls
this is for python database discussion, use the off-topic channels for other languages
Hi guys, I have a issue with psycopq2 since few weeks now, I don't arrive to install it, and don't find any solutions ...
There is the error message :
Any ideas ?
So I cannot import it from VS :/
I tried with :
```sudo pip install psycopg2````
I don't see why :/
Why not?
What's the error?
The errors you showed relate to installing the package, not importing it
No error PyPi is installed
That doesn't make sense. Are you saying that when you import it, it tries to install itself??
Just a second
So If I understand
in first :
pip install PyPi```` and then : pip install psycopg2```
right ?
What's this for pip install PyPi?
the psycopg2-binary not ?
No. That first command doesn't seem to do anything
Anyway, if you are trying to install from source (which is what pip install psycopg2 does), there are some prerequisites.
They are described here
Thank you @pure cypress , I'm going to take a look
I was on the link ^^, It seems I didn't understand well ^^
The file postgresql is not found on my path /usr/lib
Did you install libpq-dev?
Probably not
okay ^^
What is it ?
Error message as well when I install it from the terminal ...
ERROR: Could not find a version that satisfies the requirement libpq-dev (from versions: none)
ERROR: No matching distribution found for libpq-dev
yes
You need to use your system's package manager
Like apt-get, pacman, etc
What's your distro?
oh ^^
Hum ... The command is not found
Ah ok
So start here
Brew is a really big part of development on macOs
You'll probably be using it more in the future
Okay thank you for your help @pure cypress
OMG
I downloaded it
what a great feeling
Thank you @pure cypress
Now I'm going to try to import it
You never answered what happens when you import it...
I used brew to install "postgresql" then I "pip install "spycopg2", Done successfully. Then now when I Try to import it :
``ìmport psyco...```
The IDE doesn't propose psycopg2
not yet, just a second
import psycopg2
ModuleNotFoundError: No module named 'psycopg2'
I think that is coming from the path
/Users/anaconda3/lib/python3.7/site-packages
Visual cannot get access to because it's in anaconda3 ?
you think ?
Doesn't pip install install it for python2?
You're welcome
What i the DBAPI policy for database character encodings? I can't find anything within PEP 249 which indicates that there is one.https://www.python.org/dev/peps/pep-0249/
I'm using pyodbc to talk to MSSQL
i suppose this is the answer from pyodbc docs:
Microsoft SQL Server
SQL Server's recent drivers match the specification, so no configuration is necessary. Using the pyodbc defaults is recommended.
Hi @pure cypress are you there ?
Hi guys, I'm still stuck with this psycopg2 ... That drive me crazy. Now I have this issue :
The cursor is not coming up. Then I take a look on the library to get information, and then :
A new Error message, about an image not found...
psycopg2 is so bad to install and to get set to use ...
Any Ideas ?
To get this error message, I called the "help()" section
I reinstalled openssl and I installed libffi
It's still not working
psycopg2 is not „bad to install“, mac‘s „package management“ is just awful (you need to export 20 env vars for each package you want to link to), on linux systems its super simple
pip install psycopg2-binary already satisfied
And when I run again it, it's the same error message
:(:(:(
Did you uninstall psycopg2 first?
can you show the full output of pip install
Yes jus a sec
Okay actually my command didn't word
I did again and it's responding
just a second, I test now
Okay
so now the help section is responding, but I can't set the cursor :/
Like it's not found in the library
c = conn.cursor()
conn = pg2.connect(database='dvdrental', user='postgres', password=password)
r = """
--sql
SELECT * FROM customer
;
"""
c.execute(r)
There is my error message :
c = conn.cusor()
AttributeError: 'psycopg2.extensions.connection' object has no attribute 'cusor'
Weird right ?
I set the "connect" in the variable "conn" and then the cursor to the connect in the variable "c" :/
Okay it's working now
But why visual studio doesn't propose me the library ?
I mean when I start to write "cur", "cursor()" doesn't show up in the combox
Thank @ionic pecan for your help
the code you posted above doesnt work because it uses conn before its set
No idea about visual studio, sorry
Okay ^^thank you
Hey. How to update documents in MongoDB? When I use await db.contests_coll.update_one(...) it sends this error
raise CommandInvokeError(exc) from exc
discord.ext.commands.errors.CommandInvokeError: Command raised an exception: ValueError: update only works with $ operators
Hey guys using a flask/sqlalchemy setup. So there are a few things in my database that change incredibly rarely (once every 5-6 months) but are accessed frequently The total amount of records in this tables are about 1k or so. Is there a way / point to basically cache these sections of the database?
I've looked into both redis and dogpile but not sure if they meet my needs or are overkill
having 1k records and DB won't make much of the difference if you store it to redis or not
you can consider using local in-memory cache if that is possible (e.g if you have a single instance of app running) or you could also do caching with expiring within few minutes
i would say that if you are not sure about it then you dont need it
I just use a cache decorator for stuff like that
it depends on the way of accessing data
if you are using something like opengrpah it becomes complicatated
SELECT ID, FIRST_NAME, LAST_NAME FROM CUSTOMER WHERE (LENGTH(FIRST_NAME) + LENGTH(LAST_NAME)) <12 ORDER BY (lenGTH(FIRST_NAME) + lenGTH(LAST_NAME)) ASC, CONCAT(FIRST_NAME, ' ', LAST_NAME) ASC, ID ASC
im doing leetcode test ATM what am i ddoing wrong???
Hey. Does anybody use MongoDB? What is the problem here?
await db.contest_coll.update_many({"guild": message.guild.id},
{"$set": {f"ready{save}": "True"}},
{"$push": {f"answers{save}": answer}}
)
Traceback (most recent call last):
File "C:\Users\PC\PycharmProjects\Amino\venv\lib\site-packages\discord\ext\commands\bot.py", line 859, in invoke
await ctx.command.invoke(ctx)
File "C:\Users\PC\PycharmProjects\Amino\venv\lib\site-packages\discord\ext\commands\core.py", line 725, in invoke
await injected(*ctx.args, **ctx.kwargs)
File "C:\Users\PC\PycharmProjects\Amino\venv\lib\site-packages\discord\ext\commands\core.py", line 88, in wrapped
raise CommandInvokeError(exc) from exc
discord.ext.commands.errors.CommandInvokeError: Command raised an exception: TypeError: upsert must be True or False
On MongoDB website it says that upsert is default
and it's False then
import bitfinex
import pandas as pd
import numpy as np
import datetime
import time
import os
import MySQLdb
class Bitcoin:
def __init__(self):
self.limit = 1000
self.time_step = 1000 * 60 * self.limit
self.t_start = datetime.datetime(2019, 6, 19, 0, 0)
self.t_start = time.mktime(self.t_start.timetuple()) * 1000
self.t_stop = datetime.datetime(2019, 6, 27, 23, 0)
self.t_stop = time.mktime(self.t_stop.timetuple()) * 1000
self.bin_size = '1m'
self.data = []
self.api_v2 = bitfinex.bitfinex_v2.api_v2()
self.start = self.t_start
self.stop = self.t_stop
self.symbol = 'btcusd',
self.interval = self.bin_size
self.tick_limit = self.limit
self.step = self.time_step
try:
self.client = MySQLdb.connect(host='host', port=3306, user='user', passwd='passwd', db="btcusd")
self.cursor = self.client.cursor()
except Exception:
print("ERROR")
raise RuntimeError
finally:
print("Successfully connected to DB!")
def insert(self):
try:
main_query = "INSERT INTO data(time, open, close, high, low, volume) values(pair_data)"
self.cursor.execute(main_query)
self.client.commit()
except Exception:
print("ERROR")
self.client.rollback()
raise RuntimeError
def fetch_data(self):
start = self.start - self.step
while start < self.stop:
start = start + self.step
end = start + self.step
res = self.api_v2.candles(symbol=self.symbol, interval=self.interval, limit=self.tick_limit, start=start,
end=end)
self.data.extend(res)
print('Retrieving data from {} to {} for {}'.format(pd.to_datetime(start, unit='ms'),
pd.to_datetime(end, unit='ms'), self.symbol))
time.sleep(2)
return self.data
so guys how can I run this code, when I try to do, print(Bitcoin.fetch_data()) it give an error unfilled parameter self
so any ideas on running this thing>
You'll need to instantiate a bitcoin object.
im confused on how salting works
the way i understand it, passwords are hashed and the hashed password is stored, and when the user tries to log in, the password they enter is hashed and that hashed is compared to the stored hash
same.
but when the hash is salted i dont see how its possible to compare it to the stored hash anymore
if the salt changes each time i think it would be impossible to compare two salted hashes
unless the salt used for the stored hash is stored itself, in which case i dont see the point of a salt
That stuff is normally hidden for you anyway. Most web frameworks do that for you.
You just interact with User object the system created for you
If making a custom system for that you are probably in a bigger company and you would probably not be the person making it
In 2019 I would say it's irresponsible to customise that in most cases
@sweet wolf do you know how it works though
Hey. Why it's not working? I'm 99% sure it's good but it doesn't add anything to colours3: [] in my database.
save = 3
color = (await bot.wait_for('message', check=check)).content
color = {
'red': 0xff3300,
'green': 0x66ff66,
'yellow': 0xffff00,
}.get(color.lower(), 0x2c2f33)
await db.conest_coll.update_one({"guild": message.guild.id},
{"$push": {f"colours{save}": color}}
@oak pebble Yes. I've made a couple of hashers in the past at least. That was for a django project. You implement a hasher class: https://github.com/django/django/blob/master/django/contrib/auth/hashers.py
It was to support some very insecure password hashing from a a very old system made in php
Then django will upgrade the hashing type to the most secure one when the user logs on (The hashing method is store on the user)
This was coverting a phpbb forum to a Misago forum (hobby stuff)
For work/professional settings I guess most people deal with stuff like oauth2 (at least in larger organisations)
- oauth2
Totally never happened 😄
What's the best way to connect to a remote MYSQL database and insert queries via Python 3?
I meant do you know how the salting works
I don't think this belongs in #databases ...
In cryptography, a salt is random data that is used as an additional input to a one-way function that "hashes" data, a password or passphrase. Salts are used to safeguard passwords in storage. Historically a password was stored in plaintext on a system, but over time addition...
data = json.load(open("file users.json", "r"))```
How do i link to one data base?
as it does not fine that file
I think that you might need to do json.load("file users.json", "r")json.load(open("file users.json", "r")), totally correct, my fault, better look at https://docs.python.org/3/library/json.html
Traceback (most recent call last):
File "C:/Users/conno/Desktop/Nite Smells/idlebot.py", line 6, in <module>
data = json.load("file users.json", "r")
TypeError: load() takes 1 positional argument but 2 were given```
😮
Traceback (most recent call last):
File "C:/Users/conno/Desktop/Nite Smells/idlebot.py", line 6, in <module>
data = data = json.load(open("file users.json", "r"))
FileNotFoundError: [Errno 2] No such file or directory: 'file users.json'
>>>
is file users.json in the same folder as your script?
There is no such file, simple.
yes it is
Try using the absolute path for now
how are you running your script?
idle pyth
CWD needs to be the directory your file is in, I think
??
Current working directory
but try C:/Users/conno/Desktop/Nite Smells/users.json
what insted of just users.json?
^
Why should you write file in front of it if your actual file was named users.json ?
open("users.json", "r"), for the record
Does it work now ?
^
oh my
it was there twice buc of my keyboard
it always double pasters
ok now it is a single
@lucid fulcrum salting is used to store things in databases
Yes
it should only be data = json.load
you can't do that haha
i believe you can have multiple comparisons in the same line
really?
yes
>>> a = b = 5
>>> a
5
>>> b
5
its not that data = data = json.load doesnt work, its just redundant
deniz im 100% sure
your the one that told me to put data=data
same error
>>>
=========== RESTART: C:/Users/conno/Desktop/Nite Smells/idlebot.py ===========
Traceback (most recent call last):
File "C:/Users/conno/Desktop/Nite Smells/idlebot.py", line 6, in <module>
data = json.load(open(" C:/Users/conno/Desktop/Nite Smells/users.json", "r"))
OSError: [Errno 22] Invalid argument: ' C:/Users/conno/Desktop/Nite Smells/users.json'
>>>
@sudden solstice
were?
right at the start
the first character
' C:/Users/conno/Desktop/Nite Smells/users.json'
before C
yes
