#tools-and-devops
1 messages · Page 50 of 1
The Resources page on our website contains a list of hand-selected learning resources that we regularly recommend to both beginners and experts.
okaayy thank you m8
Keep in mind that pycharm, like all editors, has a learning curve. Pycharm however will have a lot of features that you don't need yet, more than most. They're just going to make it harder to use/learn.
Something simpler like atom or vscode might be a better fit.
how do WSL1, WSL2, docker for desktop, and just plain old vagrant stack up against each other for webdev on Win10?
I'm stuck on Win10 for the foreseeable near future and I'm trying to figure out how to make the best of it
also I have to use a slow HDD instead of an SSD, so I'm a bit worried that vagrant / VMs might be painfully slow
there's little reason not to use docker containers
Vagrant takes a while to boot up, it's convineient and such but docker is the future
basically a lightweight vm/rich process, with very little overhead
a rich caching mechanism, and preety intuitive declarative syntax
i recommend giving docker a look
ngl i just use cmd
dockerfile is for deployment imo
i dont want to have to rebuild a container every time i add a dependency
you don't have to if you cache it right
how does that work
if you have an example id be interested. for environment isolation i use conda mostly, but i'm also a data scientist so that makes sense for my uses
@lone linden i wasn't asking about vagrant for packaging apps, I mean as a docker host. Sorry, I was vague.
and when im doing web dev i mostly am on linux so there i'd use nvm to switch node versions. idk if there's a windows port/version of nvm
i do web stuff very rarely anyway
tbh id rather use docker than conda. solves a lot of problems and avoids a lot of complexity
does anyone here have experience with tensorflo
I think with containers you can define a tower of image layers for groups of dependencies, if i remember right
just ask your question @heavy knot , don't "ask to ask"
does anyone here have experience with tensorflow? I'm trying to train my own ai with my own images and datasets but every guide out there tells me to use their own datasets
for flowers and cars and stuff
ask again in #data-science-and-ml 🙂
thank you
i'll try to find a decent example later, should stop discording while at work 😄
same lol
that said this is pretty relevant to my work at the moment
im involved in a big project overhauling my team's workflows
fine tuning docker setups definitely makes a huge difference
fit for a position of it's own
Since we're using Dask at the core of our compute we build off the daskdev/dask Docker images which make it very easy to manage dependencies and supports an environment.yml file. Only downside is the dependencies are installed after start up. Been looking into a shared volume to store downloaded packages.
But we also have a local pip/conda repository installation on our network so at least the downloads are quick.
i forgot 95% of the things at my local devops course
but i still learned more then half the other courses

yeah @floral shale thats kind of my question. what happens if i want to add something as trivial as tqdm to my docker image?
do i now have to rebuild the whole thing?
@tawdry needle depends. If you're currently still testing/developing and it's just you and the container? No. You can manually put it in there.
afaik, if you built your image right, no
all the other dependencies are cached, the new one is downloaded
isnt building a container itself often kind of slow
The initial image layer downloads can be a bit depending on your internet speed.
^, bandwith is the factor afaik
hm. i'll have to experiment with this
as i mentioned, conda has a lot of associated annoyances
can someone explain why do we use docker? it seems like a virtual environment does the same thing as docker..
@bitter vine much, much lighter than a virtual environment
@floral shale so you dont need a venv to upload your project to docker?
wouldn't agree that it's lighter, it's basically a light VM, without the operating system
Those things are not intrinsically related
but i'll butt out, I NEED TO WORK
@lone linden it leverages your kernels native system calls instead of mocking the entirety out.
on linux, yes ;P
so docker pretty much bundles up all your project for all OS.. is that right?
Correct. On Windows it runs a virtual machine to accomplish the same thing which is not lighter.
@bitter vine that's pretty much it, yeah.
ahh okay, thank you
No problem. Happy to help.
noone said they can't
Windows Containers don't require HyperV
I think they're referring to @bitter vine's portion of "for all OS" -- I misread that bit.
I'm just saying, Windows containers exist, on Windows, it doesn't run VM unless you go with Linux containers which obviously then require a VM
Ah, gotcha. Last I heard that's how it was accomplishing it but I am much more Linux involved than Windows.
I run like 2 Windows Containers, and about 40 Linux ones
but Azure (and possibly others) now support Windows nodes under Kubernetes control
Guys, would anyone please recommend a place to host cheaply or for free a twitter Python bot made with Tweepy please? 🙂
i have no idea what tweepy is
but
have you used AWS or Azure before?
@ivory barn
they have free tiers that let you run a single tiny vps free for a year, i think
i wouldn't trust it to handle real heavy load, but a VPS should do it
Thanks for the quick response @rough marlin, also would you reccomend some library for making a Twitter bot? Because Tweepy is exactly that and might be a little dated.
i have no idea. i had heard that twitter was trying to be as hostile to developers as possible
yes they have one, but they kept chopping off features and restricting stuff
or at least that's what i heard
i don't bother with their api though so
i may have heard wrong.
anyway
That is completely reasonable and understandable from the company's perspective and we are developers. Last time I checked we thrive on limited circumstances. 😄
a VPS is probably what you need
What are you into (development-wise) @rough marlin
uhhhhh
thats a complicated question
the stuff that interests me isn't stuff that makes money
emulators, esoteric and absurd language stuff are sort of interesting to me lately
i've worked on automation and scraping a bit in the past
scraping in the much more distant past
all of that was boring internal projects
its not a very rare issue to have the stuff you're interested in be very different from the stuff that pays bills
lately i'm considering getting ready to run away from the GUI forever and live in the terminal. That's sort of preparation for eventually doing more with forths, lisps, and strange, minimal languages that fit into tiny spaces.
I love this community and the fact that there are people like you in it. That is very interesting @rough marlin and since you find esoteric programming interesting, you may find Quine (programs that output their own source code) computing interesting. 🙂
to bring this channel back on topic, powershell is weirdly responsive on windows
i don't understand how it has zero input lag or lag of any kind while the rest of win10 feels like cold tar
oh yeah, is powershell just an advanced version of command prompt?
I don't believe so. I believe it borrows from the notion of Linux where values can be "piped" between processes, unlike Command Prompt
that sounds really advanced to be honest
it's both a different language with new features, and what seems to be a new rendering stack for console
let's say i have
big-dir/
git-repo1/
git-repo2/
what's the sanest way to turn big-dir into a git repo, and set git-repo1 an git-repo2 as submodules thereof?
seems like the answer is to git git init big-dir then git submodule add each of the git-repo*s from their remote URLs
not sure what happens if the git-repo1 directory already exists though ...
so not sure
im setting up a server on nginx
umm my question is regarding http and https
setting up nginx is a bit complicated
searching the docs requires some basic knowledge of how everything works
can you explain how you want to set it up?
then maybe we can help
it's also complicated because nginx configuration is sometimes different on different OSes
all of my nginx experience is on debian for example
Can I get an opinion on what is best practice in a following scenario:
- Every version I release can be toggled (enabled/disabled) which allows users to be able to upgrade to that version or not
- Released version 1 - enabled
- Released version 2 - disabled due to major bug found
- Released version 3- enabled
Now when user upgrades from version 1 to version 3, they only see release note of version 3. Should version 2 release notes also be included in version 3 release notes?
good question. maybe just link to the ver 2 release notes
Ok that's not a bad idea. Just subtly hint there was a version I skipped.
Powershell is object oriented scripting language
yes it is
@tawdry needle i figured it out, I got caught up in something else and couldn't finish my question
but i got it working
kubeadm, kops or kubespray for someone new to kubernetes for a small kubernetes cluster on 8 machines (playground)?
oh kops is only an AWS thing, then that's out of the picture
i'll just go with kubeadm
Kubeadm generally, but for real production stuff, try Cluster API
I tend to use a combo of terraform and ansible, but ClusterAPI will be taking it's place in my toolbox as soon as I get around to automating it with a terraform provider
so i have a flask app that i built on my computer
and i am following this guide https://www.digitalocean.com/community/tutorials/how-to-serve-flask-applications-with-gunicorn-and-nginx-on-ubuntu-18-04
im just curious, since I already have a venv on my local machine, id still have to make one on the server im sure
would I create a requirements file and port it over to the server?
Requirements file should be in git repo
yes that is an option
option? not really
lets argue!
Of course it should be in vcs
Hi, I've finally started to learn how to use git a couple of hours ago, and I'd be really grateful if anyone could answer these questions:
When you create a branch, does git automatically create a new copy of your code?
Also, is it possible to merge the master copy into a branch, and if yes, is there a way to undo it?
Thanks in advance!
Anyone here uses Spyder IDE? Specifically Spyder3?
i havent used it in a while. still waiting for jupyter kernel support...
I'm trying to work in the "dev" branch of a repo I wish to make a pull request on. However, when I do "git check dev" it seems to create a new dev branch completely.
Can anyone help with vscode? All my conda envs disappeared
Even though this one already exists:
@calm tapir do git checkout origin/dev
@river sinew Ah thank you!!
Another stupid Git question. If I make changes and commits to a local branch called "fix-issue51" and I want to create a pull request to merge into the dev branch, can I simply do "git push origin dev" or do I do "git push origin fix-issue51" and then adjust the branch I want to merge it into on GitHub.com when making the PR?
just do git push and make the pr on github
Trying to learn some software architecture and was wondering if this is a viable path for a python API service.
My other confusion is the static content. I have a react app which I will most likely host on S3, so does that mean I still need Nginx if it's only being used to point to a machine with docker swarm manager? I was thinking I needed to load balance this as well which is why I was thinking of nginx
bleek, what's with all compute engines?
and docker swarm?
No, this is bad design, if you are going to containerize everything, use Kubernetes

If your design starts with EC2/Azure VMs/GCP Compute Engine, you might be making mistake
even then, why not use GCP Functions instead of Flask Microservices
hi @wooden ibex , sorry it was confusing that i said S3, i meant GCP bucket. From my initial understanding of docker swarm, it needs 3 nodes minimum
1 for the mananger and 2 for balancing
Could you elaborate on the mistake part? Should these be in a single vm?
or avoiding VMs all together and use the managed service they provide?
@rugged spoke Could you elaborate on the mistake part? Should these be in a single vm? You should never start any design with VM. That's most expensive way to run anything in the cloud in both compute costs and management costs.
avoiding VMs all together and use the managed service they provide? Absolutely, there already a "docker swarm" system available, it's called Kubernetes. That being said, why docker? GCP/AWS/Azure provide this service called Cloud Functions/Lambdas/Azure Functions where you expose REST endpoints. There is no reason you need Flask involved at all. Bonus, you only pay for execution so they can be crazy cheap. We use Azure Functions all the time with AngularJS SPAs
@wooden ibex
why docker
i'm already using docker on local development and it was easy to integrate with the CI/CD system i have. push to registry -> pull on server. It also allows me to keep dependencies separate for each of my services (but maybe serverless could do this?)
why flask
its a full on webservice with multiple modules within it. I never played with cloud functions but it sounds like I would need to decompose the application into many serverless APIs
k8s
I suppose I avoided k8 as I keep reading it adds complexity where I don't need it unless I'm Google Scale
Wouldn't k8 be just as expensive since it would involve spinning up VMs for nodes? I just feel like im missing something. Any resource you can point to that I could read more on this? I feel like i'm overcomplicating things
I'm currently avoiding all of this and have a single VM with docker-compose to run my app
kubernetes handles failover and bunch of other things
What does k8s failover do beyond docker swarm?
lightmode is trash
To each their own @pulsar nest, I generally use a lighter theme myself
What does k8s failover do beyond docker swarm?
@floral shale what part are you talking about? Application high availability? Cluster high availability? Lots of thing in k8s could be labeled as a fail over mechanism lol
There is also blue green / rolling deployments, load balancer, etc...
Wouldn't k8 be just as expensive since it would involve spinning up VMs for nodes? I just feel like im missing something. Any resource you can point to that I could read more on this? I feel like i'm overcomplicating things
@rugged spoke this depends a lot on how you run the cluster. EKS for instance can run with fargate containers now I think as well as a VM backed cluster
Guess that depends on the organization. Most I've worked for had clusters that preexisted managed offerings on their platform of choice.
That is starting to dwindle though
And I'm still not sure if AWS govcloud has EKS or not yet
Is there any relevant corp out there that uses an alternative to git? Had a talk with a recruiter that hadnt heard about alternatives like SVN, Mercurial,...
just curious
Mozilla still uses mg I think
But they are a dying breed
I haven't seen subversion in a long time
But I'm sure they still exist out there
you mean mozilla is the dying breed or alternatives? Guess 2nd, just want to make sure
Yeah second, Mozilla isn't going anywhere lol
Sorry @river sinew , I don't use conda
Reboot? Lol
Tried that already
Also tried reinstalling vscode python extention
Next is anaconda if no one has a better idea
Not that I can think of
@gusty willow I've just never needed anything other than Docker Swarm and sometimes Nomad I guess.
@floral shale when you really really need to have distributed applications available, k8s is the way to do it. Swarm is ok, but even docker says to use k8s lol
As it turns out, writing a container orchistrator is hard lol
I've just never had a problem with availability.
I'm not saying swarm is bad. But when you get to the hundreds or (hundreds of) thousands of containers, swarm breaks down
We've a damn Clearcase install for a legacy system. A seated, kernel module VCS. 😭
@floral shale nice to hear someone out there is still using swarm. I have a small project and k8 seemed intense for what I was wanting to do
Great knowledge people! Can someone explain to be what PATH is?
Specifically for a program like Visual Studios? 🙏 Please and many thank yous!
PATH is a list of directories that will be checked when you run a program unqualified
so for example if your PATH was "/home/foo/bin:/usr/bin" and you tried to run banana it would look for /usr/bin/banana then /home/foo/bin/banana then tell you can't find banana
works similarly on windows, though the paths will look different
I see I see, so Mister Gary would PATH be necessary no matter what then?
no, you can run your command by specifying an absolute or relative path
I apologize, It's a confusing question because I'm not educated on stuff associated with this
oh
in my example if i was in the /home/foo directory i could run bin/banana
that's a relative path
or i could run /home/foo/bin/banana from anywhere
that's an absolute path
so the list of directories, would that be from an open source?
the PATH environment variable just adds directories to be search when you run some command without specifying a path
i could be from anything, doesn't have to be open source
just some executable
oh! So it's like "auto correct" in a way
it gives you "recommendations"
for what it thinks you might want
well it just runs the first one it finds
lmao
doesn't give you reccomendations 🙂
okay, I'm starting to understand it better now!
Thank you for the enlightenment

If I have a Vue.js app which connects to a Django backend through REST API, can I deploy it as a single app on Heroku or I'll have to deploy them as two separate apps?
If the former is possible, then can someone guide me to a tutorial or something for the same?
vue.js app runs client side. what would you deploy separately to heroku for that?
just put your vue.js app where you server all your other static files from
No, vue.js running through NPM as an application. Like, where I have to do npm run serve to access my Vue.js app
you sure that's just just running a webserver? this is for a frontend app run in the browser, correct?
Has anyone used onivim(2) or neovim?
I want to focus on and get really good at a universal set of keymaps, so I started learning vim a few days ago. I was previously using a mix of atom + vscode + custom keymaps, which wasn't transferable anywhere.
Im loving the vim experience, im just curious how closely the vanilla vim keymaps transfer over to onivim and neovim.
you sure that's just just running a webserver? this is for a frontend app run in the browser, correct?
@half sentinel
Vue.js app runs it's own server on a separate port to serve the front-end application
yeah but i imagine that's just for development ergonomics
it's just static files being served to the browser
you could serve them from any webserver
I don't know what you mean by static files, it's a reactive app, same as react.js
yeah, i'm familiar but it's just html and js files being served. the browser is doing the interpreting
The vue-cli-service serve command starts a dev server (based on webpack-dev-server) that comes with Hot-Module-Replacement (HMR) working out of the box.
even the docs say it's just meant as a dev server
Oh, ok. So basically I don't know how to deploy a Vue.js application in production
you take your build files and can serve them with nginx, cache them in a cdn, etc
you could even serve them straight from flask
just drop them in a static directory and flask will serve them automatically
though depending on your scale you probably want to put them behind a dedicated webserver or cache them in a cdn
it's just a bundle you need to get delivered to the browser somehow
Ok. I'll read up a bit. That was very helpful, I know very little about front-end applications, so don't know anything about deployment
no problem
Vue/react/svelte --- they're all the same. You need to do the production build and ship that minified code to whatever web host you want ... Most folks are doing netlify as an easy start
Otherwise, you will be managing a VM somewhere and doing all the nginx/Apache configs yourself (although, I do recommend doing that at some point lol)
I've done it with S3 buckets in the past as well
Can I deploy it on Heroku with my Django app? @gusty willow
Probably
i'm not sure exactly how that would work ... i suppose you would just have the production stuff in there with your other static assets ... but the it would be hard to drive any django dynamic content like that because the html template variables are going to get washed out by the JS build
i normally see them as two seperate deploys with and API tween the two
I'm not using Django templates, only django models and API as backend. The entire front end is in Vue
have you shipped anything yet?
if so, just follow that process. Otherwise, I think you would be better off with two deployments
mostly because the runtimes are going to be different
WSGI for Django will be the one entrypoint and i don't think multiple entrypoints can coexist on a heroku app
should my project files be inside my venv folder
or should my venv folder be inside my project folders
im trying to deploy to cloud
@sand thistle the latter
my-project/
src/
tests/
venv/
.gitignore
readme.md
setup.py
and of course you would exclude the venv files from any deployment or packaging
OK. I am just trying to understand how to do it. I haven't deployed any front end so far, only one a very small pure Django app on Heroku.
have you shipped anything yet?
@gusty willow
Any clue why (in a conda environment) that pytest and py.test would function differently?
I have pytest-cov installed and, for some reason, running pytest it doesn't see it, but py.test does?
@spice pulsar you should have an html and a js bundle from your webpack build i imagine, you just need to drop that in a static directory and make sure your html refers to your bundle in said /static/bundle.js or whatever it's named
i have an app that uses an exe whos folder is listed on my env path. so when i want to make a docker image of my app, do i have to bring the exe into the docker container or can images in a container still access env path
Can anyone help me remove these
they are useless almost all
It happened after typing launch os
i have an app that uses an exe whos folder is listed on my env path. so when i want to make a docker image of my app, do i have to bring the exe into the docker container or can images in a container still access env path
@heavy knot you would need the exe, docker is isolated-ish from the host
@heavy knot you could do a volume mount to see the exe, but that comes with it's own things lol
and if the exe also has some dependencies like dlls i would also have to bring them in?
yup
in general, windows containers are no fun lol
unless the DLL is a system level dll, you would need it
how would the app search for the exe
i tend to use kata containers these days now, so I get the added isolation of a virtual kernel, but that adds even more complexity (but also, i'm able to really trim the attack surface all the same)
what do you mean "how would it search for it"? in your path variable like everything else
and/or use full paths (which is my recommendation)
oh so u would have to make your own path variable in the container
yeah, if you don't want to call it from where it actually lives
or i guess, rather, you maybe don't know where it lives
in which case, that is a different problem lol
luckily though, in a container, you have full control over those things lol
what is it that you need to run in a windows container?
9/10 things i see people trying to to in a WinContainer can be much more easily done in a linux container
most .Net stuff runs just fine on .Net core that can live in a unix container now 🤷
the pure windows app might be the sole use case, but running those in a container is probably not what you are after lol
given the lack of a UI and all lol
only time i come across them now-a-days is for building windows apps in a CI pipeline. Granted, I'm no windows dev so I really can't speak for them
So, I'm trying to run a simple print program on sublime, and it says Python is not recognised as an internal or external command
Any ideas on how to fix this?
It must not be in your path or Sublime doesn't see it. Open a cmd prompt/terminal and type in python.
If it fails then it's not in your path. Otherwise sublime doesn't see it.
Run the python installer again and make sure the "add to path" option is enabled.
Thank you
i think it worked?
ill let you know
Still doesn't seem to be working
I'm quite confused on how this works still @covert kindle
which is failing? CMD or just sublime at this point?
so when you cmd, you can just type "python" and you get an interpreter?
when i run cmd i type "py" and it works
but when i run sublime it doesnt seem to work
@gusty willow im so confused
sounds like you just need to select an interpreter
I'm kind of neither here nor there on LSPs
Spotty connections make them worse than useless
!ask
Asking good questions will yield a much higher chance of a quick response:
• Don't ask to ask your question, just go ahead and tell us your problem.
• Don't ask if anyone is knowledgeable in some area, filtering serves no purpose.
• Try to solve the problem on your own first, we're not going to write code for you.
• Show us the code you've tried and any errors or unexpected results it's giving.
• Be patient while we're helping you.
You can find a much more detailed explanation on our website.
If it's stuck, w40. If it's loose, duck tape.
I'm working on a windows system, so Powershell became my go to console. However I find it fairly annoying that I need to have a window opened for each notebook instance /flask app I got running. On Linux I used tmux, simply to spawn a session, spin up the notebook sever and disconnect. I've googled for tmux alternatives, but they are mainly concerned with this whole window splitting capabilities of tmux, which I don't particularly care about. All I care about is decreasing the number of windows I've got opened.
@lyric bobcat https://eugeny.github.io/terminus/
Hello I have a graph with lots of edges I want to visualize (I used matplotlib but it’s lagging)
Do yoy have a module or any other application I can use to easily visualise my Graph ?
Btw I use networkx
@frank compass Thank you!
@lyric bobcat
I use GitBash for everything. You can push jobs to the backhround to be like daemons.
Plus you get the unix commands, feel, and bash navigation.
@lyric bobcat go with Microsoft solution https://github.com/microsoft/terminal
Fishy sezer, pushing jobs to background? You mean like Powershell Start/stop/get-job feature
Don't know, I've never used powershell. Just basic cmd.
ouch
Has anyone set up pipenv on Git Bash? It seems to work, but when I type pipenv shell the shell starts but doesn't show the virtualenv name the way a normal venv or conda env does
^not sure. I've got a question: my primary IDE is Spyder + Anaconda - I've installed all the requirements for that -> switching to Atom + Hydrogen I needed to do a pip install ipython - does that mean I have an entirely separate installation of 'Python'? I'm sorry, I'm relatively new to this
you should be able to have Atom use the version of python that came with Anaconda. If you still want to use conda, that's the way to go
I use virtualenv on GitBash. No problems, but I did make a shell script that colorizes the name. It was defaulting to the same color as my cwd.
You were told that was against the tos in #python-discussion and yet you asked here and #async-and-concurrency, both channels completely unrelated to your question...
ok, why is it against ToS?
@half sentinel why is it against ToS, i've never heard about it
that's not really on topic here and neither is your question. if there's not an appropriate topical chat you can use #❓|how-to-get-help to learn how to get a help channel.
I JUST GOT INTO THIS SERVER LESS THAN 2 HOURS AGO, EXCUSE ME FOR TRYING TO ASK A QUESTION
i'm telling you where to ask it
your question doesn't have anything to do with async or devops
there's a whole system for getting a dedicated help channel
and i'm supposed to already know this before i get into the server?
thanks for being so helpful
i'll go ask where i should
well yeah, it's in the welcome guide
well yeah, it's not completely clear to start off
okay well now you know. sorry for any confusion
@misty pilot I'm not sure what your original question was, but @half sentinel is right that we cannot help you with anything that would violate our or another party's terms of service, as per our rules
Each channel has a topic which explains what the channel is for, for questions relating to the community itself feel free to use #community-meta
they deleted their messages, it was about stock market bots
not a bot, just a screener/scanner to watch the charts and what not, of which i will use the api provided 🙂
i'm not super aware of why they're against tos personally but i saw it mentioned in general
Alright, thanks ~ in the future if you see something like that, it's preferred if you let us know via @balmy mulch, I'd hate to have you guys start fighting over it
@misty pilot if your project does not violate any terms of use then feel free to make use of our help channel system as gary pointed out
otherwise, welcome to the server!
where does it show tos because other people are making them.... if it is then they're all breaking tos
i'm sorry, i'm not tryign to be a knucklehead, but this is the first time ive ever heard it's against tos
I'm not sure which service your original question was relating to
ok, so i'm just trying to create a stock market scanner, to show the trend flows to be able to pick which stocks to invest in. there are others out there, was wanting to make my own instead of pay out the rearend for one
not an automated bot to buy/sell for me, i'll do that on my own
i can't speak to whether that is or is not against the tos, my annoyance was when someone told you it was against the tos you moved to two different unrelated rooms to ask the same question rather than get clarity
it's like when you get in trouble with mom and then go ask dad for permission behind her back
but also just wanted to point you to the correct way to get help
it showed an intention to ignore and circumvent rather than understand a concern raised
ok, the feeling i got when i asked the first time, was someone hitting me over the head with a ball bat
Alright I think we're going off topic for this channel now
very much so 🙂
feel free to continue in #community-meta if you have questions regarding what is and what isn't okay to discuss here
I just bought 2x14tb WD HDD's. Before I backup my life, I wanted to test them for bad sectors or any errors. I've never done it though, so does anyone have software they recommend? (Windows)
@covert kindle https://crystalmark.info/en/software/crystaldiskinfo/ this is what i usually use
About CrystalDiskInfo A HDD/SSD utility software which supports a part of USB, Intel/AMD RAID and NVMe. Standard Edition Shizuku Edition Kurei Kei Edition Download System Requirements OS Windows XP/Vista/7/8/8.1/10Windows Server 2003/2008/2012/2016/2019 Architecture x86/x64/AR...
I'm about to release an open-source project with an associated documentation. I wrote it with sphinx but I'm still hesitating: Github pages or Readthedoc, what is the best ?
Whatever is easier for you probably
RTD is nice because you can host multiple versions simultaneously
Whereas github just uses whatever is on your master branch
ok. And I think that github page is always working with an index.html and a static repository content ? It doesn't automatically compile the sphinx doc on changes ?
hey, how do you bypass writing sudo with docker everytime??
i read about making a user group that has the same privileges as admin but i do not want to do that
any other ideas on this?
i have a question
Go ahead @astral plinth
@orchid dock did you not add yourself to the "docker" group?
nvm i found a solution
👍
I have a question if anyone can help. I have the root folder of my project set as the source root in pycharm. When I try to import from the source root in a child module it looks starting from the child module. I'm trying to import the stonks (the root) folder but pycharm seems to think that I want to import from test.webserver.stonks Does anyone know how I can fix this?
That's one of the problems of pycharm, it tries to be too smart
I can't see your screen well enough on my phone, so I'll have to jump to a laptop to help.
@orchid dock did you not add yourself to the "docker" group?
@gusty willow i know about that workaround but it's not secure. Do you know another workaround?
@orchid dock it's not secure because anyone able to run docker can start containers with root privileges with docker run --privileged, which would also be the case if you ever find another way to bypass using sudo to run docker
there is this experimental feature you can look at otherwise: https://docs.docker.com/engine/security/rootless/
@eternal flicker yeah, saw the rootless mode but it's still in testing
@orchid dock that's not a work around, that's the process lol. Docker is simply not a secure solution. If you want secure, check out kata containers.
Would this be the appropriate channel for asking about git?
Yes
I realised those messages weren't clear whatsoever
My project structure is:
FSO > part1 > courseinfo > files, including a .git folder
I'd like to push the FSO folder to github
As in the future i intend on adding a part2, 3, 4, etc.
the .git folder inside 'courseinfo' was automatically created when I ran create-react-app so I can delete if neccessary
Yes you can delete the extra git @tired rivet
Unless you have already pushed commits from it
It makes things a little rough to clean up because remote states won't match local states
@gusty willow thank you, ill give it a go now and see what happens
So im going to delete the exiting git folder, and then run git init within the FSO dir
Then im going for git add . from the FSO dir
And that is generating a lottt of warnings
"LF will be replaced by CRLF in part-1/node_modules/..."
"The file will have its original line endings your working directory"
seems to be doing this for every single file in the node_modules dir
Node modules should be ignored
Is there a way to hide certain folders from a project in pycharm?
suppose i created a project with pipenv and i had python 3.6 back then
now i have a new machine
and i saved my projects on a usb stick
now i remove my projects from the usb and put them on the computer
i install pipenv
and run pipenv shell in the project folder
i get this output
Neither 'pyenv' nor 'asdf' could be found to install Python.
You can specify specific versions of Python with:
$ pipenv --python path\to\python```
But i don't want these projects to use Python 3.6 anymore
I want them to use 3.8
how do i do that?
that did it
I'm not sure if this the right channel. But is there any conda command along with a file that I can use to tell which environment to use for that particular project. Like with node.js' nvm where there is a command nvm use which will activate the node.js version that is set on a file name .nvmrc which contains the node.js version use on that project.
@ebon spire sadly no, you have to use 3rd party tools for that like direnv
Would be a very nice feature
Pyenv has that ability as well
Does anyone here know where I can find documentation or the source for handling install_requires that use the VCS format package-name @ git+... ?
warning: Pulling without specifying how to reconcile divergent branches is
discouraged. You can squelch this message by running one of the following
commands sometime before your next pull:
git config pull.rebase false # merge (the default strategy)
git config pull.rebase true # rebase
git config pull.ff only # fast-forward only
You can replace "git config" with "git config --global" to set a default
preference for all repositories. You can also pass --rebase, --no-rebase,
or --ff-only on the command line to override the configured default per
invocation.
This message seems kinda vague - I'm not even sure what the definition of a divergent branch is - or which are diverging
@ebon spire @tawdry needle in zshell there's a thing for that, or is it just part of virtualenv? You have a .venv file in the repo root and it activates it for you
@topaz aspen there are zshenv and zsh-autoenv but they are still 3rd party tools
Also "divergent" just means "anything you can't fast forward merge"
It's saying that it wants you to specify in your config if you want to merge, rebase, or ff-only by default
Is there a way to completely ignore local changes to a tracked file with git?
@tawdry needle ah, fair - i wasn't sure whether the zshell was wrapping something that was built in or doing the work (re the .venv thing) guess it's the latter.
i'll have to read up on the git thing, usually just works, damn.
@finite fulcrum like permanently ignore?
as in "never stage or commit changes to this file"?
Yes
@topaz aspen this was a change in git sometime last year, they wanted to make sure people are being explicit about their preferred behavior. you can just leave it alone if you dont mind the warning message every time 🙂
and to be clear, i have no idea what tool uses .venv but it's not zsh itself
you probably installed something and forgot about it
@finite fulcrum there's now way that i know of to do this built into git. you can probably write a smudge/clean filter for it, or a separate shell script that reads "do not update" filenames off a list and runs the relevant commands
https://git-scm.com/book/en/v2/Customizing-Git-Git-Attributes explanation of git filters
in fact i think what you're asking for is not generally supported by git:
https://git-scm.com/docs/git-update-index#_notes
Users often try to use the assume-unchanged and skip-worktree bits to tell Git to ignore changes to files that are tracked. This does not work as expected, since Git may still check working tree files against the index when performing certain operations. In general, Git does not provide a way to ignore changes to tracked files, so alternate solutions are recommended.
For example, if the file you want to change is some sort of config file, the repository can include a sample config file that can then be copied into the ignored name and modified. The repository can even include a script to treat the sample file as a template, modifying and copying it automatically.
how can i git clone sherlock for python?
i tried git cloning and it said syntax error
Guess I'll hope I don't tick it in pycharm then
@finite fulcrum what file are you trying to ignore changes to?
pycharm might have its own functionality built on top of git
A docker compose script I needed to change for it to work locally
@tawdry needle ideally nothing to that file
So if the remote updates the file you ignore the change?
I think your only option is smudge/clean where you replace the tracked version of the file with your own version on checkout
@topaz aspen https://asdf-vm.com/#/ maybe one could write a conda plugin for this?
An extendable version manager
I'm trying to escape conda XD
Only other option seems to be docker tbh
i read someone using conda like docker actually
it was on reddit somewhere, they made a fairly sound argument for it
though I can't remember it, as I've not used it for that, and not used docker either
The whole point of conda is to be a binary package manager for things beyond just python
e.g. python libraries with compiled c extensions
well i guess they were one of the few that actually got the point then ha
you'd be surprised
once upon a time installing numpy on windows was a shitty experience
conda had it figured out before numpy devs were shipping wheels for windows
and with conda you actually get to control which compiler version was used to build the package, which is not true for binary wheels
the dependency resolution is much more strict and precise in conda than pip
On your dev machines, do you like install postgres, redis, rabbitmq, etc (really any external service for your project) either on your dev box or dockerize those things?
oh - i'm not surprised by that - though i use unix/mac/linux or whatever, i'm aware that conda provides a good resources for windows users (esp in cases where admin permissions are locked)
conda had a long rough start to where it is today, but now its a much more capable tool for reproducible research environments than some ad-hoc stuff put together with pip
@vague lion depends what you need them to do
(I am happy about the new work going into pip and its new dependency resolver.)
@tawdry needle what about something put together with pip, with requirements files?
the point of conda isnt that its good for windows, the point is that it ships language-agnostic binary packages, which can include "software" like gcc and libssl etc
pip only covers python packages
need to compile against libblas? download it yourself or hope they ship a wheel
@vague lion I tend to run everything at least in a docker composition to make sure networking junk works well, but you need the external stuff for e2e
@tawdry needle i've never had to concern myself... food for thought though
@gusty willow Good point. I need to start doing that more.
@topaz aspen also its much better for polyglot setups
what if you have both R and Python scripts in your project?
I do 🙃
what if you wrote your own C code and need to make sure your collaborators use a similar version of GCC?
currently writing the R -> python lol
I heard Conda kicks butt for getting Nvidia drivers working with Pytorch/TF on Windows.
re C, I have no idea, I did consider learning docker for the sake of having an env for R and Python tho, but never got round to it
right. use conda instead
i think i'm going to have a hard time selling that - and getting myself to learn it over other stuff i have to do
write a yaml environment file, conda env create -n my-project -f environment.yaml
but I'll keep it in mind
I just can't imagine using windows for anything other than windows development though (kind of like the only reason for owning a Mac)
yeah I've used conda like that - but just for python
yep, its nice even for that
i didn't really prefer it to pip for that tbh
pip is fine most of the time
and conda has a LOT of rough edges
but for data science type work i will use conda every single time over pyenv+pip
oh yeah - controlling your python and r runtime versions
i stopped using pyenv actually, just use brew and pipx now
if you arent using conda or docker, have fun with your 3d party tooling salad
yeah i love pipx
i actually wish brew would support "environments" like this
i hate my 3rd party tooling salad 😭
Pipx, that's a new one for me
the brew model would easily support it. i just imagine that their codebase doesnt
hell with conda you can even specify what git version to use in your project
i dont go that far but you could
unLiKe PeRL THerE's OnlY OnE OBviOUs wAY to DO thInGs IN PytHon
@tawdry needle one of the main things i've head about conda is "dependency hell" 🤔
i've not personally run into that afaik - but i have heard from people who have lost a lot of time with it, though idk what it really entails, maybe they messed something up
For my data science stuff, venv and pip does what I need to. But, I know my Nvidia crap is pre-configured correctly, so I don't have to worry about that as much. And, I am on Ubuntu.
yeah some things simply have incompatible dependencies
pipx is cool.
and for some reason conda's dependency resolver isn't always good about handling it
@vague lion yeah conda shipping binary packages is just a nice plus
right - and that ends up in some kind of hell?
I use pip-tools, so i do pip-compile --generate-hashes and pip-sync, not much else
yeah possibly. usually i just delete my env and start over if that kind of thing happens. maybe 75% of the time it's my fault and not conda's
yeah pip-compile is really nice
im not saying there's anything wrong with your stack, im just saying i like conda and i will advocate for its usage in the right context
and that it fills a unique niche, and has additional benefits beyond what a language-specific package manager and virtual environment type of setup can offer
does R still not even have a venv equivalent?
kind of blows my mind that it doesnt
Oh - i'm always happy to have my stack dissed 😄
Not sure I see a place in my workflow for pipx
and yeah - it can get a bit circle-jerky , I've definitely dissed conda more than i have the knowledge to do so - mainly because it annoyed me by dumping in my rc and automatically activating itself i guess
@gusty willow its good for "system wide" python package installations, i use it for stuff like litecli, dvc, mypy, and poetry
but i also specify mypy as a dev dependency in most projects
i just like having a system-wide version for poking around and testing things
Ah ok, poetry makes sense
I've not touched poetry
I install cli tools with pipx. httpie is the main use case. And, I will occassionally use glances.
here's a fun one for the vim users - get syntax highlighting for MANIFEST.in:
augroup vimrc_ft_setuptools_manifest
autocmd BufNewFile,BufRead MANIFEST.in setfiletype setuptools-manifest
autocmd FileType setuptools-manifest syntax keyword setuptoolsManifestDirective include
autocmd FileType setuptools-manifest syntax keyword setuptoolsManifestDirective exclude
autocmd FileType setuptools-manifest syntax keyword setuptoolsManifestDirective recursive-include
autocmd FileType setuptools-manifest syntax keyword setuptoolsManifestDirective recursive-exclude
autocmd FileType setuptools-manifest syntax keyword setuptoolsManifestDirective global-include
autocmd FileType setuptools-manifest syntax keyword setuptoolsManifestDirective global-exclude
autocmd FileType setuptools-manifest syntax keyword setuptoolsManifestDirective graft
autocmd FileType setuptools-manifest syntax keyword setuptoolsManifestDirective prune
autocmd FileType setuptools-manifest highlight default link setuptoolsManifestDirective Keyword
augroup END
the smudge/clean seems to work, at least I think as the changes don't show up at all while they are present, thanks
good stuff
would someone happen to know how to run a input script on Atom for python. Im losing my mind trying to figure it out lmao
Finally got a decent solution for knowing when I leave caps-lock on. Rainmeter skin that shows a notice (Doesn't blink, I'm just toggling it) when caps-lock is enabled.
anyone know how to use pip2? I have python 2 and 3 installed but I specifically want to install a module for python 2, when i use 'pip2 install ___' it says pip2 is not recognized
what problems do you have? just asking the question rather than asking to ask will yield faster responses
The torch package contains data structures for multi-dimensional tensors and mathematical operations over these are defined. Additionally, it provides many utilities for efficient serializing of Tensors and arbitrary types, and other useful utilities. It has a CUDA counterpart, that enables you to run your tensor computations on an NVIDIA GPU with compute capability >= 3.0.
can some1 give an idea of what gpus support it?
me crying in amd gpu
oof. i've always been told to just avoid AMD for this
welp
when i realized i cd buy a laptop 2x as powerful for $20 more (1650 > rx560x)
double oof
also dont buy a laptop for machine learning imo
just buy something that's light and comfortable to use. just pay the few bucks for cloud computing
or buy a desktop PC and SSH in from the coffee shop
ik xd ill be moving to pc in a couple decades :3
wait is cloud computing better for that type of stuff?
not necessarily better
but you can either spend $2500 on a PC, or spend like a couple hundred $ over the course of the year on cloud machines
buying a badass machine learning rig is only worth it if you are doing a lot of heavy computation all the time
hm tru
e.g. if you are a full time machine learning freelancer or something
and even then you will probably end up working in VMs on your client's infrastructure
ok ive decided to buy a pc with 8 rtx quadros next century
looks like i have to rethink this tho
ty for the help :3
laptops are like the worst of both worlds imo. just get enough memory and processing so that you can open as many browser tabs as you want, dont overdo it
youre welcome
@tawdry needle the only laptops that make sense are ultrabooks
Idk, my system76 is pretty powerful lol
A pinnacle of performance, Serval WS features desktop AMD processors, and the latest NVIDIA GPUs wrapped in the slimmest desktop replacement on the market. System76 laptops are custom-built to run Linux flawlessly.
I don't do ML though
But the AMD 3900 + 64G ram make quick-ish work of the things I compile on the regular
or pre-usb-c macbook pros
😦
I would love a threadripper, but damn they are expensive
my point is that the same amount of computer in desktop form is a lot cheaper @gusty willow
and that for the price of a laptop that powerful you can have much much more computer if you get a PC instead
Yeah, but I need it to move ¯_(ツ)_/¯
I have servers to do the heavy lifting for me when I'm home, but I have to travel for work more often than I like
And grossly enough, I'm still running my servers with openstack lol
But I've not found a good way to create resource pools in kubernetes yet, so ¯_(ツ)_/¯
It would be great if someone can spare me a few mins, I have a fail Travis build the it's a very simple one, just clone this repo and push to another one but it fails at git config and the error message is not helping, I have no idea why but it seems like a very simple mistake... https://travis-ci.com/github/terminusdb/terminusdb-client-python-wiki
Travis CI enables your team to test and ship your apps with confidence. Easily sync your projects with Travis CI and you'll be testing your code in minutes.
Why not just post the error @idle forge? No offence, but I'm not cloning any unsolicited repo or clicking any unsolicited links in a public discord.
ok I will copy the error:
The command "git config user.name "[secure]"
It's not very helpful I think... here is the travis script:
script:
- |
git config user.name "Cheukting"
git config user.email $GITHUB_EMAIL
git remote remove origin
git remote add origin https://$GITHUB_API_KEY@github.com/terminusdb/terminusdb-client-python-wiki.git > /dev/null 2>&1
git remote add upstream https://$GITHUB_API_KEY@github.com/terminusdb/terminusdb-client-python.wiki.git > /dev/null 2>&1
git fetch origin
git fetch upstream
git merge upstream/master --no-edit
git push origin HEAD:master > /dev/null 2>&1
git push upstream HEAD:master > /dev/null 2>&1
why do you have both the origin and the upstream set to the same repo?
they are not the same, one with -wiki the other .wiki
The reason why I do this is that I want others can contribute to the wiki of the other repo, since GitHub does not allow PRs to send to .wiki repos so I am doing this
I am following a tutorial to do this... I have a link to the tutorial.... but I guess links are not allow?
If you want to see it try googling "Enabling pull requests on GitHub wikis"
Hey guys, do you know if there is a way to make a container privileged during the build phase with docker-compose?
they are not the same, one with
-wikithe other.wiki
@idle forge punctuation ftl 😄
@idle forge punctuation ftl 😄
@gusty willow yes it's very subtle
links are allowed, I just won't click them 😄
but yeah, this: The command "git config user.name "[secure]" is not exactly an error
does github store the wiki for my-repo in my-repo.wiki?
Isn't that a private repo?
does github store the wiki for
my-repoinmy-repo.wiki?
@tawdry needle yes
Isn't that a private repo?
@warm pollen the original repo is public.... but you have a point, maybe the.wikirepo is private.... but I still have access to it so the CI should work....
@idle forge maybe it's because you are using HTTPS repos rather than SSH. Unless you are passing in the TOKEN elsewhere, it's going to ask for a password and might fail silently
@warm pollen the original repo is public.... but you have a point, maybe the
.wikirepo is private.... but I still have access to it so the CI should work....
@idle forge You have access to it, but the CI system isn't necessarily logged in as you
and it 99% of cases, it's NOT logged in as you by design
Got it, will try that, Thanks @gusty willow
Is there any reason to not just do everything as a root user in a Flask+Gunicorn docker container?
@true vapor will this be exposed to the internet?
If so, there is one major reason not to do that ... and that is container escape (which is really easy to do as the root user)
You can do everything except run gunicorn as root and be fine though
though you might run into permissions issues
yeah - I tried for a little while and ended up in permissions hell
and right now it's not gonna be exposed to the internet - so I'll leave it as a problem to solve for the next guy
Although I am curious - why does running it as root lead to container escape?
i tend to move all of my files to the container, then set up the group/permissions, then do the user, seems to work ok for most things
when you run as root, it allows me to do all the things possible in the container when/if i pop your site. And escaping a container is just a google away with root access
and it tends to be scary, because docker daemons run as root on the host system, so even a decently well versed script kiddie can own an entire box
most will just run crypto miners lol, but it can be scarier
which is why i don't run docker ... i either use Kata containers (which adds a VM layer to the abstraction) or, more commonly, I just use podman
which doesn't have a daemon running as root
but for anything running that is exposed to the internet, i suggest kata containers
there are layers you can put in front (L7 firewalls and such) that limit the attack surface, but best to do all of the above lol (defense in depth)
i even have admission controllers on my k8s clusters that don't allow containers that have an entrypoint that is running as root
and there are other processes I run that watch for unexpected behaviour at the node level and will just flat kill and replace the node if something goes sideways
That sounds like a lot of effort
sorry for the wall of text, but this is what I do lol
What's the worst that can happen being horribly insecure?
unfortunately, security is a lot of effort
depends on what your app does
would someone possibly know how to use the IDE terminal for atom
9/10 times, it's the economic tolls that make companies want to do this ... not only one time GDPR fines, but a loss of revenue due to the loss of trust from your customer base
and then there are ongoing "credit monitoring" costs and such to try to gain face again
it's a lot less expensive to put in good practices from the onset, rather than trying to clean it up after a breach
what do you mean exactly @glad ferry
have you ever used the text editor atom @gusty willow
yeah, although I've moved away from it. It's a pretty quick download though if it's a simple question lol
im trying to use the Platformio Ide Terminal package to run my python scripts seeing that Atom doesnt have a built in one to run input commands
However when i go to run the program Atom is saying the files cant be found
never heard of that plugin, i've only ever used the atom-terminal one
with Atom, you cant run input required code. WOuld you happen to know how to?
looks like platformio is specifically tailored for C/C++ development
this plugin has good documentation on how to use it: https://atom.io/packages/atom-terminal
if you need something that is free that has in integrated terminal, I suggest VSCode (https://code.visualstudio.com/)
i will check it out, thanks for the help
to get the terminal in VSCode, you just press ctrl+`
Im new to coding and getting this figured out has been annoying, I appreciate the help
no problem
although atom is still viable, there are better options for IDEs out there
personally, I use the jetbrains ones (IDEA Ultimate specifically), but it's not free 🤷
PyCharm is a great one that has a free version though
the only problem im having with atom is its not wanting to recognize install locations, I think i might go with pyCharm
PS C:\Users\Mislav\Desktop\django> pipenv install --python 3.4
Virtualenv already exists!
Removing existing virtualenv…
Creating a virtualenv for this project…
Pipfile: C:\Users\Mislav\Desktop\django\Pipfile
Using c:/python34/python.exe (3.4.0) to create virtualenv…
[=== ] Creating virtual environment...created virtual environment CPython3.4.0.final.0-64 in 3007ms
creator CPython3Windows(dest=C:\Users\Mislav\.virtualenvs\django-9O6aIxu9, clear=False, global=False)
seeder FromAppData(download=False, pip=bundle, setuptools=bundle, wheel=bundle, via=copy, app_data_dir=C:\Users\Mislav\AppData\Local\pypa\virtualenv)
added seed packages: pip==19.1.1, setuptools==43.0.0, wheel==0.33.6
activators BashActivator,BatchActivator,FishActivator,PowerShellActivator,PythonActivator
Successfully created virtual environment!
Virtualenv location: C:\Users\Mislav\.virtualenvs\django-9O6aIxu9
Creating a Pipfile for this project…
Pipfile.lock not found, creating…
Locking [dev-packages] dependencies…
Locking [packages] dependencies…
Updated Pipfile.lock (fa2f51)!
Installing dependencies from Pipfile.lock (fa2f51)…
================================ 0/0 - 00:00:00
To activate this project's virtualenv, run pipenv shell.
Alternatively, run a command inside the virtualenv with pipenv run.
PS C:\Users\Mislav\Desktop\django> python --version
Python 3.8.5
PS C:\Users\Mislav\Desktop\django> python
Python 3.8.5 (tags/v3.8.5:580fbb0, Jul 20 2020, 15:43:08) [MSC v.1926 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>>
KeyboardInterrupt
>>>
KeyboardInterrupt
>>>
KeyboardInterrupt
>>>
KeyboardInterrupt
>>>
KeyboardInterrupt
>>> quit()
PS C:\Users\Mislav\Desktop\django>
Pretty much this
Told it to target 3.4
went to 3.8
AHH ok I'm an idiot
I forgot it doesn't put you in the shell as soon as you install it
this is actually pretty fuckin sick
https://code.visualstudio.com/docs/remote/wsl
this alone is enough for me to use vs code lol
in using pytest is there a watch mode? I mean re-running the test every time a file is updated
Yeah the WSL is awesome.Best of both worlds. Super handy for running sevices lile postgres/reddis. Pair it with Dockers and it gets even better.
Could somebody explain to me how I get a docker image to run after pulling it down from docker hub? No matter how I "docker run" the image it always creates an "EXITED" container that can't be ran
maybe it just finished, is the exit code 0 ?
@vernal mirage my guess is that it's a non blocking entrypoint
Running it with "-it" (interactive mode) seemed to fix it?
Need to read up on docker though tbh
@vernal mirage yeah, it just means interactive
So if the entry point is say: a shell, then you have to use interactive
I've seen others not really understand and try to run nginx as a daemon and then not know why the container doesn't run
Am I right in thinking that from working in the shell I can still install and make as I would within the actual machine?
Yes, until the container stops
Then when you launch it again, it's all gone unless you are doing some volume things
I assume this is a Linux container?
Yep an ubuntu container for automated planning
Just remember, things that rely on an init system (like systemd) don't work in containers ... Mostly because containers don't need an init system
Ah okay, don't know much about init systems so will have to look into it
Init systems are what run daemons or services
JSON, why would you make and install every time?
i am assuming it's more of just a testbed, but it's also, very much an assumption
I will do that from time to time myself because I hate debian based OSs lol
lol @lone linden, I didn't want to go too far down the rabbit hole (no pun intended @wooden ibex)
Fair
hello
im trying to deploy using gunicorn
this section
Next, let’s create a file that will serve as the entry point for our application. This will tell our Gunicorn server how to interact with the application.
Let’s call the file wsgi.py:
nano ~/myproject/wsgi.py
In this file, let’s import the Flask instance from our application and then run it:
~/myproject/wsgi.py
from myproject import app
if __name__ == "__main__":
app.run()```
isn't this section supposed to be the .py file
from myproject import app
or is it supposed to be the directory containing the .py file
@graceful hatch send here the problem
mm well i installed python but my interpreter doesn't appear
says that python is not installed
@graceful hatch ask in #python-discussion this time
@sand thistle depends how you all is structured, but generally, I like a separate file in the root of my projects that is the wsgi entrypoint
@wraith lodge this happens when you are doing an "insert", so either your keyboard is wonky or you have a keybinding issue
@graceful hatch did you figure this out?
Hi, anyone knows how to remove the first few characters from every line in a non repetitive way? I'm using Atom text editor
i don't use atom but you probably want to search for "atom column select"
Lol solved it with python re
But still wonder if there's a way to do it with just atom
Most editors have some way to add cursors to multiple lines.
Looks like its the first one
most editors let you do some kind of column select and delete
@sleek grove 👍
Most editors have a "find and replace" feature as well. And most allow regex.
my flask app fails when I deploy it to heroku. it says not found create_app function but I can ctrl+click and go to the that function.also it works locally
my project structure -> https://hastebin.com/karotikixi.css
build error-log -> https://hastebin.com/luxiciroru.sql
so I am trying to have my discord.py bot be online 24/7
and started an instance hosted by Amazon
using google and some guides, I was able to launch it and connect to it
but I am a bit lost on what to do from this screen on:https://i.imgur.com/TPhTJtV.png
would anyone know a guide or something I could use to do stuff from here on?
@digital axle "up 24/7" can mean a lot of things
that's a terminal
a quick and dirty way that isn't crash-proof is using something like screen or tmux
and just running it in there
the better way might be to write a service specification
idr how to do that for current systemd
when i try to click "install packaging tools"
it asks for elevation twice, then throws that error
Linux Mint 20, Pycharm 2020.2
hm, installing python3-pip fixed it
Hi! Trying to get back into Python. What are the the current most popular solutions for managing different environments and dependencies? Using Linux.
@south widget depends on your use case, but generally, the two most popular envrionment managers are conda and pyenv ... for dependencies, I tend to use poetry so I haven't followed them as closely ... and as always, conda for the DS workloads lol
testing wise, I will probably always use tox and pytest 🤷
Anyone have experience with Sphinx?
I want to take all modules in my package, get the docstrings, and generate a doc page with it, but I have no clue how any of this works.
I've also tried pdoc, and pydoc
Man, I haven't used Sphinx in years
@sharp wren @ me in like.. 2 days. i can show you how to get set up with sphinx
Anyone know of a free product that will let you mount an sftp server like a windows drive?
Like Expandrive, but free. I'm breaking up with my smb shares and sticking with sftp. I use winSCP, but for my family something simple works best.
@tawdry needle I think I got the majority of it figured out, I found a library that does almost all of it
Thanks though
@sharp wren i mean, sphinx has an autostart or init thing you should use
like sphinx init-project or something
That's the gist of what the library does
but basically it's just a matter of calling sphinx-build on the right files
Just type D:
Yeah, you have to "switch" to the drive before navigating around. That's one of the things I hated about windows lol
This gets me every now and then when I need to use the cmd prompt because GitBash isn't working. I've looked up the solution so many times and for some reason I can never retain it.
how do I make the image in a readme stretch across the whole page?
The repository for the 2020 summer code jam. Find more information at https://pythondiscord.com/pages/code-jams/code-jam-7/ - python-discord/summer-code-jam-2020
its just a big banner i think
[](#)
no special code here
ah, okay, thanks! i'll look at the dimensions
huh, keeps shrinking https://github.com/thaniel-c/summer-code-jam-2020/tree/readme-dev/talented-tigers
The repository for the 2020 summer code jam. Find more information at https://pythondiscord.com/pages/code-jams/code-jam-7/ - thaniel-c/summer-code-jam-2020
two things are different between yours and the one you linked
- they have an empty link
- there's is above any header
so maybe try each of those things
maybe specifically set it as an anchor
alright, ill try that
hmm, tried that, didnt work
The repository for the 2020 summer code jam. Find more information at https://pythondiscord.com/pages/code-jams/code-jam-7/ - thaniel-c/summer-code-jam-2020
Hey so anyone who knows a lot about sublime, could you tell me why sublime and sublimerepl dont understand fstrings.
Just ping me if you have an answer and/or solution as I will be busy this evening
@summer tinsel is sublimerepl using the internal python? or your own python?
you might also want to try installing PythonImproved for better syntax highlightin
but my own version of sublime does appear to have f string support in the built in python syntax
and my sublimerepl is using py 3.8 which i installed
Has anyone used PySpark on Databricks(AWS) in Production? If so, what does your CI/CD pipeline look like for different environments and are there any community tools which can help in CI/CD of PySpark job scripts.
What ide would you suggest on i3-5010U 4gb ram?
Actual performance depends on many factors such as graphics card, RAM speed, disk speed etc.
VSCode with Python+Pyright+some other garbage like coloured parentheses runs just fine for me (I have similar hardware). PyCharm can be a tiny bit slow with large projects.
"IDE" and "text editor" is not a dichotomy and are pretty vague terms.
@shadow crow on my other pc, i tried once vs code with python, and took like 10 seconds to start a simple file with hello world
Is it normal
?
I don't know then
@tawdry needle Im using whatever is set up by default. I checked what version sublime is using with this:
import sys
print(sys.version)
and got
3.8.5 (tags/v3.8.5:580fbb0, Jul 20 2020, 15:43:08) [MSC v.1926 32 bit (Intel)]
then it should work with f strings
@orchid thorn fwiw i dont even know how to test pyspark code correctly, let alone set up a ci/cd pipelie
curious if you have any experience with that
just "run a local pyspark cluster"?
@tawdry needle I am not worried about testing pyspark code right now. I am more interested in setting up a git project that can take configuration and pyspark scripts and deploy them to different Databricks accounts (dev/staging/prod). I currently just have one account and am using that for explore Databricks; but I need to productionize it at some point and was wondering if someone has experience with it and how people have done it.
hm
wouldnt you normally use Databricks Jobs for that
@summer tinsel it looks like sublime is giving you f string highlighting. im not sure what your problem is
you arent running your file in a repl, you're running the whole file at once
you don't have a print() anywhere so it's not printing
because the repl emits the repr of the result of the last-executed expression
it has nothing to do with f-strings
type 1 into the repl
it prints 1
put 1 into a .py file and run it
it prints nothing
that's what you are experiencing here
@tawdry needle Yes, exactly, the Databricks Jobs is just a REST api and I would have to build all the tooling to be able to place all the config (job/cluster/pools/schedules/etc...) in code on git.
I am looking for something as robust as Serverless or Cloudformation framework but for Spark.
ah, im not aware of anything like that
anyone had issues trying to set things up on a personal GCP account?
i just want a GPU, and keep getting my requests to increase my limit refused, and I can't get in touch with a sales rep
ping me if you response please
How do I make sure that after the terminal executes all the python code it doesn't close?
@heavy knot python -i <script>
when I open the .py file and run all the code it closes and I want it to stay open
@heavy knot i've no idea what you're talking about
@heavy knot if you run from the terminal then -i sounds like the flag that you want
( i don't know why you're opening the file for this at all though )
i don't use terminal i open the .py file at once
when i open the .py file and all the code is executed it closes
well you said terminal - i dont' know what you're opening in, you need to give more info
sorry - i run from terminal i'm not sure what you're doing, hopefully someone else does
ok
If the code is done and has nothing else to do, why do you want it to keep running ?
is that I want to see something the code does and I want it to stay open and when I press a key it closes
You should actually run it from a terminal then, even when the code is done you'll still have your prints and everything else showing
Look at the last section
@drowsy lava theres a flag to allow you to cd across drives
@tawdry needle DoubleFelix's solution helped
doing D: got me there
how can i run ansible in a virtual env?
do I need 2 docker images, 1 for windows, 1 for linux if I want my code to be runnable cross platform? (through docker)
No, that's the point of docker, it'll run the same on any platform
thanks!
Although windows containers don't run on Linux all that well
So build it with Linux containers lol
👍 also, a docker tag is just a name for the image, right?
@mild lance Yes, you will need two images if you want to run Windows/Linux containers
Few people run Windows containers and even fewer still can only handle Windows containers
so if you just provide a Linux Container for Python, you should be fine
@wooden ibex so this will be fine? FROM python:3.7-buster (for the version)
and people uring windows and *nix systems will be fine?
sure, but buster is Debian
so you will need to be run Linux on container host
for "Windows" that means running Linux VM
Is there any version that will be fine for both os's?
not docker
or providing docker image for Linux, "FROM python:3.7-buster"
and docker imaage for Windows, "FROM python:3.7-windowsservercore-ltsc2016"
as someone who runs alot of Windows Server, we are used to running Linux VMs for occasional Linux software
it depends on your target market
"Few people run Windows containers and even fewer still can only handle Windows containers"
code jam judges lol
I would check with Code Jam but I'd be shocked if any Judges can't handle Linux Docker
I could and I code on Windows 10 Pro
if you have Win10 Pro, you can install Docker which installs Linux VM to host docker on
somewhere, you should put docker image in repo
or just give them dockerfile and bash file to build it
or even docker-compose
so they can do "docker-compose up
if you have Win10 Pro, you can install Docker which installs Linux VM to host docker on
@wooden ibex like, vmare?
where does it install it?
install what?
like, is it a full on vm
are you trying to distribute the docker image?
@wooden ibex distribute something docker
you either upload your image to docker repository OR you provide code with dockerfile
and instructions on how to do it
or host it yourself
alright, so assuming they don't have a vm, I need 2 dockerfiles?
just ask
if they have vms?
just ask
@wooden ibex ask what?
"Hey f1re, can we assume that everyone will be able to handle Linux Docker container?"
if so, one dockerfile and you are done
gonna copy and paste that lol
like I said, most serious python programmers, even if they run Windows, should be able to run Linux docker containers
I can
thanks for all of the help :D
I develop on Windows but deploy my Python bot on Linux using Linux Python containers
@wooden ibex https://discordapp.com/channels/267624335836053506/728010915580084336/741855128163975178 just to check, this is a no, right?
Yep, that's misunderstanding
Docker Desktop does magic sauce to make it appear that it's running Linux Container on Windows but it's really Linux VM
Yea, like f1re has never seen Windows container
it's because of how rare they are, we run them at work
so, they just need to have docker desktop and one dockerfile will be fine?
looking at message history, you should provide easy way to run your app
so make sure you have instructions like docker build . then docker run
or docker-compose where they can do "docker-compose up"
version: '3'
services:
drblistener:
build:
context: .
dockerfile: discord_listener.dockerfile
image: 'drblistener:latest'
restart: always```
that's a basic docker-compose file
to run it, it's docker-compose up -d --build
it will build the container then run it
still need to build the dockerfile though
k, so, do I give one dockerfile and say to use docker desktop on windows and normal docker on linux
no
just provide dockerfile, instructions on building your dockerfile and running your dockerfile
f1re seems to indicate everyone will assume it's Linux Docker
which is smart thing
alright, so my teammates need a linux vm?
no
so, docker desktop is needed to use a linux dockerfile on windows (didn't meak linux) @wooden ibex
no
if you are running Linux, it's just install docker and docker-compose
followed by docker up
*windows
if you are on Windows, you install docker desktop which installs all docker tools and assuming you are set to use Linux (default) it redirects all docker commands into Linux
👍 will wsl 2 work as well?
Yes it'll work. Docker desktop relies on WSL2. It won't let you install it until you've setup WSL2.
You also need to have a pretty recent version of windows. 19xx. It'll tell you if you need to update first.
thanks! :D
Hey, I am really new to python and I am getting this error I don't know how to fix. https://i.imgur.com/s7ETMrX.png Would anyone be able to help out?
@heavy knot are you on windows, mac, or linux?
win10 64bit
run pip install numpy in the terminal
pip: The term 'pip' is not recognized as the name of a cmdlet, function, script file, or operable program. Check the spelling of the name, or if a path was included, verify that the path is correct and try again.
Did you exclude it from your install by any chance?
Maybe, do I download it off python.org?
yes
Okay, I'll reinstall then
Make sure to click Add to Path box on the bottom of the screen when yiu install it
Yeap, thank you!
👍
does anyone know a good and comfortable vscode theme for large programming sessions?
Personal preference.. I use overnight/ariake dark/dracula @honest lily
definitely personal. Just look for easy on the eyes. My choice is Ayu.
Darcula or zenburn, while wearing blue light-filtering glasses
I was going to buy a pair, but then read that they were a scam. The real problem is that the screens cause you to blink less often and therefore your eyes get irritated and damaged. Just remember to blink and you'll be fine.
Well, either they're a scam and I've been duped by my buyer's remorse (though they weren't even that expensive), or the slight filtering of blue light has caused my eyes to feel less strain. I've been using them for about 2 weeks and my eyes do not feel as tired as they used to. I've been aware of blinking frequency being potentially problematic for eye strain for years and have made attempts to blink more frequently, but I don't believe my blinking patterns have changed since I started wearing them.
Personally, I choose a solarized theme. I don't really like dark themes lol