🔒 How To Use Twine???

[tawdry pasture]

so I've gotten to this part of the tutorial and when I run python.exe -m twine upload --repository testpypi dist/* it says that it's uploading and then prompts me for my api token. I've tried copying/pasting -> didn't work, I've tried typing in the console -> input not allowed. I press enter to leave it empty, and I get the screenshot. I have no idea what I've done wrong, but I've followed this tutorial step-for-step in setting everything up and such, and this also stems from my previous post. Thanks in advance

[stray vigil]

@tawdry pasture

Python help channel opened

Remember to:

• Ask your Python question, not if you can ask or if there's an expert who can help.
• Show a code sample as text (rather than a screenshot) and the error message, if you've got one.
• Explain what you expect to happen and what actually happens.

:warning: Do not pip install anything that isn't related to your question, especially if asked to over DMs.

Closes after a period of inactivity, or when you send !close.

[tawdry pasture]

@tropic pawn apologies for the ping if I'm not allowed to do that, but I know you were helping me before in the previous post and i wasn't sure if you wanted to continue or not. again apologies

ok so I figured out that when entering the API Token, it's just invisible, however, copy/paste does work, but now I'm getting an invalid authentication error and I'm looking to see how I'm supposed to login to testpypi via termine with twine

[tropic pawn]

Providing the token to the terminal should work, not sure why not here. Here are the PyPI help page (that last part is important).

Alternatively, you can also create a .pypirc file in your home directory and specify the credentials there.

[tawdry pasture]

how do I do that?

[drifting skiff]

random guess, did you make sure you generate your token from the TestPyPI site rather than the main PyPI?

since they keep separate credentials and all

[tawdry pasture]

yes

so the .pypirc file would look like

[testpypi]
username = __token__
password = api_token

[drifting skiff]

seems right, in your user home directory

[tawdry pasture]

so given that I went the route of the .pypirc file, what do I do now? try the upload command again with twine?

setup.cfg: https://pastebin.com/F3d5ZVcq
pyproject.toml

[build-system]
requires = ["setuptools"]
build-backend = "setuptools.build_meta"

.pypirc

[testpypi]
username = __token__
password = api_token
```ok I have these three files setup this way so far. I'm wanting to check that everything looks right before I run the python build command if that's alright

[metadata]
name = Diary
version = 1.0.0
author = mekasu0124
author_email = [email protected]
description = A diary application
long_description = file: README.rst, CHANGELOG.rst, LICENSE.rst
keywords = diary, diary_app
license = GNULPv3
classifiers = Programming Language :: Python :: 3.12

[options]
zip_safe = False
include_package_data = True
package_dir =
    = src
packages = find:
python_requires = >= 3.8
install_requires = 
    bcrypt
    build
    clr
    colorama
    setuptools
    tqdm
    wheel

[options.packages.find]
where = src
exclude = 
    examples*
    tools*
    docs*
    DiaryConsole.tests*
```i updated the .cfg file to this

[drifting skiff]

been a while since i used setup.cfg but that looks fine

have you checked the contents of your .tar.gz and .whl distributions to see if they have all your modules and data files?

[tawdry pasture]

tbh if I can just use one of these three files 😂 I didn't realize that going about using setuptools, wheel, setup.cfg, pyproject.toml, and .pypirc were so en depth and needed so much to function

[tawdry pasture]

I couldn't figure out why twine wasn't working, and so I deleted the dist folder and the ./src/_.egg folders so that I could start over. I want to make sure the files that I need are there and that they are setup right

like this is my project structure

[drifting skiff]

pyproject.toml can handle the entirety of your build configuration in most cases, replacing setup.cfg

[tawdry pasture]

tbt: I went off half cocked a few years back on learning and using python and it wound up hurting me and so now I'm goign back and learning to do this properly

[drifting skiff]

.pypirc is meant to be in your home directory, not your project directory

[tawdry pasture]

if I can just use the .toml file, then bet. Let's do that

I'll delete the other two now

[drifting skiff]

though i guess with it there you could run twine upload --config-file .pypirc ......

dont delete .pypirc, your credentials aren't meant to go in pyproject.toml

[tawdry pasture]

I like the idea of using the .toml file. i'd rather have one file that handles everything than multiple

[tawdry pasture]

gotcha and you said that file is supposed to go where now?

[drifting skiff]

your home directory, i.e. wherever cd $HOME takes you (or is it %USERPROFILE% on windows?)

[tawdry pasture]

C:\Users\Mekas

so there?

[drifting skiff]

ya

[tawdry pasture]

ok I moved it. now what?

[drifting skiff]

twine should automatically load it now

[tawdry pasture]

ok. I'm goign to follow this tutorial on setting up the pyproject.toml file really fast

[drifting skiff]

this one? https://packaging.python.org/en/latest/guides/writing-pyproject-toml/

[tawdry pasture]

fixed it. apologies

is the dependencies array for the py-packages that I pip install and use?

[drifting skiff]

zipsafe, include_package_data, packages, and packages.find should not be necessary to keep, everything else you can try to find an equivalent for

ya, dependencies in [project] is equivalent to install_requires

[tawdry pasture]

class Main:
    def __init__(self, json_engine, database_engine):
        self.json_engine = json_engine
        self.db_engine = database_engine

    def start_program(self):
        self.json_engine.check_setup_files()
        self.db_engine.check_tables_setup()

if __name__ == '__main__':
    from src.infrastructure.json.engine import JsonEngine
    from src.infrastructure.database.engine import DatabaseEngine

    json_eng = JsonEngine()
    db_eng = DatabaseEngine()

    Main(json_eng, db_eng)
```this is my `main.py` file that lives in `root/src/main.py`. In my `pyproject.toml` file when I write

[project.scripts]
cli-name = "mypkg.mymodule:some_func"

[drifting skiff]

you'll have to refactor your main block into a function

also, src. imports are very unlikely to be correct once your package is installed

[tawdry pasture]

from src.infrastructure.json.engine import JsonEngine
from src.infrastructure.database.engine import DatabaseEngine

def start_program():
    json_engine = JsonEngine()
    database_engine = DatabaseEngine()
    
    json_engine.check_setup_files()
    database_engine.check_tables_setup()
```ok done

[tawdry pasture]

I use the Python AutoImports extension in VSCode. it does it on it's own

[project.scripts]
cli-name = "DiaryConsole.src:start_program"
```so with refactoring my main class to a function, is it written like this?

[drifting skiff]

er, pyproject.toml is inside DairyConsole right? you can omit that then

[tawdry pasture]

yea everything is inside DiaryConsole as DiaryConsole is the root folder for the entire project

I'm using the src-layout for a file structure

[drifting skiff]

the "feature" of src-layout is that it prevents you from importing any modules inside it until you've installed the package like an end-user, so if you ever have to import src for something that's going to be installed, thats a sign that your workflow isn't set up correctly (or that src/ is a bad name for your parent package)

see also:
https://packaging.python.org/en/latest/discussions/src-layout-vs-flat-layout/
https://setuptools.pypa.io/en/latest/userguide/development_mode.html

[tawdry pasture]

DiaryConsole (root folder)
|- .vscode/
|- env/
|- src/
| |- infrastructure/
| | |- database/
| | | |- __init__.py
| | | |- engine.py
| | |- json/
| | | |- __init__.py
| | | |- engine.py
| | |- models/
| | | |- __init__.py
| | | |- user_model.py
| | |- services/
| | | |- __init__.py
| | | |- result.py
| | |- __init__.py
| |- program/
| | |- __init__.py
| | |- new_user.py
| |- main.py
|- .gitignore
|- LICENSE.TXT
|- pyproject.toml
|- README.md
|- requirements.txt
```ok so what's wrong with my structure? How I'm using the `src/` folder?

[drifting skiff]

src-layout conventionally means any modules/packages inside it will get installed, so that means you'd have infrastructure, program, and main all added straight to site-packages
therefore, the imports would look like: py from infrastructure import xyz from program import xyz from main import xyz

[tawdry pasture]

ok I get that. so how do I need to change my file structure to look?

[drifting skiff]

given that your project name is Diary, i would have one single top-level package diary/ which contains the entirety of your code: src/ └── diary/ ├── infrastructure/ │ ├── __init__.py │ └── ... ├── program/ │ ├── __init__.py │ └── ... ├── __init__.py └── __main__.py pyproject.toml that keeps site-packages cleaner by containing everything in diary, and your imports would look like: py from diary.infrastructure import xyz from diary.program import xyz from diary.__main__ import xyz

sidenote, __main__.py allows a package to be invoked directly with python -m my_package, rather than doing python -m my_package.main
https://docs.python.org/3/library/__main__.html#main-py-in-python-packages

[tawdry pasture]

well see normally my main.py file is in my root folder and I just open the terminal in the project dir and run python main.py to run the application, but with all this setuptools stuff with the pyproject.toml thing, it's like well how do I start my project from the root directory so that I can test my code???

[drifting skiff]

thats where using python -m path.to.module is needed to run modules in packages, and editable installs to manage src-layout projects

[tawdry pasture]

ok this is my project structure now. What's my next step?

[drifting skiff]

in general whenever i set up a project, i start with: src/ └── my_package/ └── __init__.py .gitignore LICENSE README.md pyproject.toml where pyproject.toml contains: ```toml
[build-system]
requires = ["setuptools"]
build-backend = "setuptools.build_meta"

[project]
name = "my-package"
version = "0.1.0"
... # other keys as neededand to set it up:sh

python -m venv .venv
.venv\Scripts\activate
(.venv) > pip install --editable .
...
Installed my-package-0.1.0
(.venv) > python -m my_package # or import my_package in a test script```

yes its a bit of boilerplate, i tend to copy these files from my previous projects to save time writing them

so get the editable install set up, and if it's all correct you should be able to run py -m diary.main

[tawdry pasture]

I just wrote all of this to a sticky note. Thank you so much

[tawdry pasture]

fixing to give it a go now

[drifting skiff]

and with your new layout, your entry point to main might look like: ```toml
[project.scripts]
diary = "diary.main:main"

package.module:function```

[tawdry pasture]

[build-system]
requires = ["setuptools"]
build-backend = "setuptools.build_meta"

[project]
name = "Diary"
version = "0.0.1"
dependencies = [
    "bcrypt",
    "build",
    "clr",
    "colorama",
    "setuptools",
    "tqdm",
    "wheel"
]

[tool.setuptools.packages]
find = {}

[tool.setuptools.packages.find]
where = ["src"]

[project.scripts]
diary = "diary.main:start_program"
```is this correct now?

[tawdry pasture]

ok so I followed your steps to the T right. here is my pyproject.toml and the error. I've attached a project structure screen shot and followed the steps:

• python.exe -m venv env && env\Scripts\activate
• python.exe -m pip install --upgrade pip
• pythoon.exe -m pip install --editable .

[drifting skiff]

er yeah, find = {} and [tool.setuptools.packages.find] represent the same key

but you don't need either of them, setuptools automatically discovers src-layout when you use pyproject.toml for configuration

[tawdry pasture]

gotcha. I'll remove that and try again

[build-system]
requires = ["setuptools"]
build-backend = "setuptools.build_meta"

[project]
name = "Diary"
version = "0.0.1"
dependencies = [
    "bcrypt",
    "build",
    "clr",
    "colorama",
    "setuptools",
    "tqdm",
    "wheel"
]

[project.scripts]
diary = "diary.main:start_program"
```updated toml file, right?

[tawdry pasture]

ok so now I've received a new error

[drifting skiff]

right, invoking a package directly requires the presence of __main__.py

you should have the diary command tho

that or you can do py -m diary.main

[tawdry pasture]

ok well everything in the error message is all the things I entered into the console. I'll try that command really quick. one second

[tawdry pasture]

(env) D:\GitHub\DiaryConsole>python.exe -m diary.main
Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "D:\GitHub\DiaryConsole\src\diary\main.py", line 1, in <module>
    from infrastructure.json.engine import JsonEngine
ModuleNotFoundError: No module named 'infrastructure'
```ok that did something lol

[drifting skiff]

ye, your absolute imports should start from diary.

[tawdry pasture]

(env) D:\GitHub\DiaryConsole>python.exe -m diary.main
Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "D:\GitHub\DiaryConsole\src\diary\main.py", line 1, in <module>
    from diary.infrastructure.json.engine import JsonEngine
  File "D:\GitHub\DiaryConsole\src\diary\infrastructure\json\engine.py", line 4, in <module>
    from infrastructure.services.result import Result
ModuleNotFoundError: No module named 'infrastructure'

[drifting skiff]

same in engine.py

[tawdry pasture]

oh sorry

i should've known that. apologies

[drifting skiff]

no worries

alternatively since main.py is part of your diary package, you can write it as a relative import: ```py

main.py

from .infrastructure.json.engine import JsonEngine``` https://docs.python.org/3/tutorial/modules.html#intra-package-references
convenient for sibling modules, but wouldn't recommend it for relative parent imports, i.e. 2 or more leading periods .., (it'll get confusing after a while)

[tawdry pasture]

thank you so much for your help thus far. I'm not going to hold you up. I have to debut some of my actual code lol however I do have one more question. When a user downloads my app, they'll have to go through setuptools installing the packages and stuff. Since it's a console application, how can I display all of that to the console with loading bars?

[drifting skiff]

dunno, that'd be the responsibility of pip, or pipx if they use that

[tawdry pasture]

oh ok. thanks!

[drifting skiff]

oh, diary is taken on pypi by another project: https://pypi.org/project/diary/

so you'll need to pick a different name in pyproject.toml for pypi to accept it

[tawdry pasture]

I thank you for that as well. Another question. Since I did the pip install --editable . command, when I write new code in my editor and I run python.exe -m diary.main in terminal, will it auto update all the changes in like the build folder that is made with python.exe -m build command and the python.exe -m twine upload -r whatever_else_belongs_here command?

or will I need to rebuild and reupload every time

[drifting skiff]

o no, it only links your source code to site-packages so you see your changes during local testing

run py -m build each time before you upload, and optionally use 7zip or some other archive manager to look inside them for missing files

(if you really want to get sophisticated you can pip install dist/your.whl and run tests on it, but maybe at that point you'd do some tox setup or whatever, im not all that familiar with automated testing)

[tawdry pasture]

I apprecaite it. So when I rename my root folder and my src/project folder to a new project name, are there any files that I need to update other than the pyproject.toml file and the import statements throughout my code?

[drifting skiff]

should be all, but feel free to keep the root directory and diary/ the same name

"import packages" like diary/ are different from the "distribution package" defined by your pyproject.toml

pypi only cares that your distribution package name is unique, but you can include as many import packages as you want without having unique names for them

i mean if the import package clashes with another package then it could override their files or vice versa (a bit annoying that pip doesn't warn you when this happens), but that risk is fine if you don't expect both packages to be installed at the same time

if you do pick a radically different pyproject.toml name though, that might be worth renaming your diary/ package and entry point so it's more obvious to the user

[tawdry pasture]

yea I just went ahead and renamed it all and am rebuilding it all again going back through the steps that I wrote to the sticky note 😛

[tawdry pasture]

another question if you don't mind. In my main.py file, I am trying to use the tqdm package for the loading bar progress display. I know it's done in the manor that is in the commented code at the bottom of the link, however, I cannot figure out how to incorporate it with my start_program() function's code. I use a Result class to return results from my code like this. Any ideas on how I can fix my start_program() function to use tqdm for the progress bar on the setup function and still be able to get the result object from the function call?

[drifting skiff]

hmm, most likely you need a way for check_infrastructure_setup()'s implementation to report its progress to the caller

[tawdry pasture]

the problem is that the result object isn't iterable so I presume that in these specific functions, I don't need to result return a result, but instead return an integer to increment the progress bar with just pass during the iterations

[drifting skiff]

maybe you can pass in a regular tqdm object to check_infrastructure_setup(), abstracted with a protocol if desired, and manually call update as you progress through your function:
https://github.com/tqdm/tqdm#manual ```py
with tqdm(total=400) as pbar:
pbar.set_description("Doing x")
for _ in range(100):
...
pbar.update(1)

pbar.set_description("Doing y")
...```

[tawdry pasture]

that's not a bad idea

[tawdry pasture]

"""
with tqdm(total=400) as pbar:
pbar.set_description("Doing x")
for _ in range(100):
    ...
    pbar.update(1)

pbar.set_description("Doing y")
...
"""

def check_infrastructure_setup(self):
    with tq(total=100) as pbar:
        pbar.set_description("Checking for settings.json file")
        
        src_path = 'src'
        sub_path = os.path.join(src_path, 'DreamersDiscoveries')
        infrastructure_path = os.path.join(sub_path, 'infrastructure')
        setup_path = os.path.join(infrastructure_path, 'setup')
        self.file_path = os.path.join(setup_path, 'settings.json')

        if not os.path.exists(setup_path):
            os.makedirs(setup_path)

            setup_file_check = self.create_setup_file()

            if not setup_file_check.is_success:
                return Result.fail(setup_file_check.message)
            
            return Result.ok("Setup File Create Successfully")
        
        return Result.ok("Setup File Already Exists")
```ok so in trying to do that, here's where I'm at and I'm not sure how to incorporate your for-loop. So like, this function would need to equal 50/100% and then checking for the database file would need to equal the other 50% so that when the bar was finished, the setup check process is then finished. Can I apply this methodology of tqdm to the main file instead?

so like the main file would instantiate the pbar, and then pass the pbar to the functions that are called inside of it and then the inner functions that are called would have access to that pbar function, update the description, etc....

i'mma try that

[drifting skiff]

i imagine the difficult part would be trying to break down your function into individual steps

that or tedious or messy, you'd have to call update() at each step you decide on

and figuring out what's worth turning into a step, since those would be illegible to the user if it just flashed for a fraction of a second

logging might be more useful in that case, perhaps with a flag to let the user increase the verbosity of your logging from WARNING to INFO to DEBUG

apparently there's also tqdm.write() to print stuff without overwriting the progress bar, but that seems to have a bunch of bugs according to the linked issue

rich also has its own progress bars if you want to try it out:
https://rich.readthedocs.io/en/latest/progress.html
https://github.com/Textualize/rich/blob/master/examples/downloader.py

https://github.com/textualize/rich/raw/master/imgs/progress.gif

[tawdry pasture]

ok so saying I go this route

import time

from rich.progress import Progress

with Progress() as progress:

    task1 = progress.add_task("[red]Downloading...", total=1000)
    task2 = progress.add_task("[green]Processing...", total=1000)
    task3 = progress.add_task("[cyan]Cooking...", total=1000)

    while not progress.finished:
        progress.update(task1, advance=0.5)
        progress.update(task2, advance=0.3)
        progress.update(task3, advance=0.9)
        time.sleep(0.02)
```as I have two functions that will be executed, how would I go about executing them and still getting access to the Result object that is returned from those functions?

[drifting skiff]

updating the progress would happen on the side while you're doing your actual work
(which is true for tqdm too, their APIs seem similar in this aspect)

[tawdry pasture]

oh ok

[drifting skiff]

presumably you could write something like: ```py
def make_something_happen(progress: Progress):
task = progress.add_task("Making something happen...", total=3)

do_one_thing()
progress.advance(task)

do_another_thing()
progress.advance(task)

do_the_final_thing()
progress.advance(task)

return my_result

with Progress(...) as progress:
make_something_happen(progress)```

[tawdry pasture]

ok so based off your suggestions, I'm trying to figure this out. In the screen shot, letter A only runs like this, and letter B runs like this, but then just hangs up. I'm not sure if I'm doing something wrong, or if I need to run these async but what I was hoping for was for Test 1 to be running while Counting To 100 was running showing the progress of Counting To 100 as apart of the progress of the first test if that makes sense

[tawdry pasture]

I'm starting to get a headache. I just posted above where I'm currently at, but I need to go lay down. Idk hwo to re-open this thread if I'm not back in time to see your response so if I can't re-open it, I'll just create a new thread mentioning this one and mention you in it if that's alright?

[drifting skiff]

ya thats fine, linking back to the thread will help others with gathering context

[drifting skiff]

ergh, this is where threading/concurrency comes into play

in a typical single-threaded program, each statement must run to completion and nothing else can happen

so calling self.increment_to_one_hundred() means your program must fully execute that method before it returns to the caller

without async/await or threading, you can only get both tasks to update by well, updating both of them in the same function

[drifting skiff]

which is what this example does

[tawdry pasture]

So then I could make them both async and then await the second one in the first one?

[drifting skiff]

roughly like that, although async/await would also involve creating a second task too

something like: ```py
import asyncio, random
from rich.progress import Progress

async def run_job(progress, task_id):
while not progress.finished:
await asyncio.sleep(0.1) # asynchronous "work"
progress.advance(task_id, random.random() * 3)

async def main():
with Progress() as progress:
task_ids = [
progress.add_task(f"Task {i+1}", total=100)
for i in range(3)
]

    # Create tasks to concurrently advance each progress bar
    async with asyncio.TaskGroup() as tg:
        for task_id in task_ids:
            tg.create_task(run_job(progress, task_id))

asyncio.run(main())```

although this approach only works if the work you're doing is also asynchronous

[tawdry pasture]

I can make everything async. It’s a console based application but u prefer my applications async just in case like the computer lags or something then the program doesn’t get it off sync

[drifting skiff]

im not sure what you mean by it going off sync

i think the more accurate word i meant to use is "non-blocking", since certain operations like creating directories can be fast enough to not need an awaitable version of that function, but in contrast making an HTTP request or querying a database can take a long time due to network latency. so those require libraries that support async/await (httpx, asyncmy, asyncpg, etc.), otherwise their functions will block the event loop and freeze the rest of your tasks

[stray vigil]

Python help channel closed

This help channel has been closed and it's no longer possible to send messages here. If your question wasn't answered, feel free to create a new post in #1035199133436354600. To maximize your chances of getting a response, check out this guide on asking good questions.