#๐Ÿ”’ hi need help with my api structure

442 messages ยท Page 1 of 1 (latest)

strong windBOT
#

@vocal ginkgo

Python help channel opened

Remember to:

  • Ask your Python question, not if you can ask or if there's an expert who can help.
  • Show a code sample as text (rather than a screenshot) and the error message, if you've got one.
  • Explain what you expect to happen and what actually happens.

:warning: Do not pip install anything that isn't related to your question, especially if asked to over DMs.

vocal ginkgo
#

i want to break the code in multiple files can someone guide its structure

desert kestrel
#

For me:
Move model to different file
Move database action to different file
The current main file call those two file

#

That's a basic structure

vocal ginkgo
#

where is the envrioment variable goes and lifespan functions

#

and other function like get_current_user create_access_token etc

desert kestrel
#

!pypi pydantic_settings

strong windBOT
desert kestrel
#

You might try look at this

desert kestrel
vocal ginkgo
#

and where is the endpoint goes

#

@barren sluice here

desert kestrel
vocal ginkgo
#
.
โ”œโ”€โ”€ app                  # "app" is a Python package
โ”‚   โ”œโ”€โ”€ __init__.py      # this file makes "app" a "Python package"
โ”‚   โ”œโ”€โ”€ main.py          # "main" module, e.g. import app.main
โ”‚   โ”œโ”€โ”€ dependencies.py  # "dependencies" module, e.g. import app.dependencies
โ”‚   โ””โ”€โ”€ routers          # "routers" is a "Python subpackage"
โ”‚   โ”‚   โ”œโ”€โ”€ __init__.py  # makes "routers" a "Python subpackage"
โ”‚   โ”‚   โ”œโ”€โ”€ items.py     # "items" submodule, e.g. import app.routers.items
โ”‚   โ”‚   โ””โ”€โ”€ users.py     # "users" submodule, e.g. import app.routers.users
โ”‚   โ””โ”€โ”€ internal         # "internal" is a "Python subpackage"
โ”‚       โ”œโ”€โ”€ __init__.py  # makes "internal" a "Python subpackage"
โ”‚       โ””โ”€โ”€ admin.py     # "admin" submodule, e.g. import app.internal.admin
desert kestrel
#

For me, a good structure is when a dev have no idea what exactly the components does can easily used the API (different to the HTTP API, basically the class/function you let other dev use) without need to understand what it does in detail

vocal ginkgo
#

this is main fastapi doc here

desert kestrel
barren sluice
#

did you find my workshop repo?

vocal ginkgo
#

yes

#

@barren sluice need some informatiom

#

for all endpoints i need to create seprate .py file and add router = fastapi.APIRouter() in it

#

i have redis functions and some other function where should i put that in which folder and file.

#

@barren sluice and there is no database connection file in your repo

barren sluice
#

hello

vocal ginkgo
#

hi

barren sluice
#

you can group them into a package with multiple submodules

#

that way they stay logically connected, but in different files

vocal ginkgo
#

yes i need that

barren sluice
#

it depends how much granularity you want

vocal ginkgo
#

can you guide me step by step

#

as i never create project in multiple file

#

always in one file ๐Ÿ™‚

#

for the moment i want apis granularity there are alot of apis in 1 file and now its very hard for me to go through ๐Ÿ˜„

barren sluice
#

move things in steps, and check that its still working

vocal ginkgo
#

yes

#

can you provide steps pls

#

like the diretory structure

#

and files in each folder

#
.
โ”œโ”€โ”€ app                  # "app" is a Python package
โ”‚   โ”œโ”€โ”€ __init__.py      # this file makes "app" a "Python package"
โ”‚   โ”œโ”€โ”€ main.py          # "main" module, e.g. import app.main
โ”‚   โ”œโ”€โ”€ dependencies.py  # "dependencies" module, e.g. import app.dependencies
โ”‚   โ””โ”€โ”€ routers          # "routers" is a "Python subpackage"
โ”‚   โ”‚   โ”œโ”€โ”€ __init__.py  # makes "routers" a "Python subpackage"
โ”‚   โ”‚   โ”œโ”€โ”€ items.py     # "items" submodule, e.g. import app.routers.items
โ”‚   โ”‚   โ””โ”€โ”€ users.py     # "users" submodule, e.g. import app.routers.users
โ”‚   โ””โ”€โ”€ internal         # "internal" is a "Python subpackage"
โ”‚       โ”œโ”€โ”€ __init__.py  # makes "internal" a "Python subpackage"
โ”‚       โ””โ”€โ”€ admin.py     # "admin" submodule, e.g. import app.internal.admin```
#

like this

barren sluice
#

so you have things in your main.py ffile right now?

vocal ginkgo
#

yes

barren sluice
#

so step one is to pick something simple to move

#

like data models

#

do you have that?

vocal ginkgo
#

you mean these

# Pydantic models
class UserCreate(BaseModel):
    username: str
    password: str
    email: EmailStr

class UserUpdate(BaseModel):
    username: str
    new_status: Optional[bool] = None
    new_role: Optional[str] = None

class UserResponse(BaseModel):
    username: str
    email: EmailStr
    status: bool
    role: str

# Pydantic model for UserList
class User(BaseModel):
    username: str
    email: str
    status: int
    role: str
barren sluice
#

so take all those models, and move them over to models.py

vocal ginkgo
#

ok just a sec

barren sluice
#

and in main.py you have to import them back so the code works

vocal ginkgo
#

first i create models.py in app folder

barren sluice
#

what app folder?

vocal ginkgo
#

what about this

# Define ORJSONResponse
class ORJSONResponse(ORJSONResponse):
    def render(self, content: Any) -> bytes:
        return orjson.dumps(content)```
barren sluice
#

you said you had main.py has that changed since i asked about it?

vocal ginkgo
#

should i move this in models?

vocal ginkgo
#

i mean now we are structure code so i create new folder called app

#

and inside that i am creating files

barren sluice
#

dont create unneeded things

#

maybe later you see that you do need a package for your modules

#

but at this point you only have main.py

vocal ginkgo
#

yes

#

do i need to import anything

barren sluice
#

yes, the module needs to import everything it needs to run

vocal ginkgo
#

ok just a sec

#

done

#

next

barren sluice
#

show me

vocal ginkgo
barren sluice
#

yes

vocal ginkgo
#
from typing import List, Optional
from pydantic import BaseModel, EmailStr

# User models
class UserCreate(BaseModel):
    username: str
    password: str
    email: EmailStr

class UserUpdate(BaseModel):
    username: str
    new_status: Optional[bool] = None
    new_role: Optional[str] = None

class UserResponse(BaseModel):
    username: str
    email: EmailStr
    status: bool
    role: str

class User(BaseModel):
    username: str
    email: str
    status: int
    role: str

# Models for TopTalkers
class TopTalker(BaseModel):
    src_ip: str
    dst_ip: str
    src_port: int
    dst_port: int
    proto: str
    geo_country_code: Optional[str] = None
    geo_region: Optional[str] = None
    geo_city: Optional[str] = None
    geo_latitude: Optional[float] = None
    geo_longitude: Optional[float] = None
    connection_count: int```
barren sluice
#

great

vocal ginkgo
#

๐Ÿ™‚

#

next next

barren sluice
#

now dont change anything, but lets imagine something before we continue

vocal ginkgo
#

yes

barren sluice
#

well, at that point, you might want to make a models package instead

vocal ginkgo
#

yes

barren sluice
#

and inside this folder called models you have user.py for your user data models

#

and whatever.py for your whatever models

vocal ginkgo
#

user.py you mean the endpoints for user related task?

barren sluice
#

no, models

vocal ginkgo
#

yes for the moment i ahve only models.py and it has all models

#

like above

barren sluice
#

i do not jump ahead when I talk about python

#

im still on models.py but imagining you had 100 models instead of four

vocal ginkgo
#

yes

barren sluice
#

i did not use a models.py module, because i got to many models

vocal ginkgo
#

oh yes

barren sluice
#

i could have had all my models inside models.py

#

and in the future you might have to change from models.py to models as a package

#

not now, you only have a few models

#

is this part clear?

vocal ginkgo
#

i have 19 models

#

yes its clear

barren sluice
vocal ginkgo
#

yes

barren sluice
#

alright i see..

vocal ginkgo
#

but its clear for me now and in future

#

i have more models

#

so thats why i said models folder

barren sluice
#

sure thing, you can refactor it from models.py to models package later

#

now, next step, clean up main.py and add project imports

vocal ginkgo
vocal ginkgo
#

pls guide

barren sluice
#

so remove unneeded imports

#

you can probably remnove BaseModel

#

and maybe EmailStr, if you dont use it in main.py

#

and List and Optional is probably used somewhere in main.py already

vocal ginkgo
#

also remove all models

barren sluice
vocal ginkgo
#

yes i mean from main.py no models is present now

barren sluice
#

now last step is to import the models back to main.py

vocal ginkgo
#

yes

barren sluice
#

from models import UserCreate, UserUpdate, ... and so on

vocal ginkgo
#

so i need to add all 19 model

barren sluice
#

yeah

#

next thing to refactor is the api

#

same here, that can be an api.py module or an api package

#

the api here would mean all your routes

vocal ginkgo
#

wait wait

#

let me import

#

just a sec

barren sluice
#

import, run it to see it working

#

i have given you the steps you need, now you just repeate them

vocal ginkgo
#

done

#

what need to copy?

barren sluice
#

your routes

#

thats your API

vocal ginkgo
#
# Route to sign up
@router.post("/signup", response_model=UserResponse)
async def signup(request: Request, user: UserCreate):
    # Check if the username already exists
    query = "SELECT username FROM users2 WHERE username = %(username)s"
    parameters = {'username': user.username}

    result = await request.state.clickhouse_pool.query(query, parameters=parameters)

    if result.result_rows:
        raise HTTPException(status_code=400, detail="Username already registered")

    hashed_password = await get_password_hash(user.password)

    user_query = """
    INSERT INTO users2 (username, email, password, status, role)
    VALUES (%(username)s, %(email)s, %(password)s, 0, 'analyst')
    """
    user_parameters = {
        'username': user.username, 
        'email': user.email, 
        'password': hashed_password
    }
    await request.state.clickhouse_pool.query(user_query, parameters=user_parameters)

    return UserResponse(username=user.username, email=user.email, status=False, role="analyst")```
#

should i just copy all in api.py

barren sluice
#

yes

#

just like with models

vocal ginkgo
#

what about func use inside it

barren sluice
#

move all routes into api

barren sluice
vocal ginkgo
#

get_password_hash verify_role get_current_user create_access_token verify_password cache_data get_data_from_cache

#

these

barren sluice
#

you add t hem to auth.py and import them

#

since those are based on auth

#

looks like JWT stuff right?

vocal ginkgo
#

yes

barren sluice
#

here is a good point, are you ready?

vocal ginkgo
#

yes

#

let me list all these functions

barren sluice
#

you CANNOT import those functions from main.py

vocal ginkgo
#

get_data_from_cache , cache_data for cache

barren sluice
vocal ginkgo
#

get_password_hash verify_password create_access_token get_current_user verify_role

barren sluice
barren sluice
#

i made this in py repo, thinking, i will for sure have many more things

#

but i ended up with only a few things, so no need for a infrastructure package

vocal ginkgo
#

should i create folder called infrastrure or cache ?

#

here

#

i have a question

#

i have a file where i have many function which are used inside apis

barren sluice
vocal ginkgo
#
from HealthMonitor import (
    get_zeek_health, get_suricata_health, process_subnet_data,
    modify_suricata_config
)```
#

where to add these

vocal ginkgo
#

ok creating folder called infrastructure

barren sluice
#

so have all your infrastructure code in this package

barren sluice
vocal ginkgo
#

all redis , auth and other functions?

barren sluice
#

name it for what it is

vocal ginkgo
vocal ginkgo
barren sluice
vocal ginkgo
#

and funcs.py where i have some other functions

from funcs import (
    get_zeek_health, get_suricata_health, process_subnet_data,
    modify_suricata_config
)```
barren sluice
vocal ginkgo
#

yes pls

barren sluice
vocal ginkgo
#

no more then this

barren sluice
#

tell me what more

vocal ginkgo
#

remember we create a func which remove color from output

barren sluice
#

yes

vocal ginkgo
#

yes this is that file

#

and some other function which create some configuration management in backend tools

barren sluice
#

so names are important, and the names you choose should reflect what the thing is

vocal ginkgo
#

suricata_remove_ansi_color_codes_from_output rememebr this

barren sluice
#

funcs.py does not really mean a lot

vocal ginkgo
#

yes i am going to change it now

barren sluice
#

monitor/health.py and monitor/temperature.py makes a lot of more sense. given that you have a health monitor and a temperatur monitor

#

do you agree?

#

naming things after what they are, like in this made up example, a health monitor, that is called duck typing

vocal ginkgo
#

yes thats correct

barren sluice
#

so when i tell you to name it for what it is, that means, you have to decide, i cannot tell you, i dont know what the thing is

vocal ginkgo
#

what if i called this file helpers.py ?

barren sluice
vocal ginkgo
#

yes

barren sluice
#

and if helpers.py gets to large, you can just refactor it again

barren sluice
vocal ginkgo
#

yes i think i'll do it this helper later on

#

wiat

#

wait

#
from funcs import (
    get_zeek_health, get_suricata_health, process_subnet_data,
    modify_suricata_config
)```
these are in helpers.py
vocal ginkgo
barren sluice
barren sluice
#

see, name it for what it is

vocal ginkgo
#

first i create folder infrastructure inside i create cache.py ?

#

agree?

barren sluice
vocal ginkgo
#

in my apis i am storing result in cache to not call DB again and again

barren sluice
#

so this is data you use in your api?

vocal ginkgo
#

yes let me show you

#
    cache_key = f"top_application_{duration}_{unit}_{hash(query)}"
    cached_data = await get_data_from_cache(request ,cache_key)
    if cached_data:
        print("From Redis Cache")
        return ORJSONResponse(content=cached_data)

    try:
        print("From Clickhouse")
        parameters = {"duration": duration}
        data = await request.state.clickhouse_pool.query(query , parameters=parameters)
        # Extracting rows and columns from the query result
        columns = data.column_names  # Get column names from the result
        rows = data.result_rows      # Get the actual data rows

        df = pd.DataFrame(rows, columns=columns)
        result_json = orjson.loads(df.to_json(orient='records'))

        # Cache the result
        await cache_data(request, cache_key, result_json)

        return ORJSONResponse(content=result_json)
    except Exception as e:
        return ORJSONResponse(content={"error": str(e)}, status_code=500)```
strong windBOT
#

Hey @vocal ginkgo!

It looks like you pasted Python code without syntax highlighting.

Please use syntax highlighting to improve the legibility of your code and make it easier for us to help you.

To do this, use the following method:
```py
print('Hello, world!')
```

This will result in the following:

print('Hello, world!')```
You can **edit your original message** to correct your code block.
barren sluice
#

yes, then it makes sense to call it generic cache.py is reasonable

vocal ginkgo
#

alright

barren sluice
#

my cache is spefic only handeling weather data from the weather api

#

so i called it weather_cache

vocal ginkgo
#

in my case it handles all apis data

barren sluice
#

notice that i focus on how I look at it

#

not what it does

#

that is the hallmark of a good api

#

it should be self evident what it does

#

i would like to write this, without knoowing your code in detail

vocal ginkgo
#
import json
from fastapi import Request

async def get_data_from_cache(request: Request, query):
    cached_data = await request.state.redis_pool.get(query)
    if cached_data:
        return json.loads(cached_data)
    return None

async def cache_data(request: Request, query, result):
    cache_entry = json.dumps(result)
    await request.state.redis_pool.setex(query, 60, cache_entry)```
barren sluice
#
from infrastrucutre.cache import get_data_from_cache
vocal ginkgo
#

alright

barren sluice
#

do a cleanup in main.py and run the code again to see that it still working

#

ofc, check that the caching is working, since you have moved that around

vocal ginkgo
#

alright

#

yes next

#
async def get_password_hash(password: str) -> str:
    return pwd_context.hash(password)

async def verify_password(plain_password: str, hashed_password: str) -> bool:
    return pwd_context.verify(plain_password, hashed_password)


async def create_access_token(data: dict, expires_delta: timedelta = None) -> str:
    to_encode = data.copy()
    if expires_delta:
        expire = datetime.utcnow() + expires_delta
    else:
        expire = datetime.utcnow() + timedelta(minutes=15)
    to_encode.update({"exp": expire})
    encoded_jwt = jwt.encode(to_encode, SECRET_KEY, algorithm=ALGORITHM)
    return encoded_jwt

async def get_current_user(token: str = Depends(oauth2_scheme)) -> UserResponse:
    print("get_current_user")
    credentials_exception = HTTPException(
        status_code=401,
        detail="Could not validate credentials",
        headers={"WWW-Authenticate": "Bearer"},
    )
    print (f"Received token: {token}")
    try:
        payload = jwt.decode(token, SECRET_KEY, algorithms=[ALGORITHM])
        print(f"Decoded payload: {payload}")
        username: str = payload.get("sub")
        email: str = payload.get("email")
        status: bool = payload.get("status")
        role: str = payload.get("role")
        if not username or not email or not status or not role:
            raise credentials_exception
    except JWTError as e:
        print(f"JWT Error: {str(e)}")
        raise credentials_exception

    return UserResponse(username=username, email=email, status=status, role=role)

# Verify Used As Dependency Injection In Routes For Role Base Access
def verify_role(required_roles: List[str]):
    def role_dependency(current_user: UserResponse = Depends(get_current_user)):
        if current_user.role.lower() not in [role.lower() for role in required_roles]:
            raise HTTPException(
                status_code=status.HTTP_403_FORBIDDEN,
                detail="Insufficient privileges",
            )

    return role_dependency```
#

these are auty functions

#

and this

class ORJSONResponse(ORJSONResponse):
    def render(self, content: Any) -> bytes:
        return orjson.dumps(content)```
barren sluice
#

same here, refactor by moving code into new modules, import them back to main.py

vocal ginkgo
#

ok

barren sluice
#

in the end, main.py will be succinct

#

it will end up only doing one thing, setting up the app and running the web server

vocal ginkgo
#

there is one more this

#

thing

#

!paste

strong windBOT
#
Pasting large amounts of code

If your code is too long to fit in a codeblock in Discord, you can paste your code here:
https://paste.pythondiscord.com/

After pasting your code, save it by clicking the Paste! button in the bottom left, or by pressing CTRL + S. After doing that, you will be navigated to the new paste's page. Copy the URL and post it here so others can see it.

vocal ginkgo
barren sluice
#

what is this?

vocal ginkgo
#

this is .env configuration

barren sluice
#

yes i agree, it is configuration

#

so what good module name could you use for that?

vocal ginkgo
#

you suggest this

#

from dotenv import load_dotenv

barren sluice
#

thats takes secrets and puts it into your environemnt, you still have to use them

#

what about config.py or configuration.py ?

vocal ginkgo
#

yes or if there is any good way other than this

#

yes thats correct

barren sluice
vocal ginkgo
#

๐Ÿ˜›

barren sluice
#

yes

vocal ginkgo
#

ok just a se

barren sluice
#

see, i dont have to tell you, you can just do it! ๐Ÿ˜„

#

there are ofc more refactoring you can do, but step one is this, move code to new moduels

vocal ginkgo
#

no no you are my boss for the moment:D

#

just a sec

barren sluice
#

what is lifespan used for?

vocal ginkgo
#

for staring the redis and clickhouse database connection when app start

barren sluice
#

so its app related?

#

you should put all those things in app.py

#

that way you can do from app import app where app is the created app

vocal ginkgo
#
import os
from dotenv import load_dotenv

# Load environment variables from .env file
load_dotenv(override=True)

# JWT Configuration
SECRET_KEY = os.getenv("SECRET_KEY")
ALGORITHM = os.getenv("ALGORITHM")
CACHE_EXPIRATION_TIME = int(os.getenv("CACHE_EXPIRATION_TIME"))
REVOKED_TOKENS_EXPIRE_SECONDS = int(os.getenv("REVOKED_TOKENS_EXPIRE_SECONDS"))
ACCESS_TOKEN_EXPIRE_MINUTES = int(os.getenv("ACCESS_TOKEN_EXPIRE_MINUTES"))

# ClickHouse SENSOR Configuration
CLICKHOUSE_HOST = os.getenv("CLICKHOUSE_HOST")
CLICKHOUSE_PORT = int(os.getenv("CLICKHOUSE_PORT"))
CLICKHOUSE_DATABASE = os.getenv("CLICKHOUSE_DATABASE")
CLICKHOUSE_USER = os.getenv("CLICKHOUSE_USER")
CLICKHOUSE_PASSWORD = os.getenv("CLICKHOUSE_PASSWORD")

# ClickHouse BRAIN Configuration
CLICKHOUSE_BRAIN_HOST = os.getenv("CLICKHOUSE_BRAIN_HOST")
CLICKHOUSE_BRAIN_PORT = int(os.getenv("CLICKHOUSE_BRAIN_PORT"))
CLICKHOUSE_BRAIN_DATABASE = os.getenv("CLICKHOUSE_BRAIN_DATABASE")
CLICKHOUSE_BRAIN_USER = os.getenv("CLICKHOUSE_BRAIN_USER")
CLICKHOUSE_BRAIN_PASSWORD = os.getenv("CLICKHOUSE_BRAIN_PASSWORD")

# Redis Configuration
REDIS_URL = os.getenv("REDIS_URL")```

this is config.py
barren sluice
#

or depening on what you use, import the router

vocal ginkgo
#

how to import this config in main

barren sluice
#

so you want to import almost all names from this config module right?

vocal ginkgo
#

yes all of them are used

#
from config import (
    SECRET_KEY, ALGORITHM, CACHE_EXPIRATION_TIME, REVOKED_TOKENS_EXPIRE_SECONDS,
    ACCESS_TOKEN_EXPIRE_MINUTES, CLICKHOUSE_HOST, CLICKHOUSE_PORT, CLICKHOUSE_DATABASE,
    CLICKHOUSE_USER, CLICKHOUSE_PASSWORD, CLICKHOUSE_BRAIN_HOST, CLICKHOUSE_BRAIN_PORT,
    CLICKHOUSE_BRAIN_DATABASE, CLICKHOUSE_BRAIN_USER, CLICKHOUSE_BRAIN_PASSWORD, REDIS_URL
)``` like this ?
barren sluice
#

let me tell you what you need to do

strong windBOT
#

Hey @vocal ginkgo!

It looks like you pasted Python code without syntax highlighting.

Please use syntax highlighting to improve the legibility of your code and make it easier for us to help you.

To do this, use the following method:
```py
print('Hello, world!')
```

This will result in the following:

print('Hello, world!')```
You can **edit your original message** to correct your code block.
vocal ginkgo
#

yes

barren sluice
#

and then you will think, thats a bit to much work, is there a better way

barren sluice
vocal ginkgo
#

yes

barren sluice
#

is there a better way?

vocal ginkgo
#

thats what i am thinking

barren sluice
#

yes there is

vocal ginkgo
#

wew like how

barren sluice
#

now, there is a problem with the next answer

vocal ginkgo
#

:

barren sluice
#

so let me tell you about it before you continue

vocal ginkgo
#

๐Ÿ˜ฆ

barren sluice
#

you need to import all

#

thats from config import *

vocal ginkgo
#

wew samll

barren sluice
#

the problem with this, is that you will polute the global namespace with names you dont control

#

you you need to control what * does

#

that means you have to write everything you want to import

barren sluice
#

but you write it another place

vocal ginkgo
#

oh

barren sluice
#

open your config.py file and make a new list on line 1

#
__all__ = ['a', 'b']
a = 3
b = 4
c = 5
#

the list must be called __all__

#

and the list must only contain strings

#

those strings are the names of the variables you are going to import

#

in my example, c will not be imported when you use * import

vocal ginkgo
#
__all__ = [
    "SECRET_KEY", "ALGORITHM"]

# JWT Configuration
SECRET_KEY = os.getenv("SECRET_KEY")
ALGORITHM = os.getenv("ALGORITHM")

like this

vocal ginkgo
#

how to import this

barren sluice
#

from config import *

#

star import

#

you must never do star import without this __all__ guard

thick grotto
#

Alternatively, you could use import config and reference the values like config.SECRET_KEY

barren sluice
#

that requrie you to rewrite the names where you use them

#

and sometimes that is a better solution

vocal ginkgo
#

its said CLICKHOUSE_BRAIN_HOST is not define

barren sluice
#

what said that where?

vocal ginkgo
barren sluice
#

share the traceback

vocal ginkgo
#

"CLICKHOUSE_HOST" is not definedPylancereportUndefinedVariable

barren sluice
#

editor warning?

vocal ginkgo
#

yes

#

i think its not importing

barren sluice
#

run the code

#

python will tell you

vocal ginkgo
#
import os
from dotenv import load_dotenv


load_dotenv(override=True)


__all__ = [
    "SECRET_KEY", "ALGORITHM", "CACHE_EXPIRATION_TIME", "REVOKED_TOKENS_EXPIRE_SECONDS",
    "ACCESS_TOKEN_EXPIRE_MINUTES", "CLICKHOUSE_HOST", "CLICKHOUSE_PORT", "CLICKHOUSE_DATABASE",
    "CLICKHOUSE_USER", "CLICKHOUSE_PASSWORD", "CLICKHOUSE_BRAIN_HOST", "CLICKHOUSE_BRAIN_PORT",
    "CLICKHOUSE_BRAIN_DATABASE", "CLICKHOUSE_BRAIN_USER", "CLICKHOUSE_BRAIN_PASSWORD", "REDIS_URL"
]

# JWT Configuration
SECRET_KEY = os.getenv("SECRET_KEY")
ALGORITHM = os.getenv("ALGORITHM")
CACHE_EXPIRATION_TIME = int(os.getenv("CACHE_EXPIRATION_TIME"))
REVOKED_TOKENS_EXPIRE_SECONDS = int(os.getenv("REVOKED_TOKENS_EXPIRE_SECONDS"))
ACCESS_TOKEN_EXPIRE_MINUTES = int(os.getenv("ACCESS_TOKEN_EXPIRE_MINUTES"))

# ClickHouse SENSOR Configuration
CLICKHOUSE_HOST = os.getenv("CLICKHOUSE_HOST")
CLICKHOUSE_PORT = int(os.getenv("CLICKHOUSE_PORT"))
CLICKHOUSE_DATABASE = os.getenv("CLICKHOUSE_DATABASE")
CLICKHOUSE_USER = os.getenv("CLICKHOUSE_USER")
CLICKHOUSE_PASSWORD = os.getenv("CLICKHOUSE_PASSWORD")

# ClickHouse BRAIN Configuration
CLICKHOUSE_BRAIN_HOST = os.getenv("CLICKHOUSE_BRAIN_HOST")
CLICKHOUSE_BRAIN_PORT = int(os.getenv("CLICKHOUSE_BRAIN_PORT"))
CLICKHOUSE_BRAIN_DATABASE = os.getenv("CLICKHOUSE_BRAIN_DATABASE")
CLICKHOUSE_BRAIN_USER = os.getenv("CLICKHOUSE_BRAIN_USER")
CLICKHOUSE_BRAIN_PASSWORD = os.getenv("CLICKHOUSE_BRAIN_PASSWORD")

# Redis Configuration
REDIS_URL = os.getenv("REDIS_URL")
barren sluice
#

looks correct to me, does it not work when you run it?

vocal ginkgo
#

nope

#

oh wait

barren sluice
#

share the error

vocal ginkgo
#

i think

#

?

barren sluice
vocal ginkgo
#

.env also presetn

barren sluice
#

that is fine, just run and see

vocal ginkgo
#

i think its working

#

yes

barren sluice
#

there you go!

#

when you move code like we are doing

#

it will always work

#

that is because python runs code from top to bottom, left to right.

vocal ginkgo
#

yes perfect

barren sluice
#

and when you import code, python will run that code

vocal ginkgo
#

oh alright

barren sluice
#

the next step in refactoring is what @thick grotto meantioned above, but that requires refactoring and rewriting names

vocal ginkgo
barren sluice
#

you can just continue until your main.py is succinct

vocal ginkgo
#
# Password hashing functions
async def get_password_hash(password: str) -> str:
    return pwd_context.hash(password)

async def verify_password(plain_password: str, hashed_password: str) -> bool:
    return pwd_context.verify(plain_password, hashed_password)


async def create_access_token(data: dict, expires_delta: timedelta = None) -> str:
    to_encode = data.copy()
    if expires_delta:
        expire = datetime.utcnow() + expires_delta
    else:
        expire = datetime.utcnow() + timedelta(minutes=15)
    to_encode.update({"exp": expire})
    encoded_jwt = jwt.encode(to_encode, SECRET_KEY, algorithm=ALGORITHM)
    return encoded_jwt

async def get_current_user(token: str = Depends(oauth2_scheme)) -> UserResponse:
    print("get_current_user")
    credentials_exception = HTTPException(
        status_code=401,
        detail="Could not validate credentials",
        headers={"WWW-Authenticate": "Bearer"},
    )
    print (f"Received token: {token}")
    try:
        payload = jwt.decode(token, SECRET_KEY, algorithms=[ALGORITHM])
        print(f"Decoded payload: {payload}")
        username: str = payload.get("sub")
        email: str = payload.get("email")
        status: bool = payload.get("status")
        role: str = payload.get("role")
        if not username or not email or not status or not role:
            raise credentials_exception
    except JWTError as e:
        print(f"JWT Error: {str(e)}")
        raise credentials_exception

    return UserResponse(username=username, email=email, status=status, role=role)


def verify_role(required_roles: List[str]):
    def role_dependency(current_user: UserResponse = Depends(get_current_user)):
        if current_user.role.lower() not in [role.lower() for role in required_roles]:
            raise HTTPException(
                status_code=status.HTTP_403_FORBIDDEN,
                detail="Insufficient privileges",
            )

    return role_dependency```

i put all of this under auth.py?
#
oauth2_scheme = OAuth2PasswordBearer(tokenUrl="login")

# Password hashing
pwd_context = CryptContext(schemes=["argon2"], deprecated="auto")```

and put this in `context.py` ?
barren sluice
#

yes anything connected to authentication, that is not a route, should go there

barren sluice
#

there are no one right way

#

they are all right ways as long as people watching and you, understand it

vocal ginkgo
#

so i go like you

barren sluice
#

to me context makes sense, but schemes also works

vocal ginkgo
#

so

#
from fastapi.security import OAuth2PasswordBearer
from passlib.context import CryptContext

oauth2_scheme = OAuth2PasswordBearer(tokenUrl="login")
# Password hashing
pwd_context = CryptContext(schemes=["argon2"], deprecated="auto")```
#

how to call this in main

#

it is in auth->context.py

barren sluice
#

its always the same, you import it where you use it

vocal ginkgo
#

i think didnt need to import

barren sluice
#

you have to import, or the code will not be run

vocal ginkgo
#

from app.context import ?

barren sluice
#

yes

vocal ginkgo
#

import ? what

barren sluice
#

the names you are using

#

im guessing you are using oauth2_scheme and pwd_context

#

you will see this in your editor

#

because when you move the code, the editor will light up where you have a missing import

vocal ginkgo
#

yes dont

#

done

#

what about auth functions

#

maybe called user_management.py ?

barren sluice
#

yeah, great name

vocal ginkgo
#

oauth2_scheme = OAuth2PasswordBearer(tokenUrl="token") this is using in get_current_user and verify_role

#

should i import like

from app.context import oauth2_scheme ?

barren sluice
#

yes

#

import it where you use it, it can be multiple places

vocal ginkgo
#

now the api part is left

barren sluice
#

and the other is used in your password hashing functions, so you have to import pwd_context there

vocal ginkgo
#

yes done

#

now i want to create folder call v1 where i want to put all my apis inside it

barren sluice
#

do you want to version your code like that?

vocal ginkgo
#

yes

#

yes

barren sluice
#

are you going to run multiple versions of the same project using differnt code versions?

vocal ginkgo
#

huh for this version i have some apis only for next version i add new apis

#

with the previous one

barren sluice
#

and when you add a new version, will the old v1 still run and work?

vocal ginkgo
#

yes

barren sluice
#

so no top level application running sub applications for each version?

#

i think i have a hard time seeing what you want to do here

vocal ginkgo
#

let me explin

#

i have to put all apis in v1

#

for v2 if tehre is change in v1 i modify them

barren sluice
#

so make a sub application

#

thats what it is for

#

and fastapi handles the root_path details for you

#

that way you dont have to polute your code with version details

#

you can just import the routes you want in your sub app

#

or v1 app

vocal ginkgo
#

yes i think will do this later

#

for the moment sepraet the api code

#

for each endpoint create new .py file ?

barren sluice
#

no

#

never do that

#

add things in groups for what they are

vocal ginkgo
#

so now how to move all apis

barren sluice
#

if you have endpoints about a user you can make a user.py

vocal ginkgo
#

ok

barren sluice
#

if you have endpoints about SalesOrders make an order.py

#

if you only have five enpoints, you dont have to split them up, just keep it all in your api endpoints module

vocal ginkgo
#

no no i have many endpoint some are for configuraiton some for user and some for stats

barren sluice
#

then you do as you have done so far.

#

move code, fix imports, do cleanup, run and see that everything works

vocal ginkgo
#

i craete apis folder

#

now i am creating users.py inside it for user endpoint

#

what about this api

# Route to login
@router.post("/login")
async def login(request: Request, form_data: OAuth2PasswordRequestForm = Depends()):
    # Query to get user details including password and status status
    query = "SELECT password, status, role, email FROM users2 WHERE username = %(username)s"
    parameters = {"username": form_data.username}

    result = await request.state.clickhouse_pool.query(query, parameters=parameters)

    # Check if result is empty
    if not result.result_rows:
        raise HTTPException(status_code=401, detail="Incorrect username or password")

    hashed_password, status, role, email = result.result_rows[0]

    # Verify the provided password
    if not await verify_password(form_data.password, hashed_password):
        raise HTTPException(status_code=401, detail="Incorrect username or password")

    # Check if the user is active
    if not status:
        raise HTTPException(status_code=403, detail="User account is not active")

    # Generate the access token with all necessary fields
    access_token_expires = timedelta(minutes=ACCESS_TOKEN_EXPIRE_MINUTES)
    access_token = await create_access_token(
        data={"sub": form_data.username, "email": email, "status": status, "role": role},
        expires_delta=access_token_expires
    )

    content = {"access_token": access_token, "token_type": "bearer"}

    return content```

its using 

`ACCESS_TOKEN_EXPIRE_MINUTES` env variable
barren sluice
#

what do you think you have to do?

vocal ginkgo
#

i think from config.py import ACCESS_TOKEN_EXPIRE_MINUTES ?

barren sluice
#

try it and check if that works

#

i have to leave in a moment, just a heads up

vocal ginkgo
#

Ok what about router import do I need to import it in all endpoint files like in apis->users.py and apis->health.py

barren sluice
#

you always have to import them where you use them

#

python will handle imports, you just have to write the lines

#

good luck! update me later tonight!

vocal ginkgo
#

router = FASTAPI() in each file then in main.py what to import I am confused

strong windBOT
#
Python help channel closed

This help channel has been closed and it's no longer possible to send messages here. If your question wasn't answered, feel free to create a new post in #1035199133436354600. To maximize your chances of getting a response, check out this guide on asking good questions.