unit-testing

[muted cargo]

Can someone help me mock sftp for unit-testing
I'll need the functionalities, read write and delete, and also check if it exists

[vernal juniper]

Can't mock aiohttp ClientSession.get

E TypeError: object MockResponse can't be used in 'await' expression

mock_response_list = test.mocks.MockResponse(
    "<div class='top-item'><a href='/123/'></a></div>", 200
)

@pytest.mark.asyncio
async def test_list_parser(mocker):
    parser = src.html_parser.ListParser(src.classes.Card)
    mocker.patch(
        "src.html_parser.aiohttp.ClientSession.get", return_value=mock_response_list
    )
    async with aiohttp.ClientSession() as session:
        parser.set_session(session)
        res = await parser.get_page(1)
        assert res == [123]

Am I doing something wrong here?

[maiden pawn]

u need some kind of AsyncMockResponse
or wrap your current mock into primitive async function that u can use for mock-replacement

[vernal juniper]

How do I do that

[maiden pawn]

https://www.google.com/search?q=Python+Async+MockResponse&oq=Python+Async+MockResponse&aqs=chrome..69i57j0i13j0i13i30l2j0i8i13i30l6.3667j0j4&sourceid=chrome&ie=UTF-8

https://stackoverflow.com/questions/32480108/mocking-async-call-in-python-3-5
google around "python async mockresponse"?

i haven't used async enough to give you answer

i just know that async stuff implements __some kind of special snowflake dunder methods__

code that works with async should be implementing those dunders, to work correctly

[vernal juniper]

My MockResponse has all of 'em

[maiden pawn]

i like comments from here https://stackoverflow.com/questions/32480108/mocking-async-call-in-python-3-5
looks like cool ideas

[lethal sparrow]

I'm looking to migrate away from nosetests. Should I consider any other testing frameworks apart from pytest? (I know about unittest and nose2, not sure if there's any other widely used ones.)

[weary quarry]

I would suggest pytest having started with unittest and not having experience with nose. They are just what I think they should be, as simple or complex as you need them. The built-in decorators for fixtures and parameterization are easy and valuable.

Everything just working with normal assert statements is a bonus.

[jolly tangle]

Quick Pytest question. I have @pytest.fixture(scope="") in one fixture. I want the scope to run more then once per function how do I do that? Thanks

[viscid forge]

I don't think a standard solution exists for this, how would it even know when to run? One thing you can do is have your fixture return a callable, then inside your test call it at the right points

@pytest.fixture()
def fixture():
    print("Entring fixture")

    def function():
        print("Calling the function")

    yield function
    print("Exiting fixture")


def test_something(fixture):
    print("Starting test")
    fixture()

stdout:

Entring fixture
Starting test
Calling the function
Exiting fixture

[vernal juniper]

Hello!
Is there a way to test whether an exception has been handled, even if it only invokes pass?

[viscid forge]

Well if it wasn't handled it would bubble up and fail the test, no?

[vernal juniper]

Sure.

Is there a way to count a line as covered? if __name__ == "__main__" is making my beautiful 100% coverage report into a 99% coverage report D:

Also asyncio.run(main()), no idea how to test that.

[jolly tangle]

@viscid forge

I assume you just mean the yield function can run more then once. If I am mistaken please correct me. Also I am little confused by the output. Sorry for the dumb question.

[weary quarry]

Use # pramga: no cover comment on that line or define excluded lines in your coverage config. https://github.com/Preocts/python-src-template/blob/main/pyproject.toml#L84-L89

[bitter wadi]

pyproject.toml lines 84 to 89

[tool.coverage.report]
exclude_lines =[
    "pragma: no cover",
    "raise NotImplementedError",
    "if __name__ == .__main__.:"
]```

[knotty tusk]

Hello guys ! I'm having problem mocking the __init__.py of a folder, do you have solution ?

• mocker.path("folder.__init__.Class") not working
• mocker.path("folder.Class") not working

[hexed cloak]

Oh yeah someone started working on a pytest alternative

[hexed cloak]

https://ward.readthedocs.io/en/latest/guide/fixtures.html#fixtures

There's also https://github.com/adriangb/di but no integration with assert rewriting

[lethal sparrow]

that's not a test framework?

[hexed cloak]

No it's a fixture framework

[lethal sparrow]

thanks! looks much less mature and harder to migrate to

[hexed cloak]

Lol

[lethal sparrow]

we have 23749 tests, I don't want to rewrite all those to use an @test decorator

[hexed cloak]

Ah

[hexed cloak]

You can just run it with subprocess.run([sys.executable, '-m', ...], check=True) or https://docs.python.org/3/library/runpy.html#runpy.run_module

[hexed cloak]

There's unittest2pytest

It needs some supervision to work though

[lethal sparrow]

the existing tests are in nosetests, pytest should mostly support them already

[hexed cloak]

I used to use nose for everything

make sure you check the coverage of your test suite

It's easy to accidentally configure some tests to not run

[violet wind]

Is it best practice to run unit tests against an installable package?

[maiden pawn]

it depends on what it is doing. and amount of its realibility.

is it wise to check purely only its code? in general no.
is it wise to test your code working in integration with installable package code, that augments your code through some mixins? I would say yes. at least one test
is it your installable package? in general it would be recommended to increase its testing coverage.... u should just add tests inside of it more.
is it not your package, and u added few tests, which document how to interact it with it? that's good to do too.

[abstract trellis]

8years working on my little python app and not a SINGLE TEST file 🙃 . Any good suggestions on testing frameworks? ( preferred native, small, colorful outputs)

[weary quarry]

Is the question along of the lines of; if I install requests should I write tests that check requests functionality?

[weary quarry]

pytest is what I would recommend.

[abstract trellis]

Why Pytest? Are you using pytest? What puts it over other frameworks?

[weary quarry]

It works. It's easy to build tests with. It works seamlessly with the existing assert statement. It works. Yes I build all my tests with pytest. It works.

[abstract trellis]

😆 that works for me

[weary quarry]

There are allow many builtin features for fixtures, control over when those rebuild, parameterization of tests, and so much more.

I've been using it for two years and I've hardly scratched the surface of what options it has.

I like simple tests with a framework that doesn't need a bunch of boilerplate. https://github.com/Preocts/runtime-yolk/blob/main/tests/yolk_test.py#L48-L53

[bitter wadi]

tests/yolk_test.py lines 48 to 53

def test_config_load_specific_ini() -> None:
    yolk = Yolk(working_directory=FIXTURE_PATH)

    yolk.load_config("not-application")

    assert yolk.config.get("DEFAULT", "yolk_test") == "eggshell"```

[maiden pawn]

😱

obviously pytest yeah

it has the best interface, and easy to use syntax

[abstract trellis]

Yes thank you good peoples!

Is the term fixtures another way of saying mock data ?

[weary quarry]

Not really. Mock data is, well, mocking. A fixture is a bit of setup, such as initializing a class or setting up the environment, which can be used by test functions. Fixtures reduce code used during the arrange step of the test.

You can use mock data in fixture or replace portions of the target class/module with stubs.

So if you have a groups of tests that are all going to assume conditions XYZ exist, you'd create a fixture to build those conditions once then use that fixture in the tests as many times as needed.

In a good testing setup, anything done by the fixture is undone once the test it completed leaving the next test to run in an unpolluted state.

[maiden pawn]

we can control scope of a fixture in python though, if it should be replaced after each function / module / or only session testing
which is convinient for the purpose of running tests with less time consumption which is especially important in python

less time reinitializing stuff and using precious time... at the cost of getting our tests potentially... polluting each other xD

[violet wind]

So its probably okay to write my unit tests against unmanaged code and then running integration tests against a pip package. Is there any best practices I should follow to structure my project so that I can run my tests against a CI pipeline?

[maiden pawn]

Technically no, and technically yes

[maiden pawn]

there are only two recommendations I think

1. Make your package importing in a way that is not requiring any PYTHONPATH hacks to discovery additional packages

2. It is really the best if u just build docker image of your project, and then reuse it to launch CI tests / running in staging / production

utilizing in a smart way docker-compose is really sneaky way to approach it

ensuring that launch of tests in dev environment is equally simple in CI

Hmm... may be I could write a public code to show it, right now? 🤔

[violet wind]

Nah that kinda make sense. I was just wanting to unit test modules of my package that I'm not exposing externally.

Oh

No you mean dockerize my dev environment

Yeah that make sense now

[maiden pawn]

as for structure. People go for two ways to structure tests
Project/
Tests/
Test_group/
test_bla

or they go for

Project/
Folder1/
tests.py
Folder2/
tests.py

I prefer the second option

[maiden pawn]

yes. It makes easy and ensuring that u use same environment / dependency services in dev env and CI and they are written only ONCE xD

in CI it is just docker-compose run app pytest, which will raise dependency services automaticlaly and run your tests

[violet wind]

Yeah that's a smart idea. I shall do that

[maiden pawn]

i want to ensure auto destruction of auxilary services though 🤔

docker-compose -p SUPER_RANDOM_NUMBER(for example unique pipeline ID) run --rm scrappy_base pytest; docker-compose down this sholuld be technically sufficient solution, if bash would work reliably in CI

[maiden pawn]

this command should ensure probably best approach to launch it? 🤔

-p flag ensures that it is differently named project / containers / networks and etc

;docker-compose down should ensure container destruction after it is done

And we can also... make some script generalizing how to launch it to a simpler command for reusage in dev env and CI env

[maiden pawn]

yay. made it

version: '3.8'
services:
  db:
    image: postgres:13
    expose:
      - "5432"
    environment:
      POSTGRES_HOST_AUTH_METHOD: trust
      POSTGRES_DB: default
      POSTGRES_USER: postgres
      POSTGRES_PASSWORD: postgres

in docker compose we use only expose to expose port only to other containers but not to host

so we would prevent... binding error if two jobs are running at same time
then we have simple makefile in python edition (no third party libs were used)

import argparse
import os
import secrets
from enum import Enum, auto, EnumMeta

class _EnumDirectValueMeta(EnumMeta):
    def __getattribute__(cls, name):
        value = super().__getattribute__(name)
        if isinstance(value, cls):
            value = value.name
        return value

class _EnumGetKey(Enum):
    @classmethod
    def get_keys(cls):
        return [e.name for e in cls]

class EnumWithValues(_EnumGetKey, metaclass=_EnumDirectValueMeta):
    pass

class Actions(EnumWithValues):
    test = auto()
    shell = auto()
    run = auto()
    lint = auto()

class Services(EnumWithValues):
    scrappy = auto()

def shell(cmd):
    exit(os.system(cmd))

class Parser:
    def __init__(self):
        self._parser = argparse.ArgumentParser()
        self._parser.add_argument('--job_id', type=str, default=secrets.token_hex(4),
            help="optional parameter random by default, ensures to run docker-compose with random -p parameter for no conflicts in parallel runs",
        )
        self._parser.add_argument('service', type=str, choices=Services.get_keys())

    def parse_all(self):
        args = self._parser.parse_args()
        return args

    def parse_service_only(self):
        args, argv = self._parser.parse_known_args()
        return args

    def registher_actions(self, *actions):
        self._parser.add_argument('action', type=str, choices=actions)

class CommandExecutor:
    def __init__(self, parser: Parser):
        self._parser = parser

    @property
    def args(self):
        return self._parser.args

    def run_inside_container(self, command):        
        return_code = os.system(
            f"docker-compose -f docker-compose.{self.args.service}.yml -p {self.args.job_id} build && "
            f"docker-compose -f docker-compose.{self.args.service}.yml  -p {self.args.job_id}"
            f" {command}"
        )
        os.system(
            f"docker-compose -f docker-compose.{self.args.service}.yml  -p {self.args.job_id} down"
        )
        print(return_code)
        if return_code != 0:
            raise Exception(f"non zero returned code={return_code}")

class CommonCommands:
    test = "run --rm service_base pytest"
    shell = "run --rm service_base /bin/bash"
    run = "up"
    lint = "run --rm service_base black --check ."

def main():
    args = Parser().parse_service_only().args

    match args.service:
        case Services.scrappy:
            parser = Parser().registher_actions(Actions.test, Actions.shell, Actions.run, Actions.lint).parse_all()
            executor = CommandExecutor(parser)

    match (parser.args.service, parser.args.action):
        case (Services.scrappy, Actions.test):
            executor.run_inside_container(CommonCommands.test)
        case (Services.scrappy, Actions.shell):
            executor.run_inside_container(CommonCommands.shell)
        case (Services.scrappy, Actions.run):
            executor.run_inside_container(CommonCommands.run)
        case (Services.scrappy, Actions.lint):
            executor.run_inside_container(CommonCommands.lint)
        case _:
            raise Exception("Not registered command for this service")

if __name__=="__main__":
    main()

python3 make.py test launches it, and supplied -p thing ensures there is no conflict in raised naming of the stuff
(technically it could be set as environment variable but whatever)

we can add now to this makefile linting and other optional stuff that would be invoked via our containers

we can utilize parallel running CI runner without conflicts between running jobs now

[maiden pawn]

had fun and configured further xD

dammit. return_code is not returned to shell correctly 🤔

Fixed

@violet wind

$ python3 make.py --help
usage: make.py [-h] [--job_id JOB_ID] {scrappy} {test,shell,run,lint}

positional arguments:
  {scrappy}
  {test,shell,run,lint}

options:
  -h, --help            show this help message and exit
  --job_id JOB_ID       optional parameter random by default, ensures to run docker-
                        compose with random -p parameter for no conflicts in parallel
                        runs

check this out. Self documented all available commands. Easy to document in README.md. Easy to launch in dev env and equally same in CI

I just launch python3 make.py scrappy lint for example

[violet wind]

Thank you so much for this, definitely looks like it's going to help me solve me problems. Currently at my dayjob so not going to have a chance to try it out until later on haha

[maiden pawn]

For full happiness I am missing only to have it in a future written for cross different environment network communications

I ll adjust it when a second docker compose for additional application will appear

Then we will be able easily launching all microservices in Dev env like it is almost fully fledged staging environment

It feels like cheating xD

[bitter wadi]

:x: There was an error adding the infraction: status 400.

[pallid spire]

guys is there anybody know selenium well

[marble wagon]

ask your question, not if there is someone to help

[pallid spire]

some functions of selenium are not available

[marble wagon]

which ones

[pallid spire]

for example findelements functions

there are only element and elements

others are not available

[marble wagon]

which function do you want? Btw this conversation should probably move to #web-development

[merry elbow]

hello

my pytest fixture breaks when ran twice

how do i make it so that pytest sessions share the same result*

[empty dust]

Does it break because your code is non-deterministic?

[merry elbow]

the fixture used httpx client wich makes an instance of my asgi app

wich cant be initialised twice*

i want the fixture to always return the same client and not initialise it

[empty dust]

What does your fixture look like?

[merry elbow]

@pytest.fixture
async def client(
    fastapi_app: FastAPI,
    anyio_backend: Any,
) -> AsyncGenerator[AsyncClient, None]:
    """
    Fixture that creates client for requesting server.

    :param fastapi_app: the application.
    :yield: client for the app.
    """
    async with AsyncClient(app=fastapi_app, base_url="http://test") as cn:
        yield cn

[jolly tangle]

Can someone help me at #help-donut? It involves unit testing. Specifically pytests. Thanks

[hollow basin]

I'm trying to understand why this works

class TestModel(unittest.TestCase):

    @classmethod
    def setUpClass(cls):
        cls.groundtruth_directory = tempfile.mkdtemp()

    def test_fit_predict_dump_load(self):
        model = Model(self.pipeline)
        model.fit(self.dataset, groundtruth_directory=self.groundtruth_directory)

but not this

def test_fit_predict_dump_load(pipeline, tmp_path):
    """Fits a model, tests that it predicts correctly, dumps and loads it, then tests that it still predicts"""

    model = Model(pipeline)
    groundtruth_directory = tmp_path / 'ground'
    model.fit(dataset, groundtruth_directory=groundtruth_directory)

Perhaps there's something I misunderstand about the semantics of tmp_path?

I end up with E FileNotFoundError: [Errno 2] No such file or directory: '/tmp/pytest-of-farnsworthsw/pytest-10/test_fit_predict_dump_load0/ground/PMC1314908.ann'

I thought it might be a permissions thing, but I get the same error even if I specify a --basetemp in my home directory somewhere

oh. I just needed to call Path.mkdir

[half stream]

I keep getting this error each time i run pytest command from my project directory.

pytest.pathlib.ImportPathMismatchError: ('numpy.conftest', '/usr/local/lib/python3.7/site-packages/numpy/conftest.py', PosixPath('/code/env/Lib/site-packages/numpy/conftest.py'))

but when i run my test per module, it runs fine e.g

pytest app_name

Setup:

• django project

[hexed cloak]

The trick is to use a create_app factory

[hexed cloak]

Instead of making your app a global make a factory instead

def create_app() -> FastAPI:
    app = FastAPI()
    app.include_router(endpoints.router)
    return app

Then you make a new one for each test

[hexed cloak]

What makes it so you can't initialize it twice?

You probably have a global async resource that you need to move into a lifespan function eg startup/shutdown

[gritty turtle]

I'd like to set up testing for this project using pytest. I was thinking I can just make src a package and install it to the venv so that my tests folder can find everything, is this a good idea?

[cobalt ore]

I'm a bit confused about fixtures in pytest - or at least - how "best" to use them - i'm wondering if I can namespace fixtures which are imported as pytest plugins

I can create structure such as :

tests/fixtures
|-- tests/fixtures/__init__.py
|-- tests/fixtures/thing.py
`-- tests/fixtures/thing_2.py

And in tests/conftest.py have:

import pytest
pytest_fixtures = [
    'tests.fixtures.thing', 
    'tests.fixtures.thing_2',
]

In thing, thing_2 and conftest.py I could have:

@pytest.fixture
def something():
    return __file__

when using this in some test

def test_a_function(something):
    ...

Because it's in conftest.py I will have __file__ as conftest

if I remove it from conftest then I get __file__ having thing.

so is there a way to namespace this so instead of

def test_a_function(something): ... 

I have

def test_a_function(thing.something): ...

Orrrr - is this a stupid question, it might well be, that would be useful to know as well

[maiden pawn]

Regulate conftest namespace by having multiple conftests at different folder levels.
Then more top folder, then more global.
Than lower to folder with code, then more local and overshadowing all higher located

Obviously conftests from neighbouring python module will be fully invisible to other neighbouring python modules at same folder level

[wise fractal]

Hey guys! I am writing some tests for an application that has a lot of interactions with various Google APIs, does somebody know if Google provides mock classes for testing purposes? I couldn't find any

[maiden pawn]

U can always write your own mocks
The purpose of clean architectured code is to abstract away implemention details between different code layers

Also u can always record data examples in tests which are launched only once/with certain pytest mark excluded by default from running

[wise fractal]

thanks! ok, i guess i'm writing my own mocks, thanks for the tip!

[plain citrus]

Hey! Could someone point me in the right direction? I have been using Python a lot lately (self taught, using it both for work and personal stuff). Sadly I do not know anyone who is good at Python and I'm always afraid of doing something wrong and not by industry standard!
That's why I want to try and learn unit testing. And hopefully automatic deployment to production too! But I struggle with one basic logic thing.

I have a script which works as windows service. The script basically reads data from database, does some logic and if it detects find specific thing in database it sends an email. So basically script consist of 3 functions. One database read, one email send and one database update. Access information to both exchange and database is read from config file. This is done at the beginning of script.

What library should I use to test something like this? I've tried following some tutorials but in the end it seems that when you write unit test you call some class and function. But all of my functions are depended on each other and cannot be run separately. So how should I go about testing something like that? And also if you just run the function using unit test it cannot even connect to database because that is done outside of function..

[maiden pawn]

This book gives theory to learn what it is, what types of testing exist, what to pursue in testing, read it first

[maiden pawn]

This book has a more practical approach and shows how to make it part of coding process and influencing code architecture of application, read it second

[maiden pawn]

i'll try to answer how to test your current situation later. feeling myself like my battery charge is really low. Not enough power for coherent thoughts.

[maiden pawn]

okay:
functions:
database read
email send
database update

What unites them? Being one Action. I would prefer seeing it as one action that invokes three tasks
each can be mocked for other tests if possible.
like
database read should be inputting database object for connection, which params to seek
and outputs dataclass/pydenticobjects u can use in other steps / and u can mock this test output by creating this dataclass/pydantic objects out of air.
email send is obviously asking for you to create email class, that you just insert as attribute into your action and parameters to invoke some action from it
database update is same.

I would prefer seeing all those layers being separated and isolated from each other... in terms of none of them knows implementation of others except for declared abstract classes which do not give away implementation details

I suppose you interact also with particular entity/table during your interactions. Lets suggest it is user...

from contextlib import contextmanager
from dataclasses import dataclass

########################### USER SCHEMAS ############################

class UserSchemaOut:
    id: int
    name: str
    description: str
    number: int

class PlayerSchemaIn:
    id: int
    email: str

########################### Database ############################

class Database:
  def __init__(self, database_params):
    init_db_connection_objects()
  
  @contextmanager
  def connection(self):
    yield self.connection_maker.create_connection()

########################### PlayerRepository ############################

class Player(ORM):
    id = PrimaryKey()
    name = CharField(max=30)
    description = Charfield(max=250)
    email = Charfield(max=250)

class PlayerRepository:
    def __init__(self, db):
        self.db = db

    def create_player(self, PlayerSchemIn):
        with Database.connection() as connection:
            statement = insert(PlayerSchemIn)
            connection.execute(statement)

    def read_players(self) -> list[UserSchemaOut]:

        with Database.connection() as connection:
            statement = select(Player).all()
            db_rows = connection.execute(statement)            

        return list([UserSchemaOut(row) for row in db_rows])

    def update_players(self, input_players: list[PlayerSchemaIn]) -> bool:
        
        with Database.connection() as connection:
            for player in input_players:
                statement = update(Player).values(**player) # preferably bulk update
                connection.execute(statement)       

            connection.commit() 

        return True

########################### EmailSender ############################

class EmailSender:
    def __init__(self):
        initialize_email_provider_params_for_connections

    def send_email(self, text, email):
        implementation details to send email

########################### MailMessage ############################

class MailMessage:
    def __init__(self, for_whom, insertable_values):
        do stuff you need to write your generate your email

    @property
    def text(self) -> str:
        return rendered email as text

########################### Tasks ############################

class TaskReadDatabase:
    player_repo_class = PlayerRepository
    def __new__(self, db, any_additional_inputs_to_do_task):
        self.db = db
        return self.run()

    def run(self) -> list[UserSchemaOut]:
        player_repo = player_repo_class(db)
        players = player_repo.read_players(any_additional_inputs_to_do_task)
        return players

class TaskSendMail:
    email_sender_factory = EmailSender
    mail_message_factory = MailMessage

    def __new__(self, any_additional_inputs_to_do_task):
        return self.run()

    def run(self):
        mail_message = MailMessage(any_additional_inputs_to_do_task)
        email_sender = EmailSender()

        email_sender.send(mail_message, any_additional_inputs_to_do_task.to_whom)

class TaskUpdateDatabase:
    player_repo_class = PlayerRepository
    def __new__(self, db, any_additional_inputs_to_do_task):
        self.db = db
        return self.run()

    def run(self) -> list[UserSchemaOut]:
        player_repo = player_repo_class(db)
        players = player_repo.update_players(any_additional_inputs_to_do_task)
        return players        


########################### Actions ############################

class ActionName:
    task_read = TaskReadDatabase
    task_send = TaskSendMail
    task_update = TaskUpdateDatabase

    def __new__(self, any_inputs_to_do_task):
        self.task_read(any_inputs_to_do_task)
        self.task_send(any_inputs_to_do_task)
        self.task_update(any_inputs_to_do_task)
        return any_appeared_results

and testing part

########################### Testing ############################

def mail_msg_is_generated():
    msg = generated MailMessage(with params)
    assert stuff is correct in msg.text

@pytest.mark.integration
def email__sender_works():
    email_sender = EmailSender()
    email_sender.send(bla)
    assert no errors happened

@pytest.fixture
def db():
    # self cleaning temporal database for testing
    database = create database leading to testing database
    database.intialize_table()
    yield db
    database.drop_tables()
    database.drop_database()

@pytest.fixture
def created_players(db):
    player_repo = PlayerRepository(db)
    for player in players:
        player_repo.create_player(player)

def players_are_read(db, created_players):
    player_repo = PlayerRepository(db)
    players = players.read_players(db)
    assert players[0] == PlayerSchemaOut(id=1, name="bla", desc="bla")

def players_are_updated(db, created_players):
    player_repo = PlayerRepository(db)
    players.update_players(db)
    
    assert players[0] == PlayerSchemaOut(id=1, name="bla_changed", desc="bla_changed")

@pytest.mark.integration
def test_integration_test_to_send_email():
    # since email sender invokes third party dependencies, lets declare it integration tests that runs more rare
    TaskSendMail()

def test_action(db, created_players):
    action = ActionName
    action.task_send = MagicMock() #mock if not wishing extra spam in regular tests

    action()

    assert what is needed

this mocked action, can be later used in other tests where it is needed being invoked as part of tests if necessary

i don't mock database, because we can have temporal databases in modern CI tools as part of tests

we just need to replace using standard database name in postgresql, with database created purely for test

[maiden pawn]

Basically the point for all of this abracadabra higher... than in order to make stuff testable, you need to think about code architecture
all parts of my code do not know implmenetation details of others except for declared dataclass/pydantic classes of objects. Everything else is abstracted away as much as possible
it is easy to test only parts of certain code gradually

and for that multiple books can be read: Head First Design Patterns, Clean Architecture Robert Martin, Code Complete by McConnel

• i like description of this porto architecture https://github.com/Mahmoudz/Porto (this is one out of many different approaches to architecture)

[maiden pawn]

book TDD suggests writing any sufficient code first, and refactorizing later

we can also plan in advance expected code architecture

and we do both things at the same time

some things in code architecture / depenendencies we plan in advance how it would be looking

some things we refactorize during our work

spotting stuff that's better be made a separate function / class / package

Martin Fowler Refactoring book would be nice to read also https://martinfowler.com/books/refactoring.html

Testing and Code Architecture are really strongly related to each other. Code Architecture can be made easy testable

https://raw.githubusercontent.com/darklab8/darklab_backend_roadmap/master/swe_backend.drawio.svg i keep track of those resources here what can be learned

[river magnet]

How do I mock a class and also mock that classes functions to return fixtures as return values?

[proper wind]

best way to unit test a package? if i run unittests or pytest directly it only works on 3.10 (otherwise it just fails on imports)

[abstract tendon]

are you using 3.10 specific syntax?

[proper wind]

no

because if i run on CI with flat hierarchy it works up to 3.6

[pearl cliff]

can you give a specific example? you can assign Mock objects to the attributes of another Mock object

mocking an entire class seems a little suspect anyway

[lusty horizon]

Hi, could somebody help me to understand tkinter, passing of arguments (current values) during event actions, please?

[pearl cliff]

@river magnet

with patch('foo.bar.MyThing.get_db_conn', return_value=test_db_conn):
    ...

something like that should be sufficient, no? just mocking the methods you need, not the whole class. hopefully you are not doing i/o in the class __init__ method... that's bad design

[lusty horizon]

I don't understand the manual here https://docs.python.org/3/library/tkinter.html?highlight=bind#bindings-and-events-1 and it does not show an example

[pearl cliff]

you should probably be asking in #user-interfaces , no? this channel is about automated software testing

[lusty horizon]

Yes, right, wrong channel. Sorry!

My brain is probably too mushy

[pearl cliff]

me too

[zinc verge]

Hi guys - I’m struggling with writing an end to end test for my system. I’m using a microservice-ish architecture with dedicated hardware for each service. I want to test a workflow that involves all services. The different steps in this workflow must be executed sequentially since they share some common resources. The run time of the entire workflow is about 60 mins.

I’m struggling with how to structure the test in a Arrange Act Assert fashion since each step of the workflow needs to complete before moving on to the next one.

Any suggestions or links to good resources on this topic?

[earnest halo]

hello all, have you seen Raymond Hettinger's excellent recent (7/24/202) talk on unit testing? I always learn something new from him: https://www.youtube.com/watch?v=jSIsyMd2-RY

[exotic marlin]

Hello @zinc verge with what i could understand and figureout, I see behave for python will work fine for you. You can refer
https://github.com/jitendar-singh/shipwright-tests.git

[river magnet]

Sorry just saw your response, thanks!
I was thinking of setting tests in the wrong way I think
Many different possible solutions so I think I was trying to stack mock classes and fixtures and parametrization together when I shouldn't have been

[pearl cliff]

indeed. i suggest minimizing the number of fixtures and mocks required for any given test

fixtures are one thing, patching is another thing, and mocks in particular should be considered a last resort

e.g. you can patch with a fixture instead of a mock, and that's somewhere "in the middle"

[river magnet]

The tests at my place tend to be written with inline mocks rather than mocking classes outside the test or using fixtures

Any good reads on best practices for testing that is technical and specifically python?

Similar to what you just suggested, what should be considered at first and then next for deeper requirements and so forth

[pearl cliff]

i usually do this too, specifically because mocks should be as local as possible and you should be very aware that a mock is in effect. non-local mocks reduce your ability to reason about them, as is the case with almost anything dynamically-scoped

[river magnet]

Also what would you prefer:
A few fixtures describing input/output dicts and then testing against those
Or writing said dicts inline within tests

[river magnet]

That's a great point

[pearl cliff]

i'd start with inline, expand to pytest parameterize after you have more than 1 or 2 cases, and then transition to a fixture if they get too big and/or they start to be re-used across several tests

consider also using property-based testing if possible instead of or in addition to static input/output pairs

[river magnet]

I'm a big fan of parametrize, I think it's very readable having an input/output table for tests

It's possible to pass fixtures through parametrize but that is messy

[river magnet]

Could you explain further?

[pearl cliff]

personally i think pytest's fixture syntax is godawful, matching parameter names to function names is way too magical and is overall bad design. but otherwise pytest is excellent

[pearl cliff]

they can explain better than i can https://hypothesis.works/

[river magnet]

I can see that
When starting out I liked the idea of splitting test models into their own fixtures
But it decreases readability so much unless you can adequately explain the test model in a function name, but even then...

[river magnet]

Thanks for your advise, I'll be sure to give that a read

[radiant mason]

I'm tasked to continue working on a script at work with over 30 files nested in multiple subdirectories without tests. How should I go about adding unit tests to this? Do I add file_test.py near each file or do I add them in tests like:
tests/path_to_file/test_file.py
or
tests/test_path_to_file.py

[maiden pawn]

both strategies are valid
I prefer first approach tests/path_to_file/test_file.py, because it allows me to have... acceptable illusion of good testing coverage without using automated testing coverage analyzer yet.
Well, at least i clearly see files with relevant tests close to necessary code

I think i just should add testing coverage analyzer 😆

i will still use first approach anyway due to ability quicker finding code

clearly specifying that code with tests can be only in tests.py/tests folder, test_abc.py, and conftest.py is enough for me

[pearl cliff]

the other advantage of keeping them in separate directories is that you can set up test-specific helper code later easily

[fair elm]

How could I write testcases for decorator functions?

[maiden pawn]

Create some test function to decorate? And test it

If functionality inside of decorator has point to be tested without decorator wrapper itself, test it in separation outside

[fair elm]

def login_required(f):
@wraps(f)
def decorated_function(*args, **kwargs):
if g.user is None:
return redirect(url_for('login', next=request.url))
return f(*args, **kwargs)
return decorated_function

@app.route("/about")
@login_required
def about():
''' Documentation about the PIN process'''
log_task()
title = "Place in Network Management Console"
return render_template('pin_doc.html', title = title,username=g.user,vars=var)

I need to write testcases for about function, could you please suggest how could I proceed.

[maiden pawn]

!code

[bitter wadi]

Here's how to format Python code on Discord:

```py
print('Hello world!')
```

These are backticks, not quotes. Check this out if you can't find the backtick key.

[maiden pawn]

is it flask code?

not important really

just use appropriate test client for your framework
https://flask.palletsprojects.com/en/2.2.x/testing/ her eit is described for flask for example

def test_request_example(client):
    response = client.get("/posts")
    assert b"<h2>Hello, World!</h2>" in response.data

write test that calls externally your /posts

write it so in one case it is called without authentification, and in another while authentificated

feel free to use even requests library if necessary

basically integration test for the win in this siutation

[fair elm]

doing so , testcases get PASSED status, but lines get missing. for example :

[maiden pawn]

then you wrote tests in a wrong way?

they were supposed to call entire view in authed version, and getting error exception when not being authed. Two tests.

[fair elm]

could you please forward me testcase written for above. I will take reference to that to cover rest of the decorated functions

[jolly shell]

anyone can help me with security checks with bandit , I am getting module not found error . Don't really understand it my tests seems to work fine

[neon flint]

i'm implementing an email verification system, and i'm transitioning from using a third-party service to test that emails are sent (mailtrap, since i've gone over the 500 email-per-month limit) to a mock object using pytest_mock. i'm not sure how i would link that up so that my flask app uses this mocked object instead of a flask_mail object, any tips on how to approach this?

like the current way i have it (i'm mocking common.plugins.mail with my custom object), when i do print(common.plugins.mail) it prints my mock object, but when i actually send a POST request using app.test_client (i.e. client.post()), i get an error with mailtrap saying i've reached my max email limit

[frosty perch]

Hello !
I dont know if i'm in the good channel for my question.
Vscode test extension dont find any test in my project but when I'm running pytest its working 🤔
Somone have an idea ?

[maiden pawn]

oh yeah. i often had a problem to set it up
it required right combination of .vscode/settings.json + it did not read env params usually properly, some sort of technic is used to inject them reliably
the most reliably it was with pytest fixture automatically started on session

or having in settings dotenv reading of env file

i got tired of those problems eventually

and found universal solution how to have it always working

just developing from within docker-compose containers 🙂

[frosty perch]

I tried many diffrents params in settings.json but its doesn't work..
i will try with docker containers thx 🙂

[maiden pawn]

oh yeah. one more thing. you are supposed to be having correct, not tangled python package organizations preferably

that is not requiring any additional Environment values to resolve it

main project folder has not __init__.py

then your main app could be having absolute paths/relative paths using

while having dependency on packages laying in this project folder in parallel

within app package and dependency packages, each folder has __init__.py

docker-compose allows to resolve even this problem for pytest, by having persistent env variables, but better be resolving it in original way without it

[frosty perch]

All my packages have __init__.py
I suppose that vscode test dont use my ./.venv/bin/python as interpreter and due to library errors dont show my tests

[maiden pawn]

don't put __init__.py to root folder where your packages are grouped. that's important moment xD

[frosty perch]

yes sure

[maiden pawn]

https://github.com/darklab8/darklab_darkbot
i have this project actually with it all setup

going to have slight refactorization to make make.py file though, a moment, i'll do it right now

[snow brook]

how can I make a setup where my tests are in a subfolder, and I am able to import from the root folder:
I would like to have a sub-folder with my my tests, and have my main.py in the root but it will not import ?

[viscid forge]

What command are you using to run your tests

[snow brook]

Python -m pytest

have moved the tests to be on the same level as that main programme... was told that it is not easy to import up the folder tree

and that issue is solved in this way.
now my issue is that i want to mock the json response i get for the request endpoint with a json file but that is not "subscrible" ...

platform linux -- Python 3.8.10, pytest-7.1.2, pluggy-1.0.0
rootdir: /home/hlynge/dev/property
collected 0 items / 1 error                                                                                                                                                                                                        

============================================================================================================== ERRORS ==============================================================================================================
___________________________________________________________________________________________________ ERROR collecting test_db.py ____________________________________________________________________________________________________
test_db.py:14: in <module>
    web_result = propertyparser(test_file, name)
main.py:74: in propertyparser
    for json in results["results"]:
E   TypeError: '_io.TextIOWrapper' object is not subscriptable

[viscid forge]

Yeah it isn't, but aren't you running from the root in that picture anyway

What does your mock look like

[snow brook]

yes and then the tests were in a subfolder and I could not import from the root

[viscid forge]

The tests being in a subfolder should be fine (I just spun up a project locally to confirm)

The important bit is where the imported module is relative to where you're running

[bitter wadi]

Hey @snow brook!

You either uploaded a .txt file or entered a message that was too long. Please use our paste bin instead.

[snow brook]

@viscid forge - this is my mock for the json response:
https://paste.pythondiscord.com/avinuhayav

[viscid forge]

Could you show the code which does the actual mocking

[snow brook]

test_data = [
    ("MILAN", "lom", "MI"),
]

for name, region, province in test_data:
    print("We are in the test function.")
    print(name)
    test_file = open("json_mock_file.json")
    web_result = propertyparser(test_file, name)
    id_list = []

I am only starting on my learning journey of testing, so please be gentle

[viscid forge]

Haha don't worry

What does propertyparser look like

[bitter wadi]

Hey @snow brook!

You either uploaded a .txt file or entered a message that was too long. Please use our paste bin instead.

[snow brook]

https://paste.pythondiscord.com/roxoditomu'

[viscid forge]

I see, your function is expecting a dictionary, but your test is giving it a file

Did you perhaps mean to read the data from the file then pass it in?

[snow brook]

yes

[viscid forge]

test_data = [
    ("MILAN", "lom", "MI"),
]

for name, region, province in test_data:
    print("We are in the test function.")
    print(name)
    with open("json_mock_file.json") as test_file:
        web_result = propertyparser(json.load(test_file), name)
        id_list = []

This should do what you want I think

The with open thing is best practice, but the important part here is the addition of json.load

[snow brook]

got it ... thanks a lot

[viscid forge]

As an aside, look into typing and type hints

[snow brook]

I am trying to move into tying and type hints but there is a lot to learn

def deserialise_property(item, region) -> Property:

this is one of my first type declarations

[viscid forge]

It can be a little overwhelming at first, but it helps a lot with situations like these :D

That's a very good start for sure

[snow brook]

thanks a lot for ou willingness to help.... really appreciate it

This is my first attempt on making as appartment search for property in italy with learning of requests module, and lookup on distance to cost in a geojson file, and the count of pubs, supermarkets, restaurants and bakeries within a distance of 2,5 km....

and now also trying to include some unit tests

[viscid forge]

That's a lot to learn in one go haha, good luck in your journey

[snow brook]

I do have the distances and the counts....now it is tesing time

@viscid forge - How can I import the item structure from main .... ?

[viscid forge]

What's the item structure?

[snow brook]

    item = {
        "region": region,
        "id": stringid,
        "is_new": is_new,
        "price": price,
        "price_drop": price_drop,
        "bathrooms": bathrooms,
        "caption": caption,
        "category": category,
        "discription": discription,
        "discription_dk": discription_dk,
        "floor": floor,
        "rooms": rooms,
        "surface": surface,
        "longitude": lon,
        "latitude": lat,
        "marker": marker,
        "photo_list": photo_list,
    }
    return Property(**item)

[viscid forge]

Isn't this already in main

The dictionary itself is not exposed outside the function, so it depends on what you can access from Property

[snow brook]

How can i use in in test like:

for name, region, province in test_data:
    print("We are in the test function.")
    print(name)
    with open("json_mock_file.json") as test_file:
        web_result = propertyparser(json.load(test_file), name)
        id_list = []
        with Session(db_engine) as session:
            for item in web_result:
                if item.latitude != None and item.longitude != None:
                    if item.dist_coast == None:
                        calc_dist_cost(item)
                    if item.pub_count == None:
```'

will i have to re-do the structuer in the test.py ?

[viscid forge]

This is getting more into program design, but yes you'd need to manually define it here as well if you want to create the Property yourself in the test function

I think that's what you're doing? I'm not entirely sure

[snow brook]

ths

[orchid arch]

module not found.. shiver my last two work days trying to write unit tests in python (first time, never developed in it before). Any tips? I mock AWS resources as necessary but pytest can't find the modules. It appears to be something to do with the directory i execute my pytest command (or in my launch.json or whatever in vscode) and i can only get it to work when i change my imports to import full.directory.path.from.root.here.even.though.this.is.unnecessary.needed_class

'the modules' as in the python files of my various classes that are imported from different folders. the code works by itself, i just can't get pytest to work. I suppose I could mock calls to other classes (maybe I should be anyways LOL) but just starting out so wanted to get some successful tests under my belt.

[last valley]

@orchid arch Are you calling "python -m pytest your-tests-folder/" from the project root ?
Also you should always be using absolute imports, things usually go FUBAR if you don't.

[maiden pawn]

:b relative imports work fine to me
Within same folder/package they are preferable in my opinion

[last valley]

Maybe it's just preference after all. It's probably a PATH issue here though

[maiden pawn]

https://github.com/darklab8/darklab_darkbot
6 codes of packages within same folder.
Usually relative imports within same package
Absolute imports when from one package to other ones

No PATH hacks

Everything works in predictable way, partially thanks to Docker compose commands to run everything, while invoked from python makefile

[proper wind]

Yho can anyone suggest some resources for unit testing I'm struggling somewhat with it or implementation thereof

[proper wind]

Thnx a mil

[frosty perch]

Hello, i'm trying trying boto
I have a lambda which publishes in an sns topic and I would like to test the message received by the sns.
But I don't really know how to go about it and I can't find anything that seems to do
If someone has an idea

[night dome]

heheh

[orchid arch]

pretty sure not using absolute imports was the problem.. got it fixed after putting my "java" hat on (lol just good coding practices).

[orchid arch]

Also, any opinions here on mutation testing? Mutmut + cosmic-ray are two I've found but curious if there is a more mainstream one than those

[royal orbit]

Either of those are good; I haven't used them much but my impression is that they're broadly similar with cosmic-ray providing more hooks if you want to customise the system.

Unfortunately mutation testing hasn't really taken off yet, perhaps because nobody has yet copied Google's system that makes it practical.

https://research.google/pubs/pub46584/

Avoiding arid nodes and respecting developer attention are huge advances!

(cosmic-ray was interested in add support for this, if anyone is looking for a project...)

[pearl cliff]

isn't the developer of mutmut a user here?

[lone dust]

I apologize if this is not the right channel - but I am working on a pixel comparison tool to automate finding differences between our preproduction server and our production server. Right now I have it set up with Python and selenium as well as the pixel match library. My issue currently is finding a good way to automate the changing of the windows hostfile so I can point to two different webservers. Is there a better way to do this or does anyone know of a good way to change that hostfile?

[hasty maple]

Hi!, I'm learning how to test, and I have a question: Is there any way for mock.patch to patch in every namespace?

[pearl cliff]

i don't think so, but i also think maybe you don't want to do this.

[hasty maple]

Why? I want to mock an external service everywhere in the test suite.

[ember maple]

It's highly error prone, in particular in the general case it's a unhinged footgun

[steady jay]

I don't believe modifying the hostile is the best approach. Aside from being a system protected file I do believe to do so that requires the Python process to run elevated which is also a bad practice. Have you tried approaching this with pytest monkeypatching?

[steady jay]

Alternatively, if the pytest monkeypatch approach doesn't seem to work for you perhaps you can make us of Docker to containerize your independent environment tests which can also allow you to run your them independently of each-other which may require a third process to do the diff of the two results afterwards.

[marble hatch]

I have combinations of classes and sub-classes I want to test with pytest.

I want to do something like this for my combination tests:

def test_name(a_type, b_type, operator_function):

Is there a clean way to do all of this with fixtures? I already have the operator fixture figured out, and suppose I could use indirect fixtures with parametrize. However, that will require parametrizing every test which takes a combination, which seems ugly.

[marble hatch]

For additional clarification, my current implementation has a problem

I use functools.product to generate all pairs and store them in a Tuple[Tuple[Type, Type], ...], then use those the inner pairs to fill in the downstream fixtures

The problem is that the output looks like this:

test_classic_operators_return_correct_values[truediv-pairing0] PASSED
test_classic_operators_return_correct_values[add-pairing1] PASSED

I don't want to show the name generated for the tuple element, I want to show the exact pair being tested, like this:

test_classic_operators_return_correct_values[truediv-Type-Type2] PASSED

[lucid spoke]

Hello

has someone here already did unit tests for a tkinter gui ?

[maiden pawn]

Purely technically it is possible
https://pyautogui.readthedocs.io/en/latest/
Crossing with pytest, and something could be made

[maiden pawn]

But it is not unit testing at all. It is end to end testing

For unit testing, just isolate your code logic out of tkinter

[strange pelican]

!e
a = 8837
b = 8837
if a == b:
print("access")
else:
print("tf")

[bitter wadi]

@strange pelican :white_check_mark: Your 3.11 eval job has completed with return code 0.

access

[rose oyster]

😄 good one!

[modern hedge]

If tkinter followed a pattern it would have been way simpler

[barren schooner]

!e
print(‘hi’)

!e
print("hi")

[bitter wadi]

@barren schooner :white_check_mark: Your 3.11 eval job has completed with return code 0.

hi

[barren schooner]

!e
while True:
print("hi")

[bitter wadi]

@barren schooner :x: Your 3.11 eval job has completed with return code 143 (SIGTERM).

001 | hi
002 | hi
003 | hi
004 | hi
005 | hi
006 | hi
007 | hi
008 | hi
009 | hi
010 | hi
011 | hi
... (truncated - too many lines)

Full output: too long to upload

[astral badger]

Hello can someone please walk me through How I would do a mock unittest please

[pearl cliff]

this should hopefully help:
https://www.youtube.com/watch?v=ww1UsGZV8fQ
https://www.youtube.com/watch?v=Ldlz4V-UCFw

i also suggest reading the docs for unittest.mock and working through the various examples

[astral badger]

Thank you so much !!!

[barren schooner]

!e

import requests

url = 'https://www.mexc.com//open/api/v2/market/depth?depth=10&symbol=ETH_USDT'
response = requests.get(url)
print(response.json())

[bitter wadi]

@barren schooner :x: Your 3.11 eval job has completed with return code 1.

001 | Traceback (most recent call last):
002 |   File "<string>", line 1, in <module>
003 | ModuleNotFoundError: No module named 'requests'

[sterile marsh]

anyone got any suggestions for python specific test driven development tutorials that doesn't just do basic, somewhat useless examples? \

Also looking for tutorials related to utilizing good design patterns in large code bases

[pearl cliff]

can't speak specifically for TDD but i think reading through the test suites of popular libraries and applications is a good practice

[pliant swallow]

Hey, i have two containers with pytest. When I run docker-compose exec [container_1] python -m pytest, it runs the tests of container_2

Anyone knows how to solve this?

[maiden pawn]

at least available options

1. not putting data of container 1 to container 2
2. pytest test_folder to limit which tests to run
3. having pytest.ini with different testpaths settings in each of them

[pliant swallow]

Love u 😁

[primal sable]

For testing some database functions that pull data, for example, is it better to just mock that database connection and provide some dummy data or is it also ok to use a dev database? I'm currently working with the latter on some project I'm contributing to to set this up with GitHub Actions and I'm not entirely sure on how to approach it. Both the mock/dev database and setting that up with GHA

[maiden pawn]

In general it is better not mocking for better realistic tests. Especially if u checking exactly that code that does exactly this thing only

But u can consider mocking for code that uses this database result as setup data. U wish mocking in order to speed up test runs.
In order to have it is done without pains, database dealings should be isolated and only SQL free dataclasses and pydantic models are allowed to move to other code parts of app

[proper wind]

# https://packaging.python.org/en/latest/tutorials/packaging-projects/#creating-the-package-files
packaging_tutorial/
├── LICENSE
├── pyproject.toml
├── README.md
├── src/
│   └── example_package_YOUR_USERNAME_HERE/
│       ├── __init__.py
│       └── example.py
└── tests/

is there a reason it recommends to keep a tests folder in the base directory? like is some packaging tool able to find tests easier if you put them in that location

[gentle sand]

It's just the general standard, if tests are on the root directory then you know there are tests for the project when you open it.

Often times reading tests give you more understanding of the project than reading the whole codebase

[earnest sequoia]

Anyone here familiar with using Hypothesis for testing? I am writing some tests to test my tests. More specifically: I have this helper class to avoid code repetition in creating the strategies for a large number of test cases (it creates a strategy that generates the data for a bunch of records to pre-insert into a database)

from more_itertools import partition
from hypothesis import strategies as st

_P, _TAttr = ParamSpec('_P'), TypeVar('_TAttr', covariant=True)
AttrStrategyFactory: TypeAlias = Callable[_P, st.SearchStrategy[_TAttr]]

class AttrsStrategyBuilder:
    def __init__(self, model_cls: type) -> None:
        self.model_cls = model_cls

        self._registry: dict[str, AttrStrategyFactory[..., Any]] = {}
        self._unique_attrs: set[str] = set()

    def register(self, *attrs: str, unique: bool = False):
        def wrapper(st_factory: AttrStrategyFactory[_P, _TAttr]) -> AttrStrategyFactory[_P, _TAttr]:
            model_cls, registry = self.model_cls, self._registry
            for attr in attrs:
                if not hasattr(model_cls, attr):
                    raise ValueError(f'Model {model_cls} does not have attribute {attr}')
                if attr in registry:
                    raise ValueError(f'Duplicate registration of attribute: {attr}')

                registry[attr] = st_factory
                if unique:
                    self._unique_attrs.add(attr)

            return st_factory
        return wrapper

    def one(self, *, exclude_attrs: list[str] | None = None) -> st.SearchStrategy[dict[str, Any]]:
        if exclude_attrs is None:
            exclude_attrs = []

        sts_by_attr = {
            attr: st_factory()
            for attr, st_factory in self._registry.items()
            if attr not in exclude_attrs
        }
        required_attrs = self._required_attrs

        optional_sts_by_attr, required_sts_by_attr = partition(
            lambda attr: attr[0] in required_attrs,
            sts_by_attr.items(),
        )

        return st.fixed_dictionaries(dict(required_sts_by_attr), optional=dict(optional_sts_by_attr))

    def _make_unique_by_func(
        self,
        exclude_attrs: list[str] | None = None,
    ) -> Callable[[dict[str, Any]], tuple[Any, ...]] | None:
        if exclude_attrs is None:
            exclude_attrs = []

        unique_by_attrs = self._unique_attrs - set(exclude_attrs)
        if not unique_by_attrs:
            return None

        return lambda x: tuple(x.get(attr) for attr in unique_by_attrs)

    def multi(
        self,
        *,
        min_size: int = 0,
        max_size: int | None = None,
        exclude_attrs: list[str] | None = None,
    ) -> st.SearchStrategy[list[dict[str, Any]]]:
        return st.lists(
            self.one(exclude_attrs=exclude_attrs),
            min_size=min_size,
            max_size=max_size,
            unique_by=self._make_unique_by_func(exclude_attrs=exclude_attrs),
        )

How would I test the multi() function to ensure that the uniqueness constraint is not degenerative (as in, the unique_by function does not necessarily evaluate to be the same for each element of the list; a failing example would be unique_by=lambda x: (1,) since the output is (1,) regardless of what x is)?

My current solution is to create an inner function decorated with @given(example=builder.multi(min_size=2)) and call it inside the test case so that Hypothesis would throw a HealthCheck error if the uniqueness constraint becomes degenerative (since there is no way it can generate more than one unique element), but it feels a bit off to depend on internals like that.

Is anyone able to come up with a better way to test this?

[calm juniper]

How can I test this package, basically I need to test the decorator I have written the unit test but it seems bogus to me https://github.com/Agent-Hellboy/log_call/blob/main/tests/test_log_call.py

[hoary ginkgo]

!e

data = {"answer":"0xaxcgx0xaxbix0xkxb8x0xkxaux0xkxakx0xkxaax0xkxa0x0xkx9mx0xkx9cx0xkx92x0xkx8ox0xkx8ex_?_gAAAAABjGXy8tbsEovVLlFvesX2xiTR2LqdSyMKXUKemuEmSMvFIKfrmrFK-QYbwQE4kQivWM1dsFF-uF1e2sXKXNXlwrtCTi0lYKcA71ekoceHZPVd2BYdffscW-mJbZo8tgZUqslxT","challenge":"12eaa4ad-ade5-4933-9b97-a0509b3e1391"}
answer = (data['answer'])
challenge = (data['challenge'])
print(answer)
print(challenge)

[bitter wadi]

@hoary ginkgo :white_check_mark: Your 3.11 eval job has completed with return code 0.

001 | 0xaxcgx0xaxbix0xkxb8x0xkxaux0xkxakx0xkxaax0xkxa0x0xkx9mx0xkx9cx0xkx92x0xkx8ox0xkx8ex_?_gAAAAABjGXy8tbsEovVLlFvesX2xiTR2LqdSyMKXUKemuEmSMvFIKfrmrFK-QYbwQE4kQivWM1dsFF-uF1e2sXKXNXlwrtCTi0lYKcA71ekoceHZPVd2BYdffscW-mJbZo8tgZUqslxT
002 | 12eaa4ad-ade5-4933-9b97-a0509b3e1391

[gusty pawn]

I want to try my hand at unit testing I have a small program, not sure how to go about testing it. Any suggestions on were to start?

[haughty pine]

.rp 3 unit testing

[little panther]

Search results - Real Python

Here are the top 3 results:

Getting Started With Testing in Python

https://realpython.com/python-testing/

Django 1.6 Test Driven Development

https://realpython.com/django-1-6-test-driven-development/

4 Techniques for Testing Python Command-Line (CLI) Apps

https://realpython.com/python-cli-testing/

Click the links to go to the articles.

[haughty pine]

perhaps check the first one here

[gusty pawn]

I get the sense that my project needs a certain structure and organisation, from reading that first link, before this makes sense to do

[haughty pine]

ideally, yeah. but you can make it work with just a file too

[gusty pawn]

Given I already just have one file with like 8000 lines, what could I read to learn better application structure?

[haughty pine]

.rp 1 application layout

[little panther]

Search results - Real Python

Here is the result:

Python Application Layouts: A Reference

https://realpython.com/python-application-layouts/

Click the links to go to the articles.

[haughty pine]

here, for instance

RealPython is really handy 😄

[gusty pawn]

Clearly, thank you

[jolly tangle]

Okay for anyone listening

As you can see the code never runs in both tries. Any suggestion when you get a chance?

I am trying to create the database run some code which has some assertion errors then delete the database.

The problem is I need the assertion error to show up and and I can't always create and delete the database if the assertion errors shows up. This leads to columns in the database being added twice which causes an error.

Any advice I will post the code in a few moments

[river pilot]

put the code in a pastebin

[jolly tangle]

Most of the code is in a pastebin thanks for the help. I am using flask-sqlalchemy and pytesting

Maybe there is something wrong with create_token or verifiy_token.

https://pastebin.com/kUvqD8yp

I can prove your code is not running by the code below.

def test_try():
    try:
        a = 1
        b = 2 
        assert a == b
    except:
        print("do nothing")

def test_try_prints():
    with pytest.raises(Exception): 
        test_try()
        # assert add_and_delete_database_try == 
        assert 'random text' in str( pytest.raises(Exception.value) )```

Also I can add more code for context

just let me know

[river pilot]

who wrote "I can prove your code is not running"?

[jolly tangle]

Me

I just meant I ran the code

and it didn't work

I didn't mean it as an arrogant thing

I could have easily made a mistake

Kurglord was helping me

a little

I can find his code he suggested if you want me to

I used the try and he suggested similar code to the try

 test_try_prints```

[vapid dome]

Well I'm still not perfectly clear on what your goal is, honestly, but the purpose of using the pytest.raises is to make sure that an error does occur, so that you can evaluate it somehow.

[jolly tangle]

def test_some_code_in_a_flask_route
# create database
# assertion error
# delete database

The problem is the assertion error stops the code so database is not deleted. Does it make sense now.

Or am i mistaken about assertion errors stopping the code?

[vapid dome]

Assertions mostly just have two purposes, one is to try to ensure that certain conditions don't occur, the other is with Pytest to write unit tests

[jolly tangle]

so what you are saying is I should use the try get rid of the assertion error and just use print

[vapid dome]

But honestly, this doesn't really sound like something you should do manually, I'd use a context manager to ensure the database is deleted

So sorry to mislead you, I didn't have enough context, but the pytest.raises thing was a total red herring

[jolly tangle]

np

By context manager do you mean this https://flask.palletsprojects.com/en/2.0.x/appcontext/

because I have that in the code

[vapid dome]

But yeah, a context manager would let you create a database, do tests inside the context manager block, then when the test exits, no matter how, the database is deleted

A contact manager is basically any with block

[jolly tangle]

Let me show you the relevant code I think I did do that If not then show me how?

brb

Thanks

[vapid dome]

Also, just a point, if you're using the same kind of database for a bunch of tests, you can create a pytest fixture to make sure they're all the same and to keep things DRY (don't repeat yourself)

[jolly tangle]

ok Here is the code. Any questions just ask

https://pastebin.com/mbQDcS2p

Also create_token and verify token may be wrong when calling the functions/method . I was going to review that

models.py and config.py and conftest.py and test_routes.py are the 4 files

Also my other goal was to test create_token and verify token by using assertions

Hence why I need to create and delete the database

[jolly tangle]

I think i may have found a solution https://testdriven.io/blog/flask-pytest/

[pearl cliff]

you dont have to delete the whole database, just truncate its tables

[modern hedge]

Why?

[jolly tangle]

@modern hedge maybe show more code

[modern hedge]

genuinely asking, how will more code help

it appears like a datetime.timedelta problem, doesn't it

[jolly tangle]

I could be wrong but is the code in the pic it looks like a class , am I correct? Or are you importing datetime from a package?

[modern hedge]

I am importing datetime correctly https://github.com/demberto/PyFLP/blob/unittests/tests/test_project.py

[jolly tangle]

let me google dateline

I can't promise I can help

@modern hedge

do you think something like this would work?

https://docs.python.org/3/library/datetime.html

>>> from datetime import timedelta
>>> delta = timedelta(
...     days=50,
...     seconds=27,
...     microseconds=10,
...     milliseconds=29000,
...     minutes=5,
...     hours=8,
...     weeks=2
... )
>>> # Only days, seconds, and microseconds remain
>>> delta
datetime.timedelta(days=64, seconds=29156, microseconds=10)

is this the correct library ?

[modern hedge]

yes

@jolly tangle

[jolly tangle]

Is this for school?

[modern hedge]

nah lol

[jolly tangle]

good don't want you to cheat

[modern hedge]

who tf would write an entire library with unit testing for a school thing?

one hell of a school that would be

[jolly tangle]

I never been to coding school so I won't know

[modern hedge]

anyways

how to make a platform spcific tox command

especially for sphinx

on windows it as a batch script, linux it has a makefile

[jolly tangle]

I have no idea what those tools are better off asking someone else

but hopefully the link helped

[modern hedge]

well in my case only the seconds remain

probably because i construct it from a 64bit double

unix timestamp

[jolly tangle]

Again I know python flask and a little bit about pytests best of luck my advice is post in python general then move the discussion here

[deep dove]

It looks like your project.time_spent has a microseconds field that isn't present in the timedelta you've built

[vapid dome]

Why are you using time.delta()? Seems to me like you probably want datetime.fromtimestamp()

[modern hedge]

how do i make it ignore the microseconds?

[deep dove]

Ok, what exactly are you doing?

[modern hedge]

wdym I am comparing the times

obtained from a unix timestamp

[deep dove]

What are you comparing it for is what I mean

[vapid dome]

But why are you doing it like that? A time delta means it's the amount of time between two times

[modern hedge]

for coverage ig

i need to know whether the parser works correctly

[vapid dome]

You should be constructing two datetime objects them comparing them

[modern hedge]

why would i need a datetime object when i am comparing duration

[deep dove]

time_spent is best represented by a duration from start to now

rather than a datetime of the end

but uh, is it alright if you use >=? Why do you need == specifically

[modern hedge]

yea ig i could use >=

but again equality means i parsed it correctly from the binary form

= can mean anything greater

not even close

is there like a math.isclose for datetime?

[vapid dome]

This is so unclear to me

What are you starting with? You said Unix time, right?

[modern hedge]

Yes @vapid dome

[modern hedge]

how do i upload coverage data from tox env

[lapis sinew]

is there an automated tool to check that nothing in the public interface of a python package has changed? i want to have it as a pre-commit hook

for example, if someone changes the name of a function parameter, it will break anything that passes a keyword argument to it

[pearl cliff]

I don't believe this exists, although it would be useful. You would likely have to build your own based on tree-sitter or difftastic or the ast module

[lapis sinew]

bummer, but thank you 👍

[spice herald]

hi, I'm struggling to write unit test which will check if method list_blogs() was called. Can I ask for any hints what I'm doing wrong? Thanks. https://paste.pythondiscord.com/jozabinaya

[pearl cliff]

!paste please share your code as text, so others can read it easily. see below for instructions on sharing large sections of code 👇

[bitter wadi]

Pasting large amounts of code

If your code is too long to fit in a codeblock in Discord, you can paste your code here:
https://paste.pythondiscord.com/

After pasting your code, save it by clicking the floppy disk icon in the top right, or by typing ctrl + S. After doing that, the URL should change. Copy the URL and post it here so others can see it.

[modern hedge]

Anybody here?

conftest.py

@pytest.fixture(scope="session")
def project():
    return pyflp.parse(pathlib.Path(__file__).parent / "assets" / "FL 20.8.3.flp")

test_arrangements.py

def test_arrangements(project: Project):
    assert len(project.arrangements) == 2
    assert project.arrangements.current == project.arrangements[1]
    assert project.arrangements.loop_pos is None
    assert project.arrangements.time_signature.num == 2
    assert project.arrangements.time_signature.beat == 8

I need a way to just pass project.arrangements to test_arrangements()

Do I just make another fixture, or is there a better way

[viscid basalt]

This is fine

[modern hedge]

Semantic versioning is used for this purpose by the library https://semver.org

[viscid basalt]

Making a fixture to extract a property seems like an overkill

[modern hedge]

Its small rn, but soon it will be a lot of project.arrangements.... soon

[viscid basalt]

You can just use a temp variable if you want your code to be a bit shorter

[viscid basalt]

If so just make it

[modern hedge]

the fixture?

[royal flax]

hi i have some problem

can you help me ?

i have error ModuleNotFoundError: No module named 'Pages '

but Pages modul is exist

[modern hedge]

configure pytest search paths

[tool.pytest.ini_options]
testpaths = "tests"

Add this in a pyproject.toml in project root. Also use lower case names for folders

Follow PEP 8

!pep 8

[bitter wadi]

**PEP 8 - Style Guide for Python Code**

Link

Status

Active

Created

05-Jul-2001

Type

Process

[viscid basalt]

They're not using pytest 😉

[modern hedge]

ah shit

yea

unittest sucks

[viscid basalt]

Also what's violating pep8 here?

[modern hedge]

Folder/module names

[viscid basalt]

The problem is that you're running your tests from Tests directory, python won't try going up the file tree to search for your modules

Try running python -m unittest tests from root directory

Also follow pep8 as @modern hedge suggested

[royal flax]

oh thanks

its woking

[royal flax]

for sure

❤️

[viscid basalt]

You can also try pytest 😉

[royal flax]

ok i will try it

but now im just learning some automation basics

[modern hedge]

@viscid basalt you there?

Is there a way to get index of the arguments passed by pytest parameterize

[viscid basalt]

I use named parameters

@pytest.mark.parametrize(
    "name1, name2",
    (
        ("duplicate name", "duplicate name"),
        ("duplicate name", "duplicate-name"),
        ("duplicate   name", "duplicate   name"),
        ("duplicate   name", "duplicate name"),
        ("duplicate-name", "duplicate---name"),
    ),
)
async def test_create_duplicate(
    name1: str,
    name2: str,
):

[modern hedge]

Nah that's not what I wanted

Anyways is there a way to pass parameters like Python's kwargs?

So it can help remove unnecessary argvalues

[pearl cliff]

parametrize maps values to parameter names. what more do you want?

if you have a specific use case, describe it

So it can help remove unnecessary argvalues
this isn't clear to me

[proper wind]

Hello guys, I'm learning unit test using pytest. I was wondering if it was usefull to test a function which is opening a json file :

`def load_json(file_name:str):
"""
Read local json file
:param file_name: json object store locally
:return: dictionary with json content
"""
with open(file_name, encoding="utf-8") as f:
gj_dict = json.load(f)

return gj_dict`

[pearl cliff]

the most important thing you can learn about testing is: what exactly am i interested in testing? what do i want to prove to myself?

usually it's some variation of "it works and doesn't break", but what does "it works" really mean and what does "it doesn't break" really mean? if you can't answer those questions, you won't be able to write useful tests

[proper wind]

@pearl cliff Hmm, I'm not sure to follow you. In this simple case, I want to be sure that a dic will be returned with the json content. It will break if the file input is not in json format

I haven't been exposed so far to unit test, so it's quite difficult to put in pratice, there is only very basic example online

[lapis sinew]

your function is very simple, to the point that what you'll be testing here is that you've passed a json file to the test

i think you realize this, since your actual question was "is it useful". my opinion is no, in this case

[proper wind]

I see, thank you 🙂

[pearl cliff]

you've found an invariant: this function should never raise an exception as long as the file contents are json.

maybe this isn't a very useful invariant to test, but you could now write a test that constructs an arbitrary json file and runs the function, failing if an exception is raised. in this case i wouldn't do it still, because you're just testing json.load at that point. but hopefully this is a good example of what an "invariant" might look like.

[merry elbow]

.

can anyone link me with a big projectthat uses pytest correctly

[maiden pawn]

Not pretending I use correctly all the time, as I was lazy to write some necessary tests
https://github.com/darklab8/darklab_darkbot

But at least it is relatively becoming big pet project which has CI established in gitlab CI for automated testing verification of commits

I ll make some more thorough cleaning when reaching MVP, which I almost did at last

[modern hedge]

Should i place fixtures used only in a particular test module inside the module or conftest.py always?

[maiden pawn]

U can put fixtures at any parent folder level that is needed you know

It depends on a scope of needed fixture reusage of course

[modern hedge]

I don't have folders for the tests

just separate modules

[steep sinew]

Since you're learning pytest, it would be helpful to create a few situational tests to ensure your function operates properly under uncertain/unknown/unknowable input conditions.

Let's say I hadn't seen your specific implementation of load_json(...), or that the function hadn't been created yet.

I'd ask if I want the function to check to make sure the file exists and exits expectedly/gracefully if it doesn't.

I would probably create a test to make sure load_json(...) gracefully handles non-json files. (right now, your function assume the input file is going to be able to be passed into json.load(f) without error; that could come back to bite you.

A part of the answer to your question of if you should test your function comes down the question of how central load_json(...) is in your system.

If it's not very central, and you're not worried about bad input coming into the function ever, then maybe creating tests isn't worth your time.

[indigo widget]

Hello, first time unit testing (pytest). Is there a better option for testing multiple class methods with different parameters? Currently testing 3 methods per class, but want to test around 15. Would I put all of them in the tuple?

[graceful berry]

hello, i am using poetry as the package manager. I wrote around 50 unit tests using mocked dynamodb, and they work locally. However, when i try to run them on ci/cd on github actions, the tests fail. Any idea how to fix this?

this is my ci.yaml file

name: CI

on: [push]

jobs:

  test:
    runs-on: ubuntu-latest

    steps:
      - uses: actions/checkout@v1
        with:
          fetch-depth: 1

      - name: Set up Python 3.9
        uses: actions/setup-python@v1
        with:
          python-version: 3.9

      - uses: Gr1N/setup-poetry@v7
        with:
          poetry-version: 1.1.14

      - name: "🔨 Setup Poetry Python environment"
        id: pyenv
        uses: Steffo99/actions-poetry-deps@0.2.3

      - name: "🧪 Run tests"
        run: |
          source ${{ steps.pyenv.outputs.pyenv }}/activate
          pytest tests

And this is the error from the github job

==================================== ERRORS ====================================
45
____________________ ERROR at setup of test_create_company _____________________
46
    @pytest.fixture
47
    def dynamodb_table():
48
        with mock_dynamodb2():
49
>           dynamodb = boto3.resource("dynamodb")
50
tests/conftest.py:18: 

...
...
partition_name = partition["partition"]
91
        if (
92
            use_dualstack_endpoint
93
            and partition_name in self._UNSUPPORTED_DUALSTACK_PARTITIONS
94
        ):
95
            error_msg = (
96
                "Dualstack endpoints are currently not supported"
97
                " for %s partition" % partition_name
98
            )
99
            raise EndpointVariantError(tags=['dualstack'], error_msg=error_msg)
100
    
101
        # Get the service from the partition, or an empty template.
102
        service_data = partition['services'].get(
103
            service_name, DEFAULT_SERVICE_DATA
104
        )
105
        # Use the partition endpoint if no region is supplied.
106
        if region_name is None:
107
            if 'partitionEndpoint' in service_data:
108
                region_name = service_data['partitionEndpoint']
109
            else:
110
>               raise NoRegionError()
111
E               botocore.exceptions.NoRegionError: You must specify a region.
112
../../../.cache/pypoetry/virtualenvs/salaris-2X4k9Fkj-py3.9/lib/python3.9/site-packages/botocore/regions.py:260: NoRegionError
113
------------------------------ Captured log setup ------------------------------
114
INFO     botocore.credentials:credentials.py:1180 Found credentials in environment variables.

[maiden pawn]

https://stackoverflow.com/questions/50016862/grouping-tests-in-pytest-classes-vs-plain-functions

This answer presents two compelling use-cases for a TestClass in pytest:

Joint parametrization of multiple test methods belonging to a given class.
Reuse of test data and test logic via subclass inheritance

I think it looks like really good thing to abuse

[modern hedge]

I have a fixture in my tests which basically loads a file, and returns an object

ChannelFixture = Callable[[str], Channel]

@pytest.fixture
def channel():
    def wrapper(preset_file: str):
        path = pathlib.Path(__file__).parent / "assets" / "channels" / preset_file
        events = pyflp.parse(path).events_astuple()
        return Channel(*events)

    return wrapper

def test_channel_color(channel: ChannelFixture):
    assert channel("colored.fst").color == colour.Color("#1414FF")

I was thinking I could just convert it as a normal function and return the result, instead of this fancy fixture.
What do you think fixture or function?

[modern hedge]

Is like nobody here nowadays

[ember maple]

Depends on the usage, if its really just fetching data you use somewhere else, certainly extract the function

The win of the fixture is moving test prerequisites to the setup phase and easy dependency injection /caching

For example if the file content won't change, just make the fixture scope session and it will only be set up once per test session

[modern hedge]

Well its like files are divided according to test functions. The fixture I have just acts like a helper function for the real test function. I am refactoring rn but most likely I won't need any session scoped fixtures at all.

Also pytest recommends fixtures in conftest.py, I currently have many fixtures located and used only in the test module.

[tired coyote]

I had been using pytest fixtures but switched to simply instantiating my fixture data at module load time and saving in a constant. Then I just reference the constants from my tests. The tests are easier to read because there's less visual noise. And this works better with strict type checking. I think this looks very clean: https://github.com/public-law/open-gov-crawlers/blob/master/tests/public_law/parsers/aus/ip_glossary_test.py

GLOSSARY = glossary_fixture("aus/ip-glossary.html", ORIG_URL, parse_glossary)
METADATA = GLOSSARY.metadata

# ...

def test_the_name():
    assert METADATA.dcterms_title == "IP Glossary"

[modern hedge]

Well my fixtures are actually function wrappers. I am doubting if I should just make them normal functions instead

[fiery arrow]

Type checking is definitely a pain with fixtures. I suppose you could have a system similar to aiohttp's ApplicationKey:

f_glossary = FixtureKey[dict[str, str]]("f_glossary")
...

@f_glossary.implement
def f_glossary() -> list[str]: # error! list[str] incompatible with dict[str, str
    ...

# test_something.py
from .fixtures import f_glossary, f_database

@f_glossary.depends
@f_database.depends
def test_something(glossary: dict[str, int], database: SomeDb) -> None:
#                            ^^^^^^^^^^^^^^ error! dict[str, int] incompatible with dict[str, str]

but that might require some extensive changes to pytest

I think the main point of fixtures is for when you want to provide several values, or if you want to override a fixture in further tests

like:

@pytest.fixture
def user_folder(user):
    with tempfile.mkdir(prefix=f"user_folder_{user}") as path:
        yield path

@pytest.fixture
def user_photos_repo(user, user_folder):
    ...

# test_something.py
@pytest.mark.parameterize("user", ["alice", "bob", "charlie"])
def test_something_useful(user_photos_repo):
    ...

[woeful dagger]

I have never done unit tests on my code what do you guys suggest on where I start from

[fiery arrow]

write one test 🙂

[woeful dagger]

That's one way to do it XD

[kindred tangle]

I'm having trouble coaxing coverage for cython pyx files. Does anyone have a repo with a working example?

[pearl cliff]

did you enable the plugin in .coveragerc? https://cython.readthedocs.io/en/stable/src/tutorial/profiling_tutorial.html#enabling-coverage-analysis

[tired coyote]

Yeah, and since these are tests, it's ok with me if it crashes because the type is wrong. So I like lower ceremony, less type hinting on my tests.

[kindred tangle]

I believe so - I have:

[tool.coverage.run]
omit = [
     "tests/*",
     ]

plugins = [
    'Cython.Coverage',
    ]

in my pyproject.toml

[pearl cliff]

huh, seems right. i honestly have no idea otherwise. can you get it to work in a "toy" project? like a clean env with a simple pyx file with some trivial hello world function in it, and a trivial test case?

[kindred tangle]

yeah, I dunno - I'll try that next

[modern hedge]

Can I monkeypatch a descriptor's setter? I don't want a change in its value to affect other tests. My descriptor's setter validates and modifies the underlying binary data. If I monkeypatch, it won't affect the binary data right?

[pearl cliff]

it might be a problem if several tests are running concurrently in the same process

[modern hedge]

Nah all tests are synchronous

Does monkeypatch like preserve the previous state and restore it back?

[pearl cliff]

the monkeypatch library specifically? that i don't know.

i've only ever used unittest.mock and pytest-mock

unittest.mock.patch supports a context manager that will restore the original state, and pytest-mock has scoped fixtures that will do the same

[modern hedge]

Yea I meant pytest-mock

[modern hedge]

It can restore the original state of the binary data too?

[pearl cliff]

can you give a specific example?

[modern hedge]

Uh

Its like a descriptor in a class modifying the data of a dictionary field which itself modifies the binary data (bytearray)

Its complicated I don't know how to make a small example out of it

[pearl cliff]

just mean an example of the test and how you're using the fixture

i think by default the fixture is function-scoped, and should restore the original state of patched objects after patching

however it won't restore the state of things that were changed as a result of the patching, if that makes any sense

[modern hedge]

I haven't written the test, plus its just a.b = c literally. All the stuff is under the hood

[pearl cliff]

so what are you trying to test here?

[modern hedge]

A descriptor's setter

[pearl cliff]

but what about it are you trying to test?

[modern hedge]

Yes that's what I meant to ask

[pearl cliff]

yeah, i don't think that's possible. if you patch BinaryDescriptor.__set__, and then assign a.b = c where b is a BinaryDescriptor, then no, pytest-mock or unittest.mock won't keep track of the changed attributes on a. it will only reset BinaryDescriptor.__set__ at the end.

[modern hedge]

If the descriptor's setter correctly modifies the underlying binary data (which is also what is used by the getter)

[pearl cliff]

it sounds like you might need to restructure your tests (less shared state between tests?) or you might need to patch the underlying binary resource, whatever it is

[modern hedge]

Less shared state wdym

Ehh oh no

Its like some descriptors won't affect anything

But some do

[pearl cliff]

actually... why are you mocking anything at all here?

[modern hedge]

Certain descriptors values are dependant on others

[pearl cliff]

def test_binary_descriptor():
    class TestClass:
        dat1 = Descriptor1()
        dat2 = Descriptor2()
     # do stuff with instances of TestClass and make assertions
     ...

[modern hedge]

For example lets say max_objects is dependant on version. If version changes max_objects will return a different value and fail the tests

[modern hedge]

The descriptors aren't initialised like that, they are a part of the model classes in the code

[pearl cliff]

okay, well the general strategy remains, imo. create fresh class instances, mess with them, and make assertions about them.

[modern hedge]

I think I might get success with just isolating the testing of setters to a different function using a function scoped fixture

[pearl cliff]

that's also a good idea, but i'm strongly questioning why you need or want a fixture or mock at all here

[modern hedge]

Yea, I won't need to mock anything if I isolate the testing for certain descriptors to by using function based fixtures.

I need a fixture to parse a binary file into a model which has descriptors. And those descriptors directly operate on the underlying binary data.

[pearl cliff]

And those descriptors directly operate on the underlying binary data.
as in, data in a file on a filesystem? or something like a bytearray stored inside the object?

[modern hedge]

If fixture is session scoped, changing the value of a descriptor will affect all future dependant descriptor values

[modern hedge]

bytearray or an in memory stream like. BytesIO

[pearl cliff]

then just create 1 instance of the class per test, or maybe a function-scoped fixture that does so

no need to worry about resetting anything then

i see no need for mocking or patching

[modern hedge]

Also

I have a fixture which really is a wrapper function which returns a model from a collections of models, but I am unsure whether it should be a fixture or just a function

[pearl cliff]

fixtures facilitate code reuse. only use a fixture after you've reused the same functions over and over and you know they are going to be the same every time.

[modern hedge]

they are going to be the same wdym

[pearl cliff]

then it's a good use case for a fixture

[modern hedge]

Yea but syntactically functions look more easier to use

Plus I don't have to deal with type annotations

[pearl cliff]

yeah i actually do not really like fixtures in pytest

i think they are "too magical"

they have some use cases

but in this case i'd just use the functions

in general, i only use a fixture if i know i need a fixture

[modern hedge]

Like imagine importing typing.Callable in a unittest

[pearl cliff]

i do stuff like that, but i'm also kind of a maniac

[modern hedge]

Yea there's nothing about that function that actually make it wanna be a fixture. It has no use for scopes fixtures get

[proper wind]

def os_connect(region: str | None = None) -> Connection:
    auth = settings.XYZ_OS_AUTH
    return openstack.connect(
        auth_url=auth["AUTH_URL"],
        project_name=auth["PROJECT_NAME"],
        username=auth["USERNAME"],
        password=auth["PASSWORD"],
        region_name=region or auth["REGION"],
        domain_name="default",
    )


def get_regions() -> list[str]:
    connection = os_connect()

    regions = [region.id for region in connection.identity.regions()]

    return regions

Is this code easily testable, or should I pass connection as a parameter to get_regions() instead of instantiating the connection in get_regions()?

And how would I test the os_connect() method?

[crystal smelt]

I think it depends on the rest of your program. My guess is that you'll want to use your connection on more than one function, in which case you'd want to pass that as a parameter to get_regions(). I don't know that os_connect() is a great candidate for a unit test, since all it does is set a variable and then call openstack.connect().

[weary quarry]

Agreed on the structure that building the connection looks like something you'll want to do once and use repeatedly.

For testing it, a simple mock/monkeypatch to assert .connect is called with the expected parameter is about all I can see to test.

[crystal smelt]

Right, or make a mock that returns a mock object that can be passed to get_regions()

[weary quarry]

Yup. Huge benefit of what you hinted at here:

or should I pass connection as a parameter to get_regions() instead of instantiating the connection in get_regions()?

That's dependency injection and makes testing just that much easier because you mock the dependency directly instead of trying to patch something in at runtime.

[silk sedge]

What do you guys use to test Python code? For some reason I presume most people don't use unittest. Would pytest take the win here? At least it seems to be the winner in the pinned comments of the channel.

[weary quarry]

I would recommend pytest over unittest.

[pearl cliff]

i think most people use pytest on new projects. i've used unittest on some projects and it's fine, but pytest has a lot more configuration options and much nicer test reporting by default

i recently learned about another test framework called Ward but i don't know anything about it other than that it exists

[maiden pawn]

pytest can do everything unittest do, but more. No reason not to choose pytest. It is comfortable for complex fixtures

[maiden pawn]

i choose unittest only in cases when i wish to write code which uses no third party libraries. Has little amount of tests needed. Sometimes it happens. Neat little package

[proper wind]

def os_connect(region: str | None = None, auth: dict = None) -> Connection:
    auth = auth or settings.XYZ_AUTH

    return openstack.connect(
        auth_url=auth["AUTH_URL"],
        project_name=auth["PROJECT_NAME"],
        username=auth["USERNAME"],
        password=auth["PASSWORD"],
        region_name=region or auth["REGION"],
        domain_name="default",
    )

def get_regions(connection: Connection) -> list[str]:
    regions = [region.id for region in connection.identity.regions()]

    return regions

def get_hypervisors() -> dict[str, list[str]]:
    region_hypervisors = {}

    for region in get_regions():
        connection = os_connect(region=region)

        sorted_hypervisors = sorted(
            h.name for h in connection.compute.hypervisors(region)
        )

        region_hypervisors[region] = sorted_hypervisors

    return region_hypervisors

Thanks for the help on get_regions() guys, I will add the connection parameter to it.

But I'm struggling with how I could refactor get_hypervisors, because this one requires creating new connection objects with os_connect for every region... If I move the for loop out of the function and do the connection elsewhere, I will simply have moved the problem elsewhere.

[proper wind]

def get_regions(connection: Connection) -> list[str]:
    regions = [region.id for region in connection.identity.regions()]

    return regions

def get_hypervisors(connection: Connection) -> dict[str, list[str]]:
    region_hypervisors = {}

    for region in get_regions(connection):
        connection = os_connect(region)
        region_hypervisors[region] = get_hypervisors_in_region(connection, region)

    return region_hypervisors


def get_hypervisors_in_region(connection: Connection, region: str) -> list[str]:
    return sorted(
        h.name for h in connection.compute.hypervisors(region)
    )

I did this but idk if it's better code or even useful as a refactor... get_hypervisors is still coupled to os_connect

[pearl cliff]

i agree with this

[pearl cliff]

you need to create 1 connection to get the list of regions, and then create another connection for each region?

[proper wind]

Yeah... if I use the same connection for each region the list of hypervisors remains the same.

[pearl cliff]

it's possible that these functions are the "wrong abstraction"

how is this being used in the broader context of the application?

[proper wind]

class AppDjangoWebConfig(AppConfig):
    default_auto_field = "django.db.models.BigAutoField"
    name = "app_django_web"

    def ready(self):
        if settings.IS_RUNNING:
            connection = os_connect()
            cache.get_or_set("regions", (lambda x: get_regions(x))(connection), 3600)
            cache.get_or_set("hypervisors", (lambda x: get_hypervisors(x))(connection), 3600)

Simply like this

And then in views I call get_hypervisors or get_regions to pass the data to the frontend.

[pearl cliff]

so this connection process just needs to happen on startup, right?

one option is to use dependency injection to accept a connector function rather than a connector object

this still gives you the ability to test without patching, but allows your routines to internally create as many connections as needed

and the caller can implement connection pooling/caching/reuse if needed for performance optimization later

i'm actually a little confused. you need to connect to a region, and then pass the region again to get the list of hypervisors in that same region?

also what's with the lambdas that you invoke immediately? that's a little odd

maybe you could do it this way:

from collections.abc import Callable, Mapping, Sequence
from functools import partial
from typing import TypeAlias

# idk what the actual imports will be
import openstack
from openstack import Connection
from . import settings

RegionID: TypeAlias = str
HypervisorName: TypeAlias = str

def get_region_ids(connector: Callable[[], Connection]) -> Sequence[RegionID]:
    connection = connector()
    return [region.id for region in connection.identity.regions()]

def get_hypervisors(
    connector: Callable[[RegionID], Connection],
    region_ids: Sequence[RegionID],
) -> Mapping[RegionID, Sequence[HypervisorName]]:
    region_hypervisors = {}
    for region_id in region_ids:
        connection = connector(region_id)
        hypervisors = connection.compute.hypervisors(region_id)
        region_hypervisors[region] = sorted(h.name for h in hypervisors)
    return region_hypervisors

def acquire_openstack_resources(auth: Mapping[str, str] = settings.XYZ_AUTH, timeout=3600):
    def _connector(region_id: RegionID = auth['REGION']) -> Connection:
        return openstack.connect(
            auth_url=auth["AUTH_URL"],
            project_name=auth["PROJECT_NAME"],
            username=auth["USERNAME"],
            password=auth["PASSWORD"],
            region_name=region,
            domain_name="default",
        )

    region_ids = get_region_ids(_connector)
    region_hypervisors = get_hypervisors(_connector, region_ids)
    cache.set("regions", region_ids, timeout=timeout)
    cache.set("hypervisors", region_hypervisors, timeout=timeout)

class AppDjangoWebConfig(AppConfig):
    default_auto_field = "django.db.models.BigAutoField"
    name = "app_django_web"

    def ready(self):
        if settings.IS_RUNNING:
            acquire_openstack_resources()

you could also use the same injection technique for acquire_openstack_resources itself too

depends on where exactly you want to locate the "control"

[proper wind]

Ok cool. This is advanced for me but I will look through the code you wrote and get back to you in a bit! Thanks for having a look 🙂

[pearl cliff]

sure. the main concept is that you are injecting a function that acquires the resource and not the resource itself

that way you can achieve the goal of move control over the resource acquisition to somewhere "outside" of the current routine, while also avoiding the non-goals of duplicating logic inside the routine and tying implementation details of connection acquisition to "downstream" uses of the connection

[proper wind]

I see, very interesting. When you use Callable[[], Connection] what does this mean? A callable that takes no arguments and returns a Connection ?

And why do you prefer using the Sequence, Mapping types from collections.abc instead of the built-in list/dict annotations? Is it because they are more general ?

[maiden pawn]

Dictionary technically includes a lot of additional functionalities.
With specifying Mapping, he specified that variable needs to implement just __getitem__() dunderhead kind of.
So he followed the rule about... specifying as minimum as possible interface to be known

[pearl cliff]

precisely

[pearl cliff]

yeah, it's kind of just a style thing. you can use dict and list, but in this case you don't need to mutate the objects at all, so you might as well mark them as non-mutable

there are different philosophies on this. i go back and forth on it

[proper wind]

Gotcha! When I test this, will I have to mock the .compute.hypervisors() and .identity.regions() return values on the connector passed to get_hypervisors() and get_region_ids()?

[pearl cliff]

one option is to set up your test to pass in some dummy object that has a .compute.hypervisors nested property thing. another option is to refactor get_hypervisors like this:

def get_hypervisors(
    hypervisor_getter: Callable[[RegionID], Sequence[HypervisorName]],
    region_ids: Sequence[RegionID],
) -> Mapping[RegionID, Sequence[HypervisorName]]:
    region_hypervisors = {}
    for region_id in region_ids:
        hypervisors = hypervisor_getter(region_id)
        region_hypervisors[region] = sorted(h.name for h in hypervisors)
    return region_hypervisors

and adjust how you call get_hypervisors accordingly

then you can write your tests like this:

def test_get_hypervisors():
    fake_region_hypervisors = {
        'region1': ['h1', 'h2', 'h3'],
        'region2': ['h4', 'h5'],
    }
    region_ids = list(fake_region_hypervisors.keys())
    def hypervisor_getter(region_id: str) -> list[str]:
        return fake_region_hypervisors[region_id]

    region_hypervisors = get_hypervisors(hypervisor_getter, region_ids)
    assert region_hypervisors == fake_region_hypervisors

basically you're just checking that the for loop isn't goofed up somehow, so this is a bit of a silly test. but i still find it valuable to have "smoke tests" like this

abstract away the unimportant stuff, test invariant properties of the relevant stuff

your tests are very often an expression of the assumptions that are built into your application

if nothing else, they serve as documentation for how your code is supposed to behave

actually no, there's a bit more to test here

you need to check that the results are sorted!

so there's legitimately something to test here

def test_get_hypervisors():
    """Test that get_hypervisors() returns the expected results, in sorted order."""

    fake_region_hypervisors = {
        'region1': ['h1', 'h2', 'h3'],
        'region2': ['h4', 'h5'],
    }
    region_ids = list(fake_region_hypervisors.keys())
    def hypervisor_getter(region_id: str) -> list[str]:
        return fake_region_hypervisors[region_id]

    region_hypervisors = get_hypervisors(hypervisor_getter, region_ids)

    # Every region id is in the results (no omitted inputs)
    # Every result is one of the inputs (no extraneous results)
    assert set(region_hypervisors.keys()) == set(region_ids)

    for region_id, hypervisor_names in region_hypervisors.items():
        # All results are present, no results are erroneously added
        assert set(hypervisor_names) == set(fake_region_hypervisors[region_id])
        # No duplicates
        assert list(set(hypervisor_names)) == list(hypervisor_names)
        # Results are sorted
        assert sorted(hypervisor_names) == hypervisor_names

this is a little contrived, but i have caught real bugs with tests like this

[proper wind]

Wow haha, didn't think I would learn so much today 😄 I'm a junior this was taking me a lot of time and I was spinning my wheels but this helped me out a lot! I'll be implementing it and going through it line by line today. I'll also write more tests

[pearl cliff]

sure. note that this "injection a function" style is not very common in python

so i would also be careful with it

and maybe make careful note in the code of why you chose to do it

sometimes i leave big block comments with my initials and the date at the end

# NOTE: I have chosen to use a dependency injection technique
# wherein the caller injects a *function* that returns a
# resource. This is meant to balance testability (avoiding mocks)
# with reuse of logic that is going to be repeated in several
# places throughout the application. It's a little idiosyncratic
# for Python, but it seemed like the cleanest way to proceed.
#  - SRL 2022-10-06

personally i love seeing comments like this when i'm reading other people's code

the only problem is when some other code chimpanzee comes along and modifies it and ignores the comment and doesn't update it

but that's imo entirely a cultural problem that will depend heavily on the organization you work for

this pattern is uncommon in python in part because lambda is so incredibly ugly/clunky

people do this all the time in ocaml and scheme where creating an anonymous function is syntactically trivial

[modern hedge]

DI is pretty common in .NET outside of testing as well

Implementing platform specific logic in different classes inheriting a common inheritance. And passing an object by inheritance to the class at runtime for example

[pearl cliff]

i think the less-common pattern here is injecting a "factory" vs. injecting the object/resource itself

in C# im sure people write IHypervisorLookupFactory all the time

but that's a little jarring in python, and unnecessary to make it a whole interface with a class, when it could just be a function

[modern hedge]

Factory classes have a different purpose but yea, the general idea is to separate the implementation from the interface.

That's something of a must in statically typed languages, especially one without multiple inheritance

[fiery arrow]

Actually it's a very common pattern. Every web framework works like this

But maybe there is resistance to do it in your own code

[pearl cliff]

what web framework uses dependency injection with callable functions? i know starlette itself kind of does that internally

[maiden pawn]

FastAPI offers by default
plus book Python Expert Programming has dedicated chapter for this, and shows that other frameworks, like Flask can have it added too

[fiery arrow]

I meant that in a web framework, you provide the concrete implementations of your endpoints as functions

which, well, inverts the dependency

Imagine if you had to actually clone flask, go into its source code and put your stuff inside some on_get giga-method

[pearl cliff]

oh, well yeah. tornado's handler classes let you actually "pull" the http data from the request like that, but via instance methods and not "injected" functions via a function parameter in your handler

[pearl cliff]

are you talking about fastapi Depends?

[maiden pawn]

Yeah

They can be overridden on application creation

For testing purposes

[pearl cliff]

yeah just to be clear, i'm not saying that dependency injection in general is weird

i'm saying that injecting a function that the caller uses to acquire a resource rather than injecting an instance of the resource is relatively uncommon in python

from the perspective of a user, fastapi Depends is the latter, even though the implementation naturally uses a function that the programmer provides in advance

i think the only practical user-facing example of this that i've seen in python is hypothesis @composite, where the first parameter is a function that you are expected to use inside the compose strategy function to draw values

https://hypothesis.readthedocs.io/en/latest/data.html#composite-strategies

[proper wind]

How do you create the dummy object that has .compute.hypervisors nested property ? With monkeypatching? I like having get_region_ids and get_hypervisors both take in a callable.

[pearl cliff]

literally just a class TestConnection or something like that

[proper wind]

Gotcha 🙂 I did it like this:

class TestConnection():
    def __init__(self) -> None:
        self.compute = TestCompute()

class TestCompute():
    def hypervisors(self, region_id: str) -> Sequence[str]:
        return [TestHypervisor("h1"), TestHypervisor("h2"), TestHypervisor("h3")]

class TestHypervisor():
    def __init__(self, name) -> None:
        self.name = name

class UtilTestCase(TestCase):
    def setUp(self) -> None:
        return super().setUp()
    
    def test_get_hypervisors(self):
        """Test that get_hypervisors() returns the expected results, in sorted order."""

        fake_region_hypervisors = {
            'region1': ['h1', 'h2', 'h3'],
            'region2': ['h4', 'h5'],
        }
        region_ids = list(fake_region_hypervisors.keys())
         
        def hypervisor_getter(region_id: str) -> list[str]:
            return TestConnection()

        region_hypervisors = get_hypervisors(hypervisor_getter, region_ids)

[pearl cliff]

btw, this is why my 2nd proposal was to inject a "function that returns hypervisors" rather than "an object that implements a .compute.hypervisors attribute". the former is easier to test and is a simpler interface in general.

[proper wind]

Right. Ok. Well now that I implemented it this way I will try the second way you proposed. Slowly getting a better understanding of things 🙂

[silk sedge]

Thank you very much! (To you as well, @weary quarry) for your takes on the matter. I'll inspect pytest more closely ;)

[lean tendon]

how to use selenium with captcha?

[silk sedge]

How would I go about having this fixture be present in all TestStack tests? I've tried the line stack = stacklord.Stack() inside the class, but then the object is the same across multiple tests (which is not what I want).

@pytest.fixture
def stack():
    return stacklord.Stack()


class TestStack:
    def test_insert_and_read(self, stack) -> None:
        test_val: int = 5

        stack.insert(test_val)

        assert stack.read() == test_val

    def test_pop(self, stack) -> None:
        test_val_1: int = 7
        test_val_2: int = 9
        stack.insert(test_val_1)
        stack.insert(test_val_2)

        stack.pop()

        assert stack.read() == test_val_1

    def test_pop_on_empty_raises_exception(self, stack) -> None:
        with pytest.raises(stacklord.EmptyStackError):
            stack.pop()

    def test_read_on_empty_raises_exception(self, stack) -> None:
        with pytest.raises(IndexError):
            stack.read()

    def test_read_out_of_bounds_raises_exception(self, stack) -> None:
        with pytest.raises(IndexError):
            stack.read(1)

[pearl cliff]

it sounds like you need to change the scope of the fixture, which determines when the fixture is "reset" and a new value is generated, instead of reusing the same value over and over

https://docs.pytest.org/en/7.1.x/how-to/fixtures.html#scope-sharing-fixtures-across-classes-modules-packages-or-session @silk sedge

[silk sedge]

But if I increase the scope to, say, class, then I'll have one value for all functions, right? That's not what I want, as I said.

I do want the value to be reset every test (namely, if it wasn't, test_pop_on_empty_raises_exception as well as the exception test for read would break). I just don't want to pass stack as a parameter everytime.

I mean, "Getting Started" states:

Something to be aware of when grouping tests inside classes is that each test has a unique instance of the class. Having each test share the same class instance would be very detrimental to test isolation and would promote poor test practices.

But that doesn't seem to work with mutable values (?)

# content of test_class_demo.py
class TestClassDemoInstance:
    value = 0

    def test_one(self):
        self.value = 1
        assert self.value == 1

    def test_two(self):
        assert self.value == 1

Like here, test_two fails because it sees self.value as 0 even though test_one did self.value = 1.
But something like stack = stacklord.Stack() refers to the same object across tests.

[river pilot]

are you sure that gives you the same object? It looks to me like it would make a new one for each test.

You might have a bug in your stacklord (class? module?) that it's coupling objects together.

[silk sedge]

stacklord is a module, Stack is a class

For a second I had forgot about Python's thing of evaluating default arguments at time of definition, so yes, all my Stack objects used the same underlying list, but that's not there anymore.

[silk sedge]

The code I have right now (the one in the message this one's replying to) is working just fine, I'm just wondering if I can make it a little better ^^

[river pilot]

one way to improve it would be to get rid of the class, and make the methods be functions. You don't use self at all.

[silk sedge]

Thought I would do so. I'll try, thanks!

[pearl cliff]

ah. try creating and assigning self.stack in the __init__ method of the class. that way you get a fresh clean object to work with in every test method, because __init__ is called whenever a new instance is created, which (as you quoted above) pytest does for every test method

assigning value = 0 in the class definition defines a class attribute, not an instance attribute, which you would have to define in an instance method (like __init__)

# content of test_class_demo.py
class TestClassDemoInstance:
    def __init__(self):
        self.value = 0

    def test_one(self):
        self.value = 1
        assert self.value == 1

    def test_two(self):
        assert self.value == 1

[silk sedge]

No way

This was the first thing I thought, but...

In the explanation for pytest's test discovery:

"inside Test prefixed test classes (without an __init__ method)" so I assumed it kind of did something special with class variables and did something else with the init function

but thanks I'll try that!

It works just fine!

I wonder why it says "without an init method", though...

Oh I guess I found out.

Turns out it doesn't work.

Test discovery really does ignore tests with an init method.

tests\test_core.py:15
  C:\anvildir\Practice\Programming\Python\StackBasedInteractiveLang\tests\test_core.py:15: PytestCollectionWarning: cannot collect test class 'TestStack' because it has a __init__ constructo
r (from: tests/test_core.py)
    class TestStack:

tests\test_core.py:49
  C:\anvildir\Practice\Programming\Python\StackBasedInteractiveLang\tests\test_core.py:49: PytestCollectionWarning: cannot collect test class 'TestStackHandler' because it has a __init__ con
structor (from: tests/test_core.py)
    class TestStackHandler:

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html

[pearl cliff]

i actually didn't realize that... weird

@silk sedge try overriding __new__ instead as a workaround/hack maybe?

i wonder why it ignores classes with an init method

what happens if you subclass a class that has an init method?

[silk sedge]

I'll try something else, actually.

I'll try defining the fixture inside the class.

But let me note down for myself to explore the source code in search of a reason why init can't be used.

Ok yeah it doesn't work still; I'd have to pass it manually to each test function.

[pearl cliff]

do note that you can get the same benefits with a fixture that returns an instance of some class that is useful... you can maybe even call it self 😉

i wonder if that restriction exists to avoid some unsupported use case in unittest or nose

https://github.com/pytest-dev/pytest/issues/7033 seems more of a refusal than a technical need

[silk sedge]

Oh well.

It's not like I'll die or anything; I'm just sort of curious and always trying to find a better way to do things.

It'd be nice if I could use my classes not only to structure tests and nest them under what essentially are namespaces, but also have related code apply to all of them (which I would say is the reason classes exist after all).

Pytest's idea of no setUp + tearDown functions is really neat until you come to something like that; with a setup I could simply do self.stack = stacklord.Stack() and that would run for each test.

[pearl cliff]

that's a function-scoped fixture

[silk sedge]

A default one, yes.

[silk sedge]

(and I wouldn't have to pass an additional parameter for every function)

(Also, features like LSP identifier renaming don't work because static analyzers don't understand that the stack parameter for a method refers to the stack function identifier)

I don't know if I'm missing a big point on here but when writing a testing lib I think that's one of the first things you'd think about.

Now pytest just seems... stupid. At least until I discover either 1) a good reason for it to be like that or 2) how to do what I first intended to do in a way that doesn't look hacky and cryptic.

[pearl cliff]

is self.stack so much better than putting it in the signature?

it seems like the objective of the fixture system is to make it inescapably apparent when a fixture is being used

[silk sedge]

In a sense, yes: renaming self.stack to self.stack_instance is a one-step action for me.

[pearl cliff]

oh, i see

yeah.... to be clear, i actually do not like the "implicit" fixture system at all

would much rather use e.g. a decorator

also i agree in that i don't understand why they can't allow init methods with 0 extra parameters other than self, and declare that test class instances are always function-scoped

[silk sedge]

Well, that works absolutely fine for standalone functions defined linearly. But grouping tests under a class without (ab)using the strengths of a class (more specifically, the power of sharing behavior across members or interaction between them) seems kind of... unintelligent.

[pearl cliff]

i guess this is just one area where pytest is "opinionated" without recourse

[silk sedge]

I guess they did a better work than the standard unittest in reporting test failures and such, but I wouldn't give it more than that, really.

[pearl cliff]

there is indeed a lot more to pytest than that

it's always frustrating to encounter something like this, but the core piece of technology is very good, and in almost all other respects is tremendously configurable

[silk sedge]

I mean, I would say the same, but it's not just about opinion once it starts to actively make my test code harder to change and look like it was designed for a language without tests.

[pearl cliff]

fwiw there is a proposed way to make this work in the gh issue i linked, although it's ugly

[silk sedge]

I don't really dislike it: but this is not something "obscure" so that I would expect it not to support this (or to have been thought about). It's like buying a painting kit and getting everything except two of the most used colors are missing. It's a thing if the pencil if half a millimetre smaller than requested, because it's a lot less noticeable.

[pearl cliff]

maybe you can make a feature request laying out the specific use case of fixture renaming being very difficult, and that instance attributes don't have that problem and are also a natural tool here anyway

[silk sedge]

I probably will just for the sake of contributing to the project. But I'll probably jump ship to another testing lib or back to unittest as well.

[pearl cliff]

from the perspective of the pytest devs it probably is an obscure corner of the library meant to avoid use cases that they don't intend to support because they don't understand them. i have encountered that before in surprisingly high profile projects. that said, ultimately this is the nature of using tools built by other people, and particularly open source tools that cost nothing and in which you have no financial or moral stake. this is also why competition is a good thing even if there is a de-facto standard or popularity contest winner: competition means choice.

[silk sedge]

All true. Still so far it looks like it's something designed for a language without classes.

[proven horizon]

would there be a reason, why requests.get(url, auth=HTTPBasicAuth(username,password)) works, but when I run the same request inside a unit test, it fails with 401 ?

[pearl cliff]

not an obvious one related to testing that i can imagine

[proven horizon]

yeah, it was the username/password... had to strip the double quotes

[pearl cliff]

when debugging, it's usually safe to assume that whatever framework or library you are using is not the problem, unless it's glaringly obvious. and even then, it's prudent to really really make sure that you didn't just make a mistake in your own code, before trying to identify other causes.

that is, bugs in your own code should almost always be your first suspicion

[proven horizon]

yip, it was a bit strange, that environment variables was loaded with double quotes in the unit test, but not when running via script

but solved none the less

[pearl cliff]

sounds like a bug in your test setup more than anything

what if the env var actually does need to contain quotes?

(it probably doesn't, but a password absolutely might)

[proven horizon]

we use tox, and we pass in the environment variable from the pipeline or .env, when using .env it loads the double quotes

[pearl cliff]

do you have quotes in the .env file?

[proven horizon]

yeah

[pearl cliff]

does tox have built-in .env handling?

or did you write your own?

i would always try to fix the problem at the source (when env vars are loaded) if possible, rather than ad-hoc inside tests where the problem happens to arise. fix it once, for all tests today and tomorrow, and fix it right so that your tests actually are realistic w/ respect to real world operating conditions, and so you don't build hacks into your application that later turn into subtle bugs (like stripping quotes from a field where quotes are perfectly valid at the beginning or end)

e.g. maybe this is a bug in tox, in which case you might need to add some kind of centrally-located test setup code that strips the quotes. but you should do your best to ensure that this isn't somehow a matter of user error!

[proper wind]

@pearl cliff I'm struggling a bit.

def get_hypervisors_in_1_region(
    hypervisor_getter: Callable[[RegionID], Sequence[HypervisorName]],
    region_id: RegionID,
) -> Mapping[RegionID, Sequence[HypervisorName]]:
    hypervisors = hypervisor_getter(region_id)
    region_hypervisors = sorted(h.name for h in hypervisors)

    return region_hypervisors


def get_hypervisors_all_regions(
    connector: Callable[[], Connection], region_ids: Sequence[RegionID]
) -> Mapping[RegionID, Sequence[HypervisorName]]:
    region_hypervisors = {}

    for region_id in region_ids:
        hypervisor_getter = connector(region_id).compute.hypervisors
        region_hypervisors[region_id] = get_hypervisors_in_1_region(hypervisor_getter, region_id)

    return region_hypervisors

I wrote this new function, because our get_hypervisors that expected a "function that returns hypervisors" didn't have a way to create a new connection for every region in the for loop.

connector(region_id).compute.hypervisors this is specifically what the old get_hypervisors did not support.

Is there a better way to do this? Because now I will have to create TestConnection, TestHypervisor, etc. classes which is what we were trying to avoid. Basically get_hypervisors_all_regions brings us back to our original problem which is that it expects a connector object and it has to know the internals of the connector object (.compute.hypervisors).

[magic gust]

anyone here familiar with pytest from dash?

[prisma tendon]

Can someone please help me with fuzzing?

[maiden pawn]

don't tell me, with FizzBuzz 🙂

[river pilot]

what do you mean "from dash"? pytest is well-known.

[magic gust]

i just thought dash was bigger

[river pilot]

what is dash?

@magic gust ^^

[magic gust]

https://dash.plotly.com/testing

[river pilot]

looks cool, though I don't have a use for it.

[magic gust]

okay...?

but you are familiar with pytest?

[river pilot]

yes

[magic gust]

do you think you could help me in #help-coconut

[swift pewter]

Is there a coverage tool (coverage.py or any other) that shows sub-line coverage? E.g.:

if x or foo():
    bar()

If x is truthy in all of my test cases, foo() never gets executed, but all lines are marked as covered. I would love to be able to inspect coverage based on AST nodes or something like that.
The only cases I can think of where this matters would be and, or, and ternary if, plus some stuff in unpythonic code (e.g. if False: foo() in one line).

[pearl cliff]

coverage.py has "branch" coverage that might cover something like this

however you have to enable it, it's not turned on by default (not sure why, maybe it's slower)

[swift pewter]

I've found branch coverage, but afaict that's still line-based, just more thorough ("did this line that can jump to two possible lines jump to both of them?")

[river pilot]

@swift pewter coverage.py doesn't do what you want.

it's all line-based, because of how sys.settrace has traditionally worked. Now you can ask for bytecode tracing, but I've never done anything with it, and there are complications to use that info to get the data you want anyway.

[pearl cliff]

this is good to know, i never considered that short-circuiting expressions would be "hidden" from coverage

[faint lake]

If anyone's available to test, I need someone to test out my OwO detection module
https://pypi.org/project/is-furry-py/

[cerulean fjord]

i need help with selenium, I'm not sure how xpath works

[swift pewter]

I've attempted it briefly, but yeah, even 3.11's co_positions() can't be relied upon (see image: green is what is supposedly covered, based on opcodes executed that claimed to have these positions, red is what is not covered). I guess the JUMP_* opcode that the if compiles to just says its position is the entire condition, which isn't exactly wrong either..

[river pilot]

i haven't taken the first look at how the bytecodes are reported. But based on previous experience, there will be bugs to be fixed in CPython before it's useful.

[swift pewter]

co_positions just yields (lineno_start, lineno_end, char_start, char_end) tuples for each bytecode instruction. I think in this case at least the positions are not wrong, they are just not helpful for my usecase.

[river pilot]

@swift pewter this is using the settrace that reports bytecode positions?

[swift pewter]

yes

[river pilot]

that's what i was afraid of 🙂

[swift pewter]

Progress: if I skip *JUMP* opcodes and change the condition such that it can't be optimized away, it sort of works.

[river pilot]

let's see

[swift pewter]

With my tiny example it works

[river pilot]

i guess the reporting would have to do something about = and return

(and if)

[swift pewter]

Sure, some cosmetic changes would be needed, but those sound doable. I'm more worried how it would behave when tested with real code.

[river pilot]

right, and if you have to special-case some bytecodes, then i worry about future changes to the bytecode

[swift pewter]

like maybe not all JUMP opcodes are irrelevant for coverage :D

[river pilot]

@swift pewter if you like, you could add to this issue: https://github.com/nedbat/coveragepy/issues/292

[swift pewter]

Will do, once I have a real prototype.

[river pilot]

cool 🙂

[signal pier]

I have a question regarding unit-testing for pandas's DataFrame:
What is the best practices?
Should I Create, for each function, two DataFrames: input DataFrame and expected output DataFrame, that I would save aside,
and then to run the function on the input DataFrame and compare it with the expected output DataFrame?

[maiden pawn]

lol. unittest has actually autodiscovery. xD

python3 -m unittest discover

[proven horizon]

this is probably a heated topic, maybe not, but why would you choose pytest over unittest ? I find both equally good and efficient, and do not really have preference, only where I've had to do a 'large' amount of testing, and there I found pytest much better, but for some of our microservices, unittest seems approriate ?

[maiden pawn]

pytest is having everything unittest has but more. Just because of fixtures alone, and more convinient ways to paramterize stuff, it is better to choose pytest
different scopes of pytest fixtures make kind of the best optimization regarding testing.

usually unittest is used only.... in legacy code where it was initially used
in small package-libraries, which aren't having third party deps it can be convenient choosing unittest

[modern hedge]

colored output is a big reason for me personally

and seamless vscode integration

unittest would fail to discover where tests are when i was new to testing itself

[analog basin]

Let me know if this is a bad place for this, but does anyone have recommendations for a good framework for API integration testing using an OpenAPI/Swagger spec? I'm looking for something that offers automated type testing of request results as well as the ability to write custom tests.

Currently looking at Schemathesis, any others I should look at?

[maiden pawn]

I went for Schemathesis too. Besides that i tried it is easy to make custom recursive class-parser of OpenAPI. So technically custom solution fitting any desires is possible, but kind of pointless. Schemathesis is already very nice solution.

[royal orbit]

Might be biased since I wrote a paper about it, but Schemathesis is the best IMO 😎
See https://arxiv.org/abs/2112.10328 for details!

[pearl cliff]

schemathesis is awesome

i just integrated it into our test suite last week, very pleased with it

[royal orbit]

You might also be interested in the github app? https://github.com/apps/schemathesis

[pearl cliff]

i just put it into our regular test suite for now, which runs in CI and blocks merges to main if the tests fail:

import schemathesis

schemathesis.fixups.install(["fast_api"])
api_schema = schemathesis.from_asgi('/openapi.json', app)

@api_schema.parametrize()
def test_api_schema_conformance(case: schemathesis.models.Case) -> None:
    """Test the API for schema-conformance.

    Note that this is not an exhaustive test. Use `make test-fuzz-api` for that.
    """
    response = case.call_asgi()
    case.validate_response(response)

the default setting of 100 tests is fast enough on our current code (although in the future i might reduce the number as we add more complicated machine learning to the API)

is there an advantage to using the github app in a private project (it is hosted on github, but in our company's private repo)? does it run a more intensive fuzzing job over a longer period of time? does it make sense to use it if we are using TeamCity for our "main" CI, and not GH Actions?

i've been wondering about best practices for when, how, and how often to run "intensive fuzzing" vs. the usual unit test suite

i had this issue w/ a previous project too using hypothesis, i had a somewhat complicated @composite strategy, the generated data was written to a database, and then a nontrivial query was executed on that data

it was great because i felt much more confident in my code and query correctness than i would have felt otherwise, but it was definitely a slow process, so i had to reduce the number of examples a lot in order to have an acceptable run time. i had it set to run a much larger number of examples if i set an env var like TEST_EXHAUSTIVELY=1, which i would use for my own testing before pushing my code

[royal orbit]

If you've got it set up in your regular tests, I'd probably just stick with that for now 👍

More generally "when and how long to fuzz" is basically a question of cost: how much do you pay to run the tests, and how much would you pay to find a bug sooner?

Decent starting point is to run tests overnight each night on a server, and see whether you feel you're getting value out of that.

https://hypofuzz.com/ adds coverage feedback and can statistically estimate the time to next bug - even if no bugs so far - though it's still very much in alpha at the moment.

[pearl cliff]

thanks, this is good stuff to think about

[lofty junco]

Hello Folks
May I know how to create a test mysql db when performing pytest.
I do have db interface injected into main system application

[pearl cliff]

use a pytest fixture that constructs the db https://docs.pytest.org/en/stable/explanation/fixtures.html

[split mason]

hey guys,
for my cs class I have to write a test for init function and it has to be done using pytest
have anyone done that before? cuz i have no idea how to do it and i havent found anything useful on stackoverflow/youtube

[maiden pawn]

https://docs.pytest.org/en/7.1.x/getting-started.html
https://realpython.com/pytest-python-testing/
Recommending to read better docs
Official docs are cool, and as last resort text tutorials are nice too

[split mason]

okie, ill take a look at it
thank you

[placid forum]

So im learning about like automated tests and how to basically combine that with your requirement status. Which is fine and I understand this but I genuinely cant wrap my head around this statement from an article im reading For this, you will need to link your Tests with your requirements, and report your automated test results from your CI environment.

[maiden pawn]

probably it means:
Write tests that assert each your requirement, then insert their invocation into CI pipeline
(each requirement is some specific piece of logic / value that can be checked. So

for requirement in requirements:
  have written test:
    that asserts it correctness

[royal orbit]

This looks to me like the author is repeating something half-understood from a textbook 😓
In some industries (aerospace, medical, natsec) you might be legally required to identify for every line of code and every commit the written requirement that makes it necessary (in the hundreds-of-pages requirements document decided far in advance). This is "fine, I guess" in 1968, but very few software projects these days have such requirements documents and that's generally a good thing!
So I wouldn't worry about it or bother with a process for this 🙂

[astral quiver]

Hello. Can someone help me with a webscrape tablular data from pdf to dataframe in python ? I tried tabula (Showing ambiguous error) and camelot libraries

[pearl cliff]

ironically i think there's actually a more charitable interpretation here, which i learned originally from Hypothesis 😆

even if we don't have a meticulously crafted requirements document for our applications, we definitely do still have various "requirements" in our minds, and implied in the behavior of the software. i think it's a good idea to try to make those requirements explicit, in order to figure out what tests to write.

[amber drift]

hello how do I regex that only validate with this path
/home/example.sh

[pearl cliff]

you don't need regex, just use ==?

[proper wind]

Guys i need help in an exercise

is there any way somone can help?

[slow mist]

Is there a particular reason why I can't provide user input while testing in VSCode? I could do it in PyCharm, but the Python Test Log in VSC is read-only.

For my tests to work I need user input (an authorization code for API calls) that changes every time.

[maiden pawn]

pass it through pip package pyexpect

[maiden pawn]

it is not meant to be done in testing in general. It makes them as not autotests

still possible though

[slow mist]

Well I can't keep the auth code the same, because the API I connect to generates a new code every time...

[maiden pawn]

automate getting this new code

[slow mist]

Why do we need to overcomplicate things? I don't mean this in an aggressive way, but why does PC allow me to simply pass the code

[maiden pawn]

and/or exclude with pytest.mark those tests from regular unit test runs

[slow mist]

And then all tests are automatically executed

[maiden pawn]

run from CLI tests if u a that picky

it is bad practice nevetheless

very bad

[slow mist]

So I'm supposed to over-engineer something my project doesn't even use for the sake of unit testing?

[maiden pawn]

at the level of catching all Exceptions and making them pass ignored, or importing only *

[maiden pawn]

it is Super idiotic move lets say it straightly

it is shooting into your own legs

from a shotgun

[slow mist]

?

Do you even know me? Why are you insulting me?

[maiden pawn]

i just name it honestly. it is super dumb move

learn from it and stop doing that

it is like using Sci fi freeze gun to cool your PC for every day usage

igniting cigarettes to smoke from nuclear reactor

[slow mist]

Maybe for a person whose entire life is coding. Not for a hobbyist. I never wrote unit tests, and decided to give it a shot just to learn something new. I don't know all the best practices for testing.

My code will work just fine without them.

Anyways, I'll see what the best way to get the auth code automatically is. You should work on your social skills.

[maiden pawn]

anway, that was explanation why such thing should not be supported by IDE

[slow mist]

By VSCode*

[maiden pawn]

VSCode is IDE (Integrated Development Environment)

[slow mist]

No need to spell IDE for me. It's a text editor with optional extensions.

[opaque gorge]

guys i want understand the steps for create a software and deploy anyone can help me?

[maiden pawn]

too broad question

both topics are large af

[opaque gorge]

the method agile is the more used?

[maiden pawn]

Steps to create software are mentioned in System Analysis and Design by dennis alan
(Planning, Analysis, Design, Implementation, Maintenance)
Steps to create implementation of software in code terms are all mentioned in Code Complete by McConnel)
Steps to deploy software i have nowhere seen yet, but if u mean web apps,

• then it is good to make a start from Docker Deep Dive for docker to solve most of first deployment issues and to pack app for deployment,
• then Terraform Up and Running for terraform that automates cloud provider actions,
• and optionally stuff like from book Practical Ansible 2 to weaponize you with first most simple way to deploy applications to linux servers directly

[opaque gorge]

thanks soo much my friend, i was lost thanks!

[maiden pawn]

U a welcome. I would mention here also Gitlab CI which u can learn from its official docs.
This tool unites all deployment steps into one automated action (optionally with manual click to verify deployment)

Learning Gitlab CI alone -> U can automate local testing of your application
Learning Gitlab CI+Docker -> U can automate building your application packed ready for deployment with frozen dependencies (important for python), and reuse image in different stages
Learning Gitlab + Ansible -> U can automate redeploying your application to target server automatically
Learning Gitlab + Ansible + Docker -> U can automate redeployment to target server with easy way to autowipe prevoius deployments
Learning Gitlab + Terraform + Ansbiel + Docker -> U can add so that your servers, DNS, managed databases from cloud provider are autocreated automatically, before proceeding to next step of automated deployment of application itself.
(+After established automated deployment u can add automatically invoked end to end tests)

[opaque gorge]

okay, thanks so much!

[bitter wadi]

Hey @quartz pilot!

It looks like you tried to attach file type(s) that we do not allow (.pdf). We currently allow the following file types: .gif, .jpg, .jpeg, .mov, .mp4, .mpg, .png, .mp3, .wav, .ogg, .webm, .webp, .flac, .m4a, .csv, .json.

Feel free to ask in #community-meta if you think this is a mistake.

[quartz pilot]

Can someone help me with this

[raven igloo]

@maiden pawn I just have to say... your handling of the super-rude guy above displayed a LOT of patience. and he/they/it had the nerve to say that YOU needed social skills! well done.

[copper lintel]

hi there, i may have to test requests.get and I wonder how to fake things to avoid hitting real urls

mocking a server ? patching requests.get ?

[maiden pawn]

Thanks. 🙂

[slender harness]

How many loops would you say is enough to measure the performance of some function? In this case I'm using a unit test for performance regression testing

[swift pewter]

It depends on how long the function runs, and on other factors. I'd just use the timeit module to make the decision for you.

[livid bay]

Hi, can someone help me configuring the Testing option from vscode for pytest? Running pytest from terminal works fine but on vscode i get the errors that it can't find any module :/

[livid bay]

Same error appears when i try to use coverage

[smoky tapir]

Did you install Python extension for it?

[long jetty]

is vscode using the same python env as the terminal?

[modern hedge]

Depends, you can check it when a py file is open at the bottom right side

[low oyster]

unittest isn't really used for writing end-to-end integration/functional tests, right?

pytest is a better option for that?

[pearl cliff]

this question doesn't make a lot of sense. people prefer pytest over unittest because it has more features and there is less boilerplate involved in writing tests. it has nothing to do with the kinds of tests being written.

[low oyster]

I see. Well, I got confused because it's called unittest so i assumed it's probably useless for integration tests

Is there a channel for things like uwsgi?

[pearl cliff]

that assumption is incorrect. the terms "integration tests" and "unit tests" refer to the amount of code being tested. the actual test framework more or less just runs arbitrary code and checks for assertions.

the name unittest comes from JUnit, a java test framework that inspired it. however pytest is probably better for most new projects, it's a lot less verbose to use, and it has a lot of very powerful features.

[pearl cliff]

#web-development

[low oyster]

I'bve heard the opposite! That pytest requires a lot of boilerplate code and unittest quickly gets one started

[pearl cliff]

who told you that?

[low oyster]

I think I read this in one of the google search results

But, if that's incorrect that's fine

I can always switch

[pearl cliff]

no, use whatever you currently use

learn one framework, then try the other one after you have experience with the first

i actually think it's great if you start with unittest, because the framework itself is simpler and has fewer complicated features that might tempt you

you will end up DIYing some features that pytest has included, but i think it's actually a good learning experience

[low oyster]

so, i wrote functional tests wayyy later

and i incorrectly assumed that unittest wouldn't work for it so I integrated pytest

[pearl cliff]

you definitely don't want to mix both

[low oyster]

exacty

[pearl cliff]

pytest supports unittest so that existing big applications can easily transition from unittest to pytest gradually

[low oyster]

so i wanna switch the func tests over to unittest if i can

[low oyster]

makes sense!

[low oyster]

Hi - I'm not sure if my google search keywords are wrong but I'm searching for how to articles for writing func test using unittest and I can't find a definitive guide - does anyone have a good resource for this?

actually

Would it make better sense to move all unittest unit tests to pytest ?

[proper wind]

How do you do unit tests for python notebook pandas code

[low oyster]

Folks, I have a question. My app uses mysql as data store. I'm starting to write functional tests where I spin up an instance of the app and test the flow end to end. What do I do about the database bit? I could spin up a SQLite db except some MySQL statements will not be SQLite syntax compatible.

What do I do? Or should I make the test such that it relies on a database is separately setup and I pass the env vars and test?

[safe arch]

Is there a channel for broader testing (eg. functional, hypothesis BDD, gherkin, greatexpectations.io , Robot Framework, etc.) ?

[swift pewter]

pretty sure this channel is fine for testing-related questions that aren't strictly about unit tests

[safe arch]

I'm workin on a little something because it doesn't seem to exist anywhere:
A kit for creating developer portfolios...
https://vitrina.readthedocs.io/en/main/howto/consume.html
I wonder if anyone can recommend a project with examples of of functional tests and reports?

[maiden pawn]

Modern tools allow to run tests locally and CI pipeline automatically for every commit in both choices with fully fledged databases instances of Postgresql / MySQL and etc

If u write for MySQL u should run your tests in MySQL.
I can give u code example how i do it at least for PostgreSQL
u need to know Docker/Docker-compose for that
they have already available MySQL to spin up in few code lines https://hub.docker.com/_/mysql

I have file in my repository like
docker-compose.yml

version: '3.5'

services:
  app:
    build: .
    environment:
      TRAINING_DATABASE_USERNAME: "postgres"
      TRAINING_DATABASE_PASSWORD: "postgres"
      TRAINING_DATABASE_HOST: "db"
      TRAINING_LOGGER_CONSOLE_LEVEL: "DEBUG"
    depends_on:
      - db
    command: sh
    volumes:
      - ".:/code"
    working_dir: /code
  db:
    container_name: postgres_container
    image: postgres
    environment:
      POSTGRES_USER: ${POSTGRES_USER:-postgres}
      POSTGRES_PASSWORD: ${POSTGRES_PASSWORD:-postgres}
      PGDATA: /data/postgres

and Dockerfile

FROM python:3.10

COPY ./requirements.txt ./
RUN pip install -r requirements.txt

my tests are equally the same run locally and in CI with command like..
docker-compose build -- app && docker-compose run -v $(pwd):/code -u 0 --rm app pytest ; docker-compose down

it is automatically building application image, spin ups app+database, runs pytest tests in it, and then removes its stuff after it is done

[safe arch]

(@maiden pawn , these are excellent examples, thank you.)
I use docker-compose and it's not for everyone.
If you want to keep it all in python there are testcontainers
https://github.com/testcontainers/testcontainers-python/blob/master/testcontainers/mysql.py

[maiden pawn]

and yes as u can see i supply values through env vars

[maiden pawn]

Gitlab CI gives same at its syntax level for people who aren't ready for Docker-compose yet
https://docs.gitlab.com/ee/ci/services/

test:2.6:
  image: ruby:2.6
  services:
    - postgres:11.7
  script:
    - bundle exec rake spec

But i just run with docker-compose instead of using Gitlab Services in CI too. Because it reuses local code to CI code. more DRY

[low oyster]

oh yeah, the how isn't an issue - the principle is

in the CI pipeline in 'Test' stage I can:

1. Spin up db
2. pytest functional_tests.py -rA

[maiden pawn]

https://12factor.net/
https://12factor.net/dev-prod-parity
principle X dev/prod parity, that is what we follow

[maiden pawn]

same u can do locally. I am used to develop stuff locally with docker-compose too

[low oyster]

Got it! I'll refactor the tests accordingly!

[maiden pawn]

great. u will receive command like that (or simpler, that depends on your taste) to run your tests locally in MySQL
docker-compose build -- app && docker-compose run -v $(pwd):/code -u 0 --rm app pytest functional_tests.py -rA ; docker-compose down

[low oyster]

I have another question wrt database testing @maiden pawn - I have a DBRepository module which initializes a database (mysql), and executes queries in different functions like execute_user_query, execute_fruits_query

While unit testing this module, I spun up a mock SQL db and checked if this module can connect to a db (sqlite) and execute some sample statements. However, I had to mock return values of execute_user_query because SQLite & MySQL syntax are different

^ Does this testing make sense? i.e. is it acceptable?

[low oyster]

thank you! Although we don't use docker compose yet, our infra/platform team can help us figure something out in the meantime 🙂 (Although I like your style)

[maiden pawn]

^ Does this testing make sense?
if u will read https://12factor.net/dev-prod-parity article, u will discovery that it is exactly against modern principles of web infrastructure
With usage of docker-compose for local dev env, your tests and code can be present only for MySQL, everything related to SQLite should be deleted if it is not used in prod

[low oyster]

even for unit testing?

[maiden pawn]

I will add here in addition that your Prod code should not be having anything Non prod.

[maiden pawn]

well, that's a bit longer to explain topic. let me try

[low oyster]

sqlite is only for unit testing, it's not even a part of the main application but a stub for an actual db while unit testing

class MockDB:
  def __init__():
    # code to just spin up a sqlite db

def setup():
  mock_db = MockDB()
  db_repository = DBRepository()
  db_repository.engine = mock_db.engine

this is what I'm doing

[maiden pawn]

yes, even for unit testing u need to use spinned up MySQL for your code runs. If your code architecture assumes need to use database engine for its run, then your Prod database like MySQL spinned up locally in container should be executing it.

it makes sense to mock requests only for the purpose of running tests faster for tests where this SQL request performed will not check any of your code. When u already checked code interacting with database. For further understanding Unit testing best principles and practices by Vladimir Khorikov + Clean Architecture by Robert Martin are required

Anything that u can spin up locally with Docker = still your unit tests, where nothing should be mocked (by directly returning values, not by replacing database engine!) for the purpose of running some test group faster

real integration tests check integration with uncontrolled third party dependencies outside of your application code / outside of things that u can spin up locally

[maiden pawn]

it is bad replacing MySQL with SQLite for tests, because u decrease tests reliability. SQLite and MySQL are too much different
Your app code can be not correct, but SQLite will say it is correct (for SQL constraints for example)
Do not mock DB with SQLite if SQLite is not used in Prod

than more your app will grow in complexity then faster your tests will become unreliable

[low oyster]

Yeah I agree that they're both very different, but i'm trying to unit test small bits of the application

so perhaps I do not unit test those functions which specifically need mysql db

[maiden pawn]

you approach it wrong then. Mocking is done in different way to test small bits of the application.

let me find the picture then

this is different goal

[low oyster]

actually, let me supply some template code

[maiden pawn]

are you familiar with basic OOP, that u can make abstract classes and implementing different versions of them?

u should be, since u are already using mocking

[low oyster]

class DBRepository:
  def __init__(config):
    self.engine = create_engine(config)
  
  def _execute_query(stmt, params):
    return self.engine.execute(stmt, params)
    
  def get_user_info(user_id):
    q = "select * from user_info where id = %(id)s"
    result = self._execute_query(q, user_id)
    return result

So, when I try to unit test this -- I can't exactly "unit test" get_user_info() unless I mock _execute_query()

[maiden pawn]

Okay, here picture 8.2 from Clean Architecture: A Craftman's Guide to Software Structure And Design by Robert Martin.

Testing code in pieces means... u first write tests for isolated code of your class implementations of abstract classes for databases first.
Once u tested it, u don't need using code that uses all of the Database code!

U can write mock requests that input some dataclasses, and output another dataclasses. U don't need then any MySQL or SQLite after that
The rest of your code can be just using returned mocks of already provided data from database

in order to understand this stuff better, better to read the book