unit-testing

[hexed cloak]

You could use capsys to check the print out

[covert heron]

The comment is worng, should be "move" of course. The second function is for copying

[covert heron]

Thank you for the idea, I have added it to the test.

[lilac ferry]

Hi everyone! I've studied how to make mocks for testing an internal API I coded for a personal project and I want to share these two articles that were really helpful for doing that

https://codefather.tech/blog/python-mock-and-patch/
https://www.pythontutorial.net/python-unit-testing/python-mock-requests/

[safe arch]

So, you know how some people are good at writing poems, or cross word puzzles?
Do you know anyone with a knack for "writing the failing test"?

[hidden marten]

When doing regression testing (after a bug report or exception) I often try to think about reducing the bug to its smallest reproduction scenario, and write a test that uses similar inputs or conditions and fails with the existing code.
For me, the knack develops over time, and still isn't perfect, and sometimes even needs some test suite tinkering to get it "just right".
What kind of failing test are you thinking about?

[safe arch]

I guess I'm thinking of a scenario where I want to write a app and I want to set up some milestones, such as:

• user can request a home page
• user can post a response
• user can login
• user can CRUD a database record
  I imagine that a testing bard can articulate the context and assertion for each scenario.
  Instead, I feel clumsy, like a kid trying to tell his right shoe from the left.
  Maybe it's practice, but maybe there's a method (like "start with the error message...")

I'm also thinking of TDD, where we write the test describing what we want to see happen next.

[hidden marten]

Thanks for sharing, that helps contextualize.

For stuff like this, I might try to use a browser-like test utility that describes the behaviors you wrote down. Many web frameworks have a testing utility to start the web app and run a test, and then ssert a condition

Some may call this "behavior driven development" and there are tons of blog posts, articles and more on that specific approach

[proper wind]

Hello

I'm creating an API wrapper and I'm currently putting together unit tests.

I'm mocking the API with pytest and requests-mock but I need to mock the JSON response of the API. However the JSON file/dictionary is rather large so it seems unyieldy to include it in the pytest fixture/function.

Is is best to include it in a seperate file/directory?

Is this common practice?

[hidden marten]

I've used responses before to record and use external files. https://pypi.org/project/responses/

I've also used raw files in a tests/fixtures directory to keep the testing code clear from the expected responses

[proper wind]

Thank you! So the expected responses sit in the tests/fixtures directory?

[hidden marten]

That's how I set one project to work, yes. If the tool I'm using has default behavior, then I tend to use that, but most things like paths are customizable

[proper wind]

Thank you!

close!

[hidden marten]

That's for the #1035199133436354600 channel system 😉

[proper wind]

oh gosh I got confused which channel I was in haha

If I might get your advice once again... I want to unit testing that my API wrapper creates a custom class from the JSON response from an API call. So let's say I have a Book class that has 10 attributes. Some of the attributes will be None because the book metadata might be missing/blank.

I'm just unsure if I should mock 10x different JSON responses, each with a different attribute being blank, and unit test that the initialisation of the Book class succesfully returns None for each attribute that has missing metadata. This seems like overkill but what do you think?

[hidden marten]

Hmm that does sound a bit like overkill.
<puts on testing purism hat> If you are looking to ensure that your code can handle different API responses and can successfully instantiate the Book class , then exercising every possible combination would be exhaustive, but confirms that if an API response of a specific share is returned, then the code behaves the desired way.

<takes off purism, puts on pragmatic hat>
Often we don't need to test that Python behaves the right way - for example if Book only requires title attribute, and all other are optional, a non-API test creates an object and asserts it is sufficient for testing class creation.
For example, b = Book(title="Good Omens"); assert b.title == "Good Omens" would ensure that the class can be created with none of the other attributes.

A lot depends on what your code does to parse out the API response and construct the class - sometimes API responses of empty strings instead of null can behave a little differently than expected.

[midnight ibex]

Hmm

[proper wind]

Thanks so much! I appreciate it.

[dense badge]

hi everyone. i just got assigned to my first unit testing job using chai mocha and the given js file is 75% covered and my job is to get it to 100%. i skimmed through it but I don't understand how to get started. i have watched tutorials but I still don't get what the procedure is and how to go about it. can anyone help

[hidden marten]

Congratulations on the new assignment!
I don't know how much help you'll find on the Python Discord server for testing approaches for JavaScript, maybe there are other communities that are better focused on that language and approach?

[dense badge]

okay I'll look into it and thanks

[lusty horizon]

Can I ask questions about pytest here, please?

[proud nebula]

yes

[lusty horizon]

I am using currently pytest, ipytest with

%%ipytest -vv --tb=long -s --showlocals -rx

in a jupyter notebook cell.
Depsite having -s
I don't see the print of

print('hello world') 

in one of my tests.

Okay, I found the statement of 'hello world' Sorry. it was hidden beside the test_id (a t followed by a long number) . Sorry

[hidden marten]

no worries! Glad you solved it

[lusty horizon]

I will see if I can get the content of the variable now.

Yes, this is also there. I just don't look correctly.

[hidden marten]

It can take a few times to get used to reading output of a new tool or process

[lusty horizon]

Thank you that I could still ask here and without being downvoted

I admit, the function is very crowded, lots of print outs in the standard version without pytest. I am currently developing.

[lusty horizon]

Now I have a more sophisticated question. I have a sorting algorithm to describe it in a shortest way. I make up some values, I put in the test function the results.
I let the function work on the incoming values, I compare the output with the expected values

For one case it worked. Now I would need to repeat this. The main developer pointed out pytest fixtures, but I have no way to deduct the sorting. Otherwise I would have done this already in the function to test

How do the experts tackle such a situation?

[proud nebula]

Seems like more of a parametrized tests situation. Maybe mutmut and hypothesis might also apply.

[lusty horizon]

Thank you. But how would I know the outcome for the result? I sort values according to some categories and average them.

[proud nebula]

I don't understand the question. How can you not know? Maybe show some code.

[lusty horizon]

My x-axis value follow sin2chi function. Present values are grouped, here visible by color. For each group exists 1-4 other values, I need to sort them according to the colored groups and average

My chi can start at -180deg or -90 or -0 or could have missing values

My function, which I want to test, i responsible for the grouping and the averaging.

[proud nebula]

you forgot a question

[lusty horizon]

Your question was, how could I not know? How could I not know the outcome? Correct?
I only know the outcome of the function, if I let run it through for a given set of input parameters.
If I write a test, I have to provide known input params and known output parms, correct?

I would not touch intermediate results of the function, because technically they are not accessible.

How would a "parametrized tests situation" look like for a more complicated case? I expect two np.arrays in, 3 np. arrays out, and two of them correctly averaged along dimension 1

[teal quartz]

I have a whole bunch of test only packages. The tests in there, are executed in a venv and found by pytest by passing pytest --pyargs <packagename>.
So far so good. I have a plugin in conftest.py provided by each of the test packages. The code in these conftest.py's is always the same. The goal of the plugin is to gather all test results to make them available for the creation of a Word report. The tests, or bunch of tests 'know' what the content should be of the test reports, but need to include the results from the plugin.
My goal is to move the generic conftest.py to top level (outside of the test packages) so that i have it only once in the source tree. Now the question is, how can the collection of tests register a callback to the generic conftest.py so that it will call the create-report function during pytest_sessionfinish in the conftest.

Am i making any sense at all ?

If i declare a fixture in the conftest, then for that fixture to be created, i would need to have at least a test that uses it.
Its not very nice to create a bogus test to have it create fixture. Is there no way to create it without the bogus test ?

[proud nebula]

I don't understand the question... You have a bunch of inputs and you declare in the test what you think the output should be, then you assert that the real output is what you expected. Like all tests....

[proud nebula]

There is the autouse flag

[lusty horizon]

@boxed Right. I was under the same impression. Until yesterday, the main developer came along and said, I can access intermediate results in the function .
Would be parameterized testing not something like: x, y in , out f(x,y), but you know exactly the function ?

But I don't think it is correct what the main developer told me. I provide defined input params , output params, and I do assert output_params expected == output params

Is this correct, please?

[proud nebula]

the main developer of what?

[lusty horizon]

Of the project I am working for

No worries, no big Python thing

[proud nebula]

oh, he probably means you can call some other function to get the intermediate data or something then

You should ask him, not us.

[lusty horizon]

🙂 I wrote tests for the other functions this function is using.

[proud nebula]

"intermediate data" is a subjective thing anyway. If the tests check it, then it't not intermediate from the point of view of the test.

[lusty horizon]

But in general, how can I get intermediate data? A function is a closed entity when running?

[proud nebula]

I mean... there's spys and stuff, but better not.

[lusty horizon]

This is too advanced for me currently 🙂

[proud nebula]

It's also a bad idea :P

[lusty horizon]

I never heard of spys for testing

[proud nebula]

Yea, people just say "mocks" for spys, mocks and fakes... it's a mess

But the first rule of mocking is to not use mocking.

[lusty horizon]

Ok. I think my approach is then correct, even if I write code for each case. (Parameterization is a bit tricky)

[proud nebula]

sure

[lusty horizon]

I don't know, but it sounds like you think I am not completely wrong

I have nobody to ask here, hence I am looking on Discord for exchange

[proud nebula]

What does that mean? Why can't you ask?

[teal quartz]

Thanks.... thats going in the right direction but its still not very nice or working...
Im new to the fixtures and plugins so bear with me...
This is what i came up with

https://pastebin.com/yUVLDm7X

I tried chaining the two fixtures 🙂

But i probably missed your point entirely 🙂

[flint kelp]

I made a little pip package to test a random sample of your parameterised arguments in pytest - https://pypi.org/project/pytest-sample-argvalues/

What do people think? Open to suggestions/criticism 🙂

[lusty horizon]

We are only two people here and that person declared I am too bad in week 1, followed by more dissing.

[proud nebula]

Hah. What? Is this some school thing?

[lusty horizon]

A research institute and no, I am not going to school

[proud nebula]

Academia. Close enough. Time to go look for a job in a company.

Or at least a different university.

[lusty horizon]

Yes, exactly my thoughts. 🙂 In the meantime, I am happy to learn from the kind people on Python discord

[teal quartz]

this works: https://pastebin.com/QrQ3pprz

Open to suggestions as to make it more compact, more pythonic

or options to make it more elegant

🙂

[proud nebula]

More pythonic: classes should be PascalCase, not snake_case.

doc_generator seems not useful. Can't you just replace the global docgen = None with docgen = doc_generator_cb_mgr() and delete the doc_generator fixture?

Actually.. I don't see the point of any of this code.

[teal quartz]

and after reading that ?

[proud nebula]

Not really. It sounds like an xy problem. What are you really trying to do?

[teal quartz]

Ok, say pytest runs a bunch of tests. The tests themselves dont know the test results; the code in the python file that contains the tests does know how to setup a readble report for the tests.
There is a plugin in conftest.py that keeps track of all the results of the individual tests, these results need to be available on the moment the report is created, to be used in the report.
So the reporting function needs to be in test_something.py together with the tests, and the code that keeps track of the results should be in conftest.py so i only have it once in the repo instead of in each package that contains a similiar test_something.py.

[proud nebula]

Several issues:

1. you didn't answer my question
2. why are you trying to rebuild pytest but worse inside pytest?

The tests themselves dont know the test results

this in particular makes no sense

[river pilot]

can you say more about the report you are creating? How is it different than the pass/fail results that pytest will give you?

[teal quartz]

i think 1 is answered, im trying to split responsibilites and avoid code duplications while still having a way to create reports that the test code knows how to create
In more abstract sense, im building an testautomation framework to execute integration tests on hardware and software.

2, not sure that i am.

The tests are not stupid unit tests, they are tests that run measurements on systems, then aquire the data, then analyse the data, then decide if all is ok, but use pytest as backend.
The report will contain charts and mathplot lib whatever, and create a complete Word file with all results.

[proud nebula]

I think we need to see more of your code.. maybe a single example of a test.

[river pilot]

sounds intricate. I might have lost track of the original question. This doesn't sound like a job for fixtures, but for pytest plugins.

[proud nebula]

or a helper function

[teal quartz]

hence the conftest, that is where you have the plugins right ?

[river pilot]

yes, you can add pytest plugins in conftest

[teal quartz]

i wil try to create pseudo code to illustrate what happens

[proud nebula]

You really should try to show the real code

99% of people showing roughly the code, they remove all that is important

[teal quartz]

I've give it some thought.... the generation of word report containing the simple results like test name, and result similar to the xml results file etc, can entirely be managed from the conftest.
The fact that a bunch of tests test something that should result in a report, can be done in the test_something file itself, next to the test_ functions... Then indeed it can be a simple helper function, and there is no need make it complicated.

then still things are nicely decoupled

thanks for bearing with me, sometimes, its needed to have people say your an idiot without saying the actual words 🙂

[river pilot]

tbh, i would generate the word report after pytest finishes, by consuming one of its output formats. There's no need to do it as part of the pytest run.

[teal quartz]

Problem with that is we

• create clean venv
• install test_infrastructure packages
• install test_package
• call pytest --pyargs test_package
  Thats it.

[river pilot]

are you saying the problem is that you don't have a chance to run a command after pytest?

[teal quartz]

Not in a generic way, the system that executes the tests, just installs test package, and calls pytest and is done.
Has no clue on what its executing.

thanks for all help!

[proud nebula]

Sounded like he wants more data than can be put into the result files. Like images.

[river pilot]

idk if you are done talking about this. If you aren't done: whatever runs "install, pytest" can run "install, pytest, report-generator". I think it's common to think that everything should be jammed into pytest, and it's just not necessary.

[river pilot]

fine, write data some place, then generate the report later.

[teal quartz]

but then after all tests are executed you loose the knowledge on what is stored where; because if you really start a command after pytest was executed, you only have the xml files to parse. Then all files generated in during test execution are still there, but you dont know what is what. Some of the tests generate dozens of files all with date/time in the filename etc. So .... its complicated.. 🙂

[river pilot]

yes, i understand it's complicated. My approach would be to not put all the complexity into one process inside pytest. Write raw data somewhere (from inside pytest), then use it to write the report.

[teal quartz]

ack

thanks

[proud nebula]

I believe you can get the test name from inside pytest too. That will help in creating a nice folder structure.

[river pilot]

I use this inside my test suite: test_name = os.getenv("PYTEST_CURRENT_TEST")

also for xdist: WORKER = os.getenv("PYTEST_XDIST_WORKER", "none")

[bitter wadi]

:ok_hand: applied timeout to @noble oyster until <t:1717595715:f> (10 minutes) (reason: newlines spam - sent 110 newlines).

The <@&831776746206265384> have been alerted for review.

[little shadow]

!unmute 917708906795499550

[bitter wadi]

:x: failed to pardon infraction timeout for @noble oyster. User was not found in the guild.

[little shadow]

Interesting

[hexed cloak]

Any reason for using getenv over os.environ?

[proud nebula]

hm.. why does getenv exist at all? seems like a silly function

[hidden marten]

I suggest you get a time machine and travel back 25 years and tell Guido 😆

Also add getenv() as a familiar alias for environ.get().
https://github.com/python/cpython/commit/5a2ca9328df3f73fcf0b38d488972ef139cc73e1

[hexed cloak]

why not a sys.getmod alias for sys.modules?

[proud nebula]

familiar... if you are a C++ developer :/

Now that Python is the largest language, C++s getenv will be familiar because people know it from Python 🤣

[hidden marten]

ouroboros

[river pilot]

no idea. i've never understood the difference.

[hexed cloak]

It's just a call to os.environ.get

[river pilot]

coverage.py is full of old usages. I'll mention os.path, but will not mention the more surprising one.

it's using ||optparse||

[hexed cloak]

Oh no

[river pilot]

it's not a problem, is it?

[hexed cloak]

Deprecated since version 3.2

One of those immortal deprecations

[river pilot]

right, but what does deprecation actually mean here?

[hidden marten]

I can't help but wonder if this is one of the more widely used examples keeping it around? https://peps.python.org/pep-0594/#optparse

[river pilot]

i have no idea. i didn't realize it was mentioned in that pep

I assume widely used means used by many projects, not, used by widely used projects. coverage.py is only one project.

[hidden marten]

True

[river pilot]

and if the core devs said they were yanking optparse from the stdlib, i'd switch to argparse. But there's no reason to change the code now.

[hidden marten]

But, but, those precious DeprecationWarnings!! 😜

[hexed cloak]

optparse doesn't actually have any warnings

[proud nebula]

Deprecated with no deprecation warning... that's never going away then :P

[hidden marten]

I think there was some controversy about adding the warning before internal stdlib usages were replaced, so as to minimize confusion to users of the CLI tools that ship with cpython.

[proud nebula]

That seems reasonable.

[gentle edge]

Any idea on why this isn't working? I import the method in question as from urllib.request import urlopen

Running on Python 3.8, if it matters

[river pilot]

you have to mock where the function is used, not where it is defined: https://nedbatchelder.com/blog/201908/why_your_mock_doesnt_work.html

[gentle edge]

That gives me AttributeError: module 'brutils.cep' has no attribute 'urllib'

[river pilot]

can you show me the line in brutils/cep.py that imports urllib?

(oh, you said it before)

you want @patch("brutils.cep.urlopen")

[gentle edge]

from urllib.requests import urlopen, I've tried to change the mock to @patch("brutils.cep.urlopen", MagicMock) but then it's saying that it's not passing the mocks to the tests :/

[river pilot]

can you show the full message you get?

[gentle edge]

TypeError: TestCEPAPICalls.test_get_cep_information_from_address_success() missing 1 required positional argument: 'mock_urlopen'

[river pilot]

I'm not sure how unittest handles patching on the class.

[gentle edge]

Well, if I remove the MagicMock arg from the patch decorator it works

Thanks so much Ned! 🙂

[bright owl]

Have anyone one of you tried running selenium testing on github codespace?

Im having this issue with finding chrome driver

[hidden coyote]

Hi, Iam new to unittest, could anyone please share resources i can use to learn testing and get to perfection in it? Thanks in advance!

[tall brook]

• https://en.wikipedia.org/wiki/Perfect_is_the_enemy_of_good
• https://www.obeythetestinggoat.com/
  you can read both things online for free

[tall brook]

🆙

[maiden pawn]

https://darklab8.github.io/blog/favourite.html#TestDrivenDevelopmentByExample

This book teaches practice behind unit testing. On a specific example it walks you through how to have unit testing as part of your development cycle at every moment. It teaches you feeling how much gap between tests is allowed in your working code.
Learn also using visual debug from within unit tests with your IDE, with easy ability to run arbitary shell commands at breakpoint time
With it combined u will be able to get used to developing with unit tests first

https://darklab8.github.io/blog/favourite.html#UnitTestingPrinciplesPracticesandPatterns
Read this one in addition for theory and better understanding what they are and what to aim for

To get a real perfection, u will be needing to learn (for python at least also Mypy on strict configuration + typing docs https://docs.python.org/3/library/typing.html )
Because it allows with more freedom to rearrange code structure, and therefore iteratively improving code easier depending on your needs
https://darklab8.github.io/blog/favourite.html#CleanArchitectureACraftsmansGuide
And also code architecture in order to understand how to change your code structure to fit testing better
U need to adopt thinking, how to design solution to be testable first during architecture design
Then it will be perfect. As u will be able to command your code to remain testable, despite influence of different "frameworks" which prevent you from testing code properly (Like Discord.py is the most awful thing i ever saw in those terms for example. Screwing up code a lot and preventing any testability)

[teal quartz]

Any BDD experts here ? Should i start with behave or with pytest-bdd ?

[proud nebula]

My personal recommendation is to not do BDD at all. It's just making your life difficult.

[teal quartz]

can you elaborate ?

[proud nebula]

BDD takes simple unit tests and inserts a layer of regexes in the middle of it, to get a report that no one reads.

[river pilot]

what's drawing you to BDD?

[torpid osprey]

I'm just learning how to unit test and I would like to know if there's a way to run a script before running any of the tests such as establishing the database and overriding dependencies.

[tall brook]

yes, https://docs.python.org/3/library/unittest.html#setupclass-and-teardownclass

[torpid osprey]

How about with pytest?

[tall brook]

yes too, https://docs.pytest.org/en/8.2.x/how-to/fixtures.html

I usually avoid accessing external resources like APIs, DB, etc. I create mock objects for that (monkeypatch for pytest)

[torpid osprey]

Are the fixtures initilized only when called by a test?

[tall brook]

since most of the things I do is using containers, I disable network access to ensure nothing comes out 😄

[tall brook]

yes

[torpid osprey]

Is there a way to run a file that can initialize a dependency for all the tests?

[tall brook]

hm... I'd create a dump of the db, ideally SQLAlchemy statements for easier management and have an isolated script to reset things, perhaps using a fresh container. But again, if I were you I'll try my best to avoid using the actual db and monkeypatch the thing. I've done that for redis, postgresql and mongo. If you run tests in CI/CD having external components might break your flow

it's not as hard as it sounds e.g. docker run --rm -it --network none -v $(pwd)/src:/src python3.12 pytest /src

and for mocking - https://docs.pytest.org/en/8.2.x/how-to/monkeypatch.html

the only thing I haven´t been able to do using monkeypatch is changing the mocked object behavior e.g. multiple requests to different URLs done from same module. For that I have to use sideeffect from unittest 😢

[river pilot]

what do you mean by a dependency?

[torpid osprey]

I was reading the official FastAPI tutorial on testing and it says to create a external connection to a database.

[torpid osprey]

FastAPI dependency

[river pilot]

sorry, i don't know what that means? You need it installed, or you need a fast api server running, or?

[torpid osprey]

For testing, FastAPI allows you to override a function that does something such as making a request to a database such as:

async def override_get_db():
    db = TestSessionLocal()
    try:
        yield db
    finally:
        await db.close()

app.dependency_overrides[get_db] = override_get_db```

[river pilot]

that looks like something you would do in a fixture.

[torpid osprey]

Okay. I maybe should take some time and read the pytest documentation.

I appreciate all of ya'll's help.

[tall brook]

can you show me where? I can´t find that in the docs

[torpid osprey]

https://fastapi.tiangolo.com/advanced/testing-database/

[torpid osprey]

ah so you run the test inside of the container?

[tall brook]

but they create a new db for that. Why do you want to do that before running the tests?

[torpid osprey]

idk tbh

I think I need to take some time to learn the process.

[tall brook]

sounds like a plan 🙂

[torpid osprey]

I thank you for your help. Very useful information.

[weary quarry]

You generally don't want your tests to have access to an existing database, such as your production database. So you create a new one for tests. This also gives you a clean and isolated testing environment to work from.

[torpid osprey]

Like an in memory database?

[formal halo]

no more of a test database and populate it with minimal data from csvs etc and reset this db everytime tests run and repopulate

[torpid osprey]

Okay thanks for clarifying.

[split field]

If the database you're using allows an in-memory mode, this can be a good idea that simplifies testing, but most databases don't have such a mode, and it's not a good idea to run your tests with a different kind of database than your real application uses

[mossy sky]

we use mongo in our production setup and mongomock is awesome.. its kinda an in memory database thats mongo compliant and resettable on a dime

[split field]

That's cool. sqlite has an in memory mode, too. But most databases out there don't have a mode like that, and having your test suite use a different db than your real app can mean that you miss bugs where something your app does works with one db but not the other

[torpid osprey]

@tall brook if I'm testing an API, do I just make external requests to the API to test it?

[river pilot]

external in what sense? You are testing your own API, is that right?

[torpid osprey]

Yes

Like make requests to my own API

[river pilot]

ok, what do you mean by external?

what kind of API are we talking about?

[torpid osprey]

FastAPI backend for my discord clone.

[river pilot]

i would test it by calling the handler functions directly

[torpid osprey]

Like mimic the functions?

[river pilot]

no, call them. Your FastAPI backend has functions that get URLs routed to them, right?

[torpid osprey]

yes

[river pilot]

ok, call those functions. Test that they do what you want them to.

[torpid osprey]

Ah ok. Will do.

[hexed cloak]

You can use https://github.com/florimondmanca/asgi-lifespan with httpx to make requests directly to an app

Sometimes it's needed when dealing with complicated dependencies like file uploads

You can use @pytest.mark.anyio to run async def tests

[bright owl]

Hi does anyone know anything about selenium webdriver
The problem im facing is im running a script which is opening chrome gui but i dont want to see the gui just see the output in terminal

[hexed cloak]

You want headless mode?

options = ChromeOptions()
options.add_argument("--headless=new")
driver = webdriver.Chrome(options=options)

[bright owl]

thank you sooo much
That was exactly how i was expecting

[shadow ridge]

aws lambda layer for playwrigght python?

[proud nebula]

that made no sense

[shadow ridge]

I need to run python playwright code on AWS lambda. Facing issues with the dependancies. Tried creating the lambda layer with playwright and also docker image but none of them worked.

Getting error like this:-
{
"errorMessage": "'PlaywrightContextManager' object has no attribute '_playwright'",
"errorType": "AttributeError",
"requestId": "857b8224-18c9-44c9-9053-465a8267d3c5",
"stackTrace": [
" File "/var/task/lambda_function.py", line 4, in lambda_handler\n with sync_playwright() as p:\n",
" File "/opt/python/lib/python3.10/site-packages/playwright/sync_api/_context_manager.py", line 79, in enter\n playwright = self._playwright\n"
]
}

[torpid osprey]

from app.models.user import Users
from aiohttp import ClientSession

async def test_login(session: ClientSession, current_user: Users):
    body = {"username":current_user.username,"password":current_user.password}
    response = await session.get("https://www.google.com/")
    assert response.status == 200``` This is my code

Whenever I'm making a test that sends an API request I get:

ERROR tests/test_login.py::test_login - ConnectionRefusedError: [WinError 1225] The remote computer refused the network connection

[river pilot]

are you actually trying to connect to google.com in your test?

if so, why?

[torpid osprey]

No. It was just to test if any API request works but I'm getting the same error.

So it's a problem with making API requests in general.

[river pilot]

sounds like something about your test environment, but I would also try to avoid making any network connections in tests.

[torpid osprey]

I hear what you're saying but in the testing section of the FastAPI docs, they make API requests.

[river pilot]

ok

[hexed cloak]

You should be using httpx.AsyncClient with the asgi app transport

https://www.python-httpx.org/advanced/transports/#asgi-transport

[lament heath]

@river pilot when testing FastAPI, they are only making "fake" network calls that call the methods of the ASGI app directly. I.e. It's not expensive

[river pilot]

it looks like it's making real network requests.

[lament heath]

Oh, I'm sorry. The author is using it incorrectly, you are right. The goal is to only make requests to your own resources and only directly — through the ASGI app, not through network

[torpid osprey]

will do

[torpid osprey]

Is there a way of running a startup script that creates the tables at the start of the test?

https://paste.pythondiscord.com/6QNA

[weary quarry]

One way is to run the needed steps in the fixture creating the database object. https://github.com/Preocts/flask_playground/blob/cbcac4d3d4929e5dba67dd120fb6fc552d079f32/tests/pizzastore_test.py#L14-L26

[bitter wadi]

tests/pizzastore_test.py lines 14 to 26

@pytest.fixture()
def store(tmpdir) -> Generator[PizzaStore, None, None]:
    tempfile = tmpdir.join("test.db")
    store = PizzaStore(tempfile)
    store.connect()

    try:
        store._build_table()
        store.disconnect()
        yield store

    finally:
        os.remove(tempfile)```

[weary quarry]

In my case the build_table method exists as part of the database object but that could easily be a separate module.

[torpid osprey]

Is it true that every time you run the fixture, the database will be created?

[weary quarry]

For the fixture I showed, yes. You can control that behavior with the @pytest.fixture decorator.

[torpid osprey]

How do you make it so a fixture runs without being called?

I'm new to unit testing

[weary quarry]

https://docs.pytest.org/en/7.1.x/how-to/fixtures.html#autouse-fixtures-fixtures-you-don-t-have-to-request
@pytest.fixture(autouse=True)

[torpid osprey]

Doesn't another fixture have to request it?

[weary quarry]

No. It is auto-used.

[torpid osprey]

What setting would you put the fixture on to make it execute one time?

[weary quarry]

For example, this fixture is always used on all tests. It masks the os.environ and prevents my tests from grabbing live credentials
https://github.com/Preocts/secretbox/blob/main/tests/conftest.py#L61-L67

[bitter wadi]

tests/conftest.py lines 61 to 67

@pytest.fixture(autouse=True)
def mask_aws_creds() -> Generator[None, None, None]:
    """Mask local AWS creds to avoid calling out to AWS"""
    with patch.dict(os.environ):
        for key in AWS_ENV_KEYS:
            os.environ[key] = "masked"
        yield None```

[weary quarry]

((wow this code is old, I know a way better way to do that now))

[torpid osprey]

Ah I get it.

[weary quarry]

You'd be defining the scope of the fixture.
https://docs.pytest.org/en/7.1.x/how-to/fixtures.html#scope-sharing-fixtures-across-classes-modules-packages-or-session

The easy breakdown:

Fixtures are created when first requested by a test, and are destroyed based on their scope:
function: the default scope, the fixture is destroyed at the end of the test.
class: the fixture is destroyed during teardown of the last test in the class.
module: the fixture is destroyed during teardown of the last test in the module.
package: the fixture is destroyed during teardown of the last test in the package.
session: the fixture is destroyed at the end of the test session

[torpid osprey]

I think what caught me off guard is that someone told me the default scope was the session one.

[weary quarry]

They were incorrect. The default is function. pytest is designed to assist with removing possible test pollution.

[torpid osprey]

There's a bit of a difference.

But yeah that explains a lot.

[weary quarry]

There is. session level scope would likely break 90%+ of every test I've ever written that uses fixtures.

[torpid osprey]

Aight thanks for clarifying everything.

[maiden pawn]

Set scope session feature

[torpid osprey]

Yep fixed it.

Someone told me that was the default scope before which threw me off.

[upbeat cairn]

https://hachyderm.io/@coveragepy/112609233347254476 very hype

[river pilot]

i am old: is hype a good thing here? 😄

[upbeat cairn]

yeah

[maiden pawn]

Hype means super popular. Usually means smth positive but does not have to 😅

or at least => It is perceived as smth positive by most of population

that will be precise

LLM, NFT, Web3, blockchains, are all hypes :/

[river pilot]

hmmm, not sure i want my work compared to all of those things.... i think most of them are dumb 🙂

[maiden pawn]

Most hypes are dumb.

[river pilot]

then how should i feel about my work being called hype?

[maiden pawn]

Ergh... in this particular case? Good i guess? Every dev wishes his work to become popular and recognizable.
In this case hype is good i guess. That means u met demands of users

[river pilot]

ok, cool

[maiden pawn]

yeah.. the most close translation will be: It makes people so excited, that they can't control emotions in anticipation about it or its usage and etc. So they share this excitement with others.
Very cool to have

[torpid osprey]

Is there a way that it can test websockets?

[plush heart]

Hello, anyone knows how to fix issues with mocks?

I have this config.py file:

import os

from dotenv import load_dotenv

load_dotenv()


class Config:
    class SUPABASE:
        KEY = os.getenv("SUPABASE_KEY")
        URL = os.getenv("SUPABASE_URL")

This is my test_config.py file:

from src.config import Config


class TestConfig:
    class TestSupabase:
        def test_supabase(self) -> None:
            assert Config.SUPABASE.KEY == "SUPABASE_KEY"
            assert Config.SUPABASE.URL == "SUPABASE_URL"

The problem with this approach is that it is trying to get the real .env variables; however i want to mock them; i have tried several approaches but none worked, anyone has any idea? Thanks!

[jaunty ocean]

o properly mock the environment variables in your tests, you can use the unittest.mock module to temporarily replace the os.getenv function during the test. This way, you can control the return values of os.getenv without having to actually modify your .env file.

[weary quarry]

unittest.mock.patch is another option. You can patch out the os.environ for the test. I usually use an autouse fixture so it is applied to all tests without needing to remember it. This also ensures the environment is restored to its original state during the run.

You do have to be aware of when the environment gets filled with what you want to remove. If you use this fixture in a conftest.py and the module imported in a test loads the environment, it won't be cleared. conftest is processed before the imports of a test module.

import os
from unittest.mock import patch

import pytest

@pytest.fixture(autouse=True)
def wipe_environ():
  mock_environ = {} # optional mock values can be added
  with patch.dict(os.environ, mock_environ, clear=True):
    yield None

[split field]

your problem here is that you're calling load_dotenv() when the module is first imported, before pytest has had a chance to mock anything. The best fix for that would be changing things so that the environment variables aren't looked up until the first time the key or URL is actually needed. An alternative would be using a pytest_configure() hook to patch the environment or replace load_dotenv() before the module gets imported

[split field]

I'd suggest pytest's monkeypatch fixture for setting or deleting environment variables. a la https://github.com/bloomberg/memray/blob/9617227fb0bffa008baa4eaa47c22d3001d3fd26/tests/unit/conftest.py#L4-L10

[bitter wadi]

tests/unit/conftest.py lines 4 to 10

@pytest.fixture(autouse=True)
def use_80_columns(monkeypatch):
    """Override the COLUMNS environment variable to 80.

    This matches the assumed terminal width that is hardcoded in the tests.
    """
    monkeypatch.setenv("COLUMNS", "80")```

[weary quarry]

I've never moved myself away from unittest.mock.patch but both should be accomplishing the same goal. Change the environment during the test and restore it before the next.

[jolly tangle]

Please ping on reply
In pytest how would I test an itsdangerous token specifically the time part?

class User:
     # code 
    def create_route_token(self):  
        SECRET_KEY = 'temp_secret_key'
        salt = self.random_route_token_salt
        serializer = URLSafeTimedSerializer(SECRET_KEY, salt)
        # random number
        data_to_serialize = {'user_id': 'self.id'} 
        # 30 minutes
        token = serializer.dumps(data_to_serialize) # Add a timestamp to ensure uniqueness
        return token

    # todo remove try see what error I get when it expires and delete the token from the db. 
    # Also check wrong token
    def verify_route_token(self, route_token):
        SECRET_KEY = 'temp_secret_key' 
        salt = self.random_route_token_salt
        serializer = URLSafeTimedSerializer(SECRET_KEY, salt)
        try:
            # remove max-age when using pytest
            flash('The token is expired or something else has gone wrong. Please login and then follow the instructions in your email.')
            token_verify = serializer.loads(route_token, max_age=1800)
        except:
            return redirect(url_for('auth.login'))
        return token_verify

[hidden marten]

You might want to look at https://pypi.org/project/freezegun/ and the associated https://pypi.org/project/pytest-freezegun/ for some interesting time-management behaviors in testing

[jolly tangle]

@hidden marten I will take a look thanks

[jolly tangle]

@hidden marten
Quick followup question why does the assert work? What I mean is how do you know later is 1 sec later and not say x seconds later?

def test_frozen_date(freezer):
    now = datetime.now()
    time.sleep(1)
    later = datetime.now()
    assert now == later

[split field]

That test passes because both calls to now() return the same time, because freezegun froze the time

[jolly tangle]

@split field Okay thanks that makes sense

[nimble bison]

Hi. I haven't unittested in Python for a while, and now I forgot this.

It isn't possible to check the variable value inside a function while testing it, right?

Also, it isn't possible to check what a function inside a function returns, correct? (while testing the outer function)

[jolly tangle]

I am not getting an exception even though I made the time greater then then verify_email_token. Why is this happening?

Here is models.py
https://paste.pythondiscord.com/GFGA

conftest.py
https://paste.pythondiscord.com/ESSQ

test_token.py
https://paste.pythondiscord.com/MHUQ

E Failed: DID NOT RAISE <class 'Exception'> Why is an exception not being raised?

[river pilot]

which line should have raised the exception?

line 30 just mentions verify_token, it won't do anything

perhaps you meant to call it? verify_token() ?

[jolly tangle]

@river pilot thanks for the help. Yes it isverify_token in test_token.py. I believe it is correct because verify_token method is called in conftest.py with (). But who knows I could have made a mistake but I looked it over earlier and didn't notice

[jolly tangle]

Maybe the error is caused by expired tokens not giving an exception let me check the doc s

[proud nebula]

Correct. You can mock stuff to check that but it's better to test the used function directly and explicitly.

[jolly tangle]

I looked at the docs and it is SignatureExpired exception. So I am confused why it is not working

I think I misread the docs and https://pypi.org/project/pytest-freezegun/ won't work in my situation

[proud nebula]

You should probably use time-machine instead. Freezegun is slow, has many bugs, and isn't maintained much. (I know, I was the only active maintainer a while, before giving up)

[jolly tangle]

I will take a look thanks

What if I just use sleep and use 1 sec and 2 seconds

[proud nebula]

sleep is basically saying "wait x seconds", but that doesn't mean you get back to the code 1 x seconds later exactly. It just means you won't get called in UNDER x seconds.

And it will of course make you tests x seconds slower.

[proud nebula]

I think you want to move the time forward like sleeper.tick()

I don't remember the exact syntax though

[jolly tangle]

Okay thanks I will take a look at it

[jolly tangle]

Sorry for another question
https://pypi.org/project/time-machine/

Here is the example from the docs

import datetime as dt
import time
import time_machine

with time_machine.travel(0, tick=False) as traveller:
    assert time.time() == 0

    traveller.move_to(234)
    assert time.time() == 234

Here is the example I am trying. In conftest.py file valid_yield_token_and_verify_token I called verify_token , along with token and user_db, which has max_age=x amount. My goal is past the max_age. But for whatever reason the code below throws the error E AttributeError: 'TimeMachineFixture' object has no attribute 'travel' ? Why is the error occurring? I followed the example above.

import pytest
from datetime import datetime, timedelta
#import time_machine
from itsdangerous import SignatureExpired
def test_token(time_machine, valid_yield_token_and_verify_token, username_form, hashed_password_form, email_form):
               
    user_db, token, verify_token = valid_yield_token_and_verify_token(username_form, hashed_password_form, email_form)        
    assert user_db != None

    print(f'The token_1 is = {token}')   
    assert token != None

    current_time = datetime.now() 
    add_time = timedelta(seconds=1801)
    past_current_time = current_time + add_time
    #with travel(datetime(1985, 10, 26), tick=True):
    #with travel(current_time_plus_30_min, tick=True):
    with time_machine.travel(current_time, tick=False) as traveller:
        assert verify_token == True
        traveller.move_to(past_current_time)
        with pytest.raises(Exception):
            verify_token

Also here is verify_email_token method in models.py

   def verify_email_token(self, email_token):
       SECRET_KEY = 'temp_secret_key' 
       salt = self.random_email_token_salt
       serializer = URLSafeTimedSerializer(SECRET_KEY, salt)
       try:
           # remove max-age when using pytest
           flash('The token is expired or something else has gone wrong. Please login and then follow the instructions in your email.')
           token_verify = serializer.loads(email_token, max_age=1)
       except Exception as e:
           print("An exception occurred:", e)
           # Re-raise the caught exception
           raise
       print('The token works')
       return True

[proud nebula]

Hmm. I don't think you should have that type there. Did you copy paste your code?

[jolly tangle]

I used the example where I show the docs any idea how to fix it?

[whole orchid]

should the unit testjng be done before or after fixing the bug

[proud nebula]

It's best to write the failing test first. You can fool yourself easier otherwise. But it's not a dogma, you can learn a lot from fixing the bug that is useful to write the test sometimes.

[bright wasp]

Hi,

I have a question about a small Python program I'm working on. My question is related to a command prompt for context.

I'm trying to create a function, a try block, or something similar that, wherever it is called, automatically estimates the progress time of the commands it is executing and the remaining time.

I want to ensure that I only need to call this function, which handles the entire progress bar part, so I don't have to rewrite the whole script many times. This function or try block will be reused very often.

The slightly complicated part is making sure that the progress bar is locked at the bottom of the terminal, while different commands, prints, and inputs are displayed above it, similar to how apt update works on Linux. I've managed to do this, but perhaps not in the best way possible.

So, my simple question is: how can I achieve this? I can't seem to find a solution other than:

Rewriting the script,
Adding a variable for the script's progress (which is not automatic),
Or constantly updating the progress bar, which isn't truly automatic either.
Thank you for your help!
||ps sry im not a pro in python but im trying my best||

For example, this is what I managed to do, but it doesn't work as I would like since I have to tell my progress bar the progression of my script. I'm trying to make it automatic.

[maiden pawn]

it were to implement me...

i would have used subprocess.Popen with intercepting line by line output in real time

checked lines for different markers regarding real process bar state (which i probably scrapped with re.search / if "line" in and etc methods)

and via Observer design pattern relayed this info to watching objects for their update

[proud nebula]

Do you really want to implement it or do you want one? tqdm is very good, and there many others

[bright wasp]

I don't know tqdm
And to be honest yeah i prefer implement it, even if this to much for my low level in python
Line that i can learn

[bright wasp]

A subprocess uh ?
Well i will try to do this when im back to my home

[bright wasp]

Mmmh i see

[bright wasp]

An observer of the current events i guess ?

[maiden pawn]

https://refactoring.guru/design-patterns/observer

Observer allows elegantly injecting into such abstraction

that some other objects wish to know updates and change their views on observed state changes

that will allow us having this TaskStateTracker to have its own code

and ProgressBar having its logic to receive updated values and rerender in its own class

no will be "code clashes" here. ProgressBar will have its code remaining in its own class, and just will register for TaskStateTracker updates.

it should be easy to unit test and clean in terms of code

Head First Design Patterns is fun book to read in more depth

[bright wasp]

Oh i see, thanks you a lot. I will learn that for implement it in my code

[maiden pawn]

tqdm looks fine. potentially it will be able to work with Popen intercepted line by line stuff
Questionable though if it iwill be able working here, since Popen lines are infinity until process is over
Can it work with Iterable

Instantly make your loops show a smart progress meter - just wrap any iterable with tqdm(iterable), and you're done!

it claimes it can... but how will it estimate the end 🤔

probably we have to supply len our own into the Popen.lines may be. it probably gets from __len__ dundermethod

https://github.com/tqdm/tqdm#manual
Ah yes it is described there. optional variables total, or len() should be provided.
Otherwise utilize bpar.update(10) for manual control

TLDR: what i described above will work with tqdm

[bright wasp]

By the way i have a little other question
Do you think for sell a script with a license key it could be interesting to:
1-When someone buy my script i give im a licence key and a little script
2-If the licence key look like good to the criteria it have for the script, it will ask a Flask server i made for verifiy the authenticity of the key ( everything in background )
3-If the key is real and unused with a database i will made
It will give the script
4-and for secure the script for avoid any re-uploading there is a auto encryption of the script or maybe a auto destruction if the script see the MAC adresse has changed

Its my first big script i made, and the first time i want to sell something, so maybe i say lot of shit
Its why i would like having ur advice

[bright wasp]

I see, that's very practice!
Thanks!

[maiden pawn]

🙈 protecting little python scripts is useless
Python can protects its code only if its execution on server side

compiling python is kind of pain in the ass

pick other language if u wish more obfuscation friendliness. or make solution server side, while having clients publicly free

[bright wasp]

Problem i only know python, i mean mainly
And i already made 40% of my script in python soooo
And the good point with python its generally installed on lot of computer/OS

Server side could be complicated

Beceause its a script for who work everyday, and need to be able to work even watchout internet

[maiden pawn]

in general security approach with obfuscating client side is useless.
Everything is scrappable. Only server side is safe.
Client side can be at best made more difficult to reverse engineer, so that talented enough people to reverse engineer will not consider it worthy

and scripting langauges to obfuscate is really bad since no one is going to trust you if u made a virus there instead or not. Same in general regarding client obfuscation, that's why keeping stuff server side is more reliable approach, where people can trust that it is not virus for their PC. And a good approach making clients public (to make people see you aren't giving them virus)
TLDR: hide paid content at server side. That is acceptable approach.

[bright wasp]

Oooh

Yeah i see
Yeah now u said that of course that should be very suspicious
Even more when sometimes windows defender block my script beceause i interact with the appdata

I see the approach you're teaching me
Well i will think about that so, thx

[proper wind]

hi

[teal quartz]

good morning !
In pytest-bdd, all steps are executed within the context of a scenario.
Is there some way for the individual steps to "know" which scneario is doing the step ?

[raven tusk]

Guys, how do you write a tester to cover the red line?

[proud nebula]

Call it twice?

[raven tusk]

This is my tester at the moment.

def test_normalize(self) -> None:
        """ Test the `.normalize()` method.
        """
        data = Data([[1, 2, 3], [4, 5, 6]])
        assert data.normalized is False

        def is_normalized(state: MutableSequence[MutableSequence[float]] | MutableSequence[float]) -> bool:
            """ Test if a state vector is normalized.

            Parameters
            -----------
            state (MutableSequence):
                A list representing a state vector.

            Returns
            --------
            (bool): True if the state vector is normalized. False otherwise.
            """
            # Calculate the norm squared of the state vector
            norm_squared = np.linalg.norm(state) ** 2

            # Set the tolerance
            epsilon = 1e-6

            # Assert the Born rule
            if np.abs(norm_squared - 1) < epsilon:
                return True
            else:
                return False

        data.normalize()
        assert data.normalized
        assert is_normalized(data.data)

Like do data.normalize() twice?

[river pilot]

i don't see where you are doing it twice. Also, I wouldn't nest these functions.

[raven tusk]

Because of this recommendation.

[raven tusk]

Yessir, moving it out in a moment.

[river pilot]

sorry, i missed that. Yes, you could call it twice.

[raven tusk]

Please don't apologize.

[lucid lark]

Can anyone help me implement a live "status bar" in the bottom of the terminal, similar to what "pwncat" is using?

[tulip leaf]

I'd take a look at "rich"/"textual" for that, but it seems "pwncat" isn't using them it does use rich.

[lucid lark]

the thing is: let's say there are 2 layouts "main" and "status", status bar will perfectly run, however, the main layout does not properly work

can i dm you?

[tulip leaf]

I'm a little busy right now and have never used rich, but if we can take the discussion to #user-interfaces instead, I'm sure more people will be able to help you.

[raven tusk]

Greetings there,

Hope all are well. I was wondering how one would write unit testers for a code that requires API token (from an account for instance).

[maiden pawn]

there are several solutions

[raven tusk]

Yeah, I learned I could mock it.

[maiden pawn]

1. using libraries that autorecord last request to API, and mock later with the recorded value for you
2. writing code that can be easy injected with mocks

[raven tusk]

Can you help me mock this for practice please?

class Service:
    def __init__(self,
                 api: str,
                 device: str="CPU") -> None:
        if api != "api":
            raise ValueError("API invalid.")
        self.api = api
        self.device = device

    def run(self, circuit: str) -> dict:
        return {"result": circuit}

[maiden pawn]

mocking can be done in its turn in dirty way through patching smth globally

and it can be done at architectural clean way

where u just input alternative class instance during initiailizations of smth

[proud nebula]

The first rule of mocking, is to avoid mocking. You are better off either with dependency injection like Darkwind said, or isolating your code from the code that does the external request (functional core - imperative shell).

[maiden pawn]

i don't see anything that can break here without API token?

i don't see why it needs to be changed in any way currently

[raven tusk]

The api key. We don't want to need an api key to run the code.

This is an example. For my actual problem I would need an api key (from my personal account) to run the code. I fixed it by just hard coding what the api call would return.

[maiden pawn]

u probably show too small amount of code to make sense

your words have meaning, but the code above shows too little example i suspect

it does not show the "Api Call" it makes

[maiden pawn]

anyway... for clean solution u pretty much already wrote solution. We make dummy like this class
and just inject it in relevant code being used instead of a real API service

the trick is in.. organizing your code, so such stuff could be easily preferably inputed (and real initialization from working code did not affect our testing code in anyway)

[raven tusk]

I honestly just made a dummy class to do this.

It's probably not the proper way of doing it.

[proud nebula]

imo it's often cleaner to have some code that fetches the data and sends it to your own code:

def foo():
    result = get_from_external_service()
    thing_you_test(result)

If you really want to mock the result from the external API this also makes that easier and more obvious imo.

[alpine lagoon]

Hi, I mock a decorator named check_perm testing a function with pytest. However, pytest considers decorator's function argument fn as a fixture although it's just an argument.
How can I say to pytest runner not to consider this fn parameter as a fixture?

def mock_check_perm(permission):
    def decorator(fn):
        async def wrapper(*args, **kwargs):
            return await fn(*args, **kwargs)

        return wrapper

    return decorator

[proud nebula]

Are you decorating tests?

[alpine lagoon]

no actually patching them.

I simplified the code a) code to be tested https://hastebin.skyra.pw/esutovebaw.pgsql and b) the test file: https://hastebin.skyra.pw/esukivubin.pgsql

I want to monkeypatch the decorator before wrapping the actual function. However, pytest runner complains by considering decorator arguments as fixtures.

[proud nebula]

I believe the problem isn't pytest but your decorator. Use the inspect module to look at the test function after decoration yourself.

[hexed cloak]

You can't use .start() as a decorator

Afaik

Oh you're relying on start() returning mock_check_perm you shouldn't do that

You also can't mock decorators after they've been applied

[torpid osprey]

    async def _websocket_connection(token: str):
        websocket = AsyncMock()
        websocket.path_params.return_value = {"token":token}
        current_user = await get_websocket_user(websocket=websocket,db=db)
        websocket.username = current_user.username
        message_inbox[websocket.username] = asyncio.Queue()
        async def send_message(data):
            await message_inbox[websocket.username].put(data)

        async def receive_message():
            return await message_inbox[websocket.username].get()

        websocket.send_json.side_effect = send_message
        websocket.recv.side_effect = receive_message

        return websocket, current_user, message_inbox``` Whenever I'm mocking a websocket that has asynchronous methods, I get this error:

FAILED tests/test_dms.py::test_create_dm - AttributeError: 'coroutine' object has no attribute 'rsplit'
=============================================================================================== 1 failed, 1 passed, 1 warning in 1.23s ================================================================================================ 
sys:1: RuntimeWarning: coroutine 'AsyncMockMixin._execute_mock_call' was never awaited
RuntimeWarning: Enable tracemalloc to get the object allocation traceback```

Does anyone know what the possible issue could be?

[proud nebula]

Can't you get the full traceback from the attribute error?

[torpid osprey]

I didn't see any more details other than the traceback I showed. Maybe its somewhere in the internal application so I'll check for clues there.

[proud nebula]

You have to get the traceback I think. Somewhere code thinks it has a string but it's a coroutine.

[torpid osprey]

Ah okay.

[torpid osprey]

https://github.com/bendeez/discord-clone/tree/main/backend/tests What do you all think about my unit tests?

[prisma trail]

For some reason, tox refuses to install multiple dependency versions. It installs the different python versions as expected, but it just installs Django v4.2.13 regardless of the specification.

It should be installing versions 4.0, 4.1, 4.2, and 5.0 for each interpreter

[river pilot]

you'll need to link us to your tox configuration file

[prisma trail]

https://github.com/nullandvoid-digital/django-npi-field

[river pilot]

I don't know how tox.ini is using your poetry.lock file

https://github.com/nullandvoid-digital/django-npi-field/blob/main/poetry.lock#L93-L94

[bitter wadi]

poetry.lock lines 93 to 94

name = "django"
version = "4.2.13"```

[prisma trail]

I haven’t the foggiest idea.

[river pilot]

i don't use poetry, so I also don't know.

[random sorrel]

can I easily use multiprocessing for unittesting?

alternatively where is the repo, maybe I can add that?

it's a bit difficult to search for.

like, I kind of expected a unittest.main(cores=4) that defaults to None.

[river pilot]

pytest has features for running tests in parallel.

[random sorrel]

that sounds promising, I will look up what I can find on that.

thanks!

[hexed cloak]

pytest-xdist is popular for that

[random sorrel]

pytest depends and the multiprocessing one don't seem to play nicely together?

[shut wave]

trying to figure out how to use indirect parametrization against a fixture that returns a list. what I have on line 15 doesn't work -- I wouldn't really expect it to, but how should I be doing this?

from copy import deepcopy

import pytest


@pytest.fixture()
def required_keys():
    return [
        "principalId",
        "issueCategory",
        "applicationComponent",
        "description",
    ]

@pytest.mark.parametrize("required_key", ["required_keys"], indirect=True)
def test_missing_required_keys(required_key, valid_data):
    test_data = deepcopy(valid_data)
    del test_data[required_key]

    with pytest.raises(ValidationError):
        FeedbackSubmission(**test_data)

[river pilot]

I would do this without a fixture:

required_keys = [
        "principalId",
        "issueCategory",
        "applicationComponent",
        "description",
    ]

@pytest.mark.parametrize("required_key", required_keys)
def test_missing_required_keys(required_key, valid_data):
    ...

but tbh, i don't know what indirect=True is supposed to do...

[shut wave]

@river pilot required_keys is used in a lot of tests so I'm trying to keep it as a fixture to avoid repeating myself 😩

but tbh, i don't know what indirect=True is supposed to do...
you mean you don't know that option, or you don't know what I'm thinking? 😄

[river pilot]

put the list in a module and import it where you need it. I've never used indirect=True.

fixtures are cool, but sometimes you just need a function/constant/etc that you import where you need it.

[shut wave]

so here's the docs on indirect parametrization https://docs.pytest.org/en/7.1.x/example/parametrize.html?highlight=indirect#indirect-parametrization

[river pilot]

ok, that looks like a way to pass arguments to the fixture function. You aren't doing that.

[shut wave]

yeah, I see what you mean

I guess the problem could be re-stated as "how can I access a fixture in the arguments to a parametrize decorator"

[river pilot]

i don't see why you need a fixture.

[shut wave]

you might be right

for some reason I remain fascinated by the question, though 😅

but since I work for a living I suppose I'll get over it and break it out into a normal list like you said

if I find the answer I'll report back

[lone hare]

Is it better to ask questions about planning tests in #web-development

?

[frigid plaza]

Hi i'm actually upgrading the python version (3.8 -> 3.10) and django version (2.2.X -> 4.2.X) of my django project and i noticed that my tests have their execution time slower than before (almost x2). I tried to display database logs but i didn't notice anything strange. I'm using DRF and pytest. Do you already heard about stuff like this ? Ty (ping me if answer)

[maiden pawn]

Observability matters

Good to use Tracing systems for backend things for finding quickly which network requests have which slow time

Time measurement and profiling are next possible lightweight thing to use. Tracing is more meant for debugging slowness of such stuff though

[frigid plaza]

Hi, ty for your answer, do you have any package recommandation for this kind of things ?

Which works well with pytest for example

[proud nebula]

The built in profiler should be pretty good.

Just a little idea: check what happens if you set the password hasher to md5 in tests (ONLY in tests). Some people see huge speedups.

[frigid plaza]

LOL it worked

do you think it's still related to my django upgrade ?

[maiden pawn]

the built in profiler is lightweight solution with short time investment. + the obvious investment into better logging (debug+info levels to use with time capture)
opentelemetry instrumenting libs submitting stuff to tempo+grafana stack can be high time investment stuff for getting visibility in different perspective with gui.
Profiling exists in grafana stack too (pyroscope), but i recommend tracing here as it is more meaningfully related for backend apps first

[proud nebula]

Yep. 100%. The password strength is always updated.

You should try to avoid hitting the hashing code in your tests. Even md5 is super silly.

[frigid plaza]

Ok ! Ty i will setup it to do this properly !

[shut wave]

this looks more promising for what I was trying to do, but still probably not justified https://docs.pytest.org/en/7.1.x/how-to/parametrize.html#basic-pytest-generate-tests-example

The idea would be to use a test generator to load the fixtures and do the parametrization

But like you said a fixture has marginal (if any) advantage over an ordinary list of literals in this case

so maybe I'll keep this in my back pocket for the day when I find myself in a situation where there isn't such an easy way out

[jolly tangle]

I am not sure if I should post this in the unit test or the database server but I am using flask-sqlalcehmy and pytest.

In conftest.py I am trying to add the UserTest columns and the AuthTokenTest columns. Then I am trying to query user_db.... then I am trying to access a column from AuthTokenTest. What am I doing wrong? I assume it has something to do with datetime but I could easily be wrong.

Also am I showing enough code?

models.py
https://paste.pythondiscord.com/X6RQ

conftest.py
https://paste.pythondiscord.com/VLTA

error
https://paste.pythondiscord.com/2RBA

I found this but the solution made but it made no difference .

https://stackoverflow.com/questions/56505400/sqlalchemy-exc-statementerror-builtins-typeerror-sqlite-datetime-type-only-ac

When I click on “flask_env\Lib\site-packages\sqlalchemy\dialects\sqlite\base.py:1068: StatementError”
I get the

raise TypeError("SQLite DateTime type only accepts Python "
                "datetime and date objects as input.")

Any advice on what I should do?

Also please ping on reply.

Thanks in advance.

[proud nebula]

The error sounds pretty clear. What are you trying to pass there that isn't a datetime?

[jolly tangle]

@proud nebula

I am passing the code below but I looked at the documentation and it should work.

Here is the docs https://docs.python.org/3/library/datetime.html .

Here is the code

@pytest.fixture 
def time_token_expired_form():
    current_time = datetime.now()
    # add time_token_expired
    thirty_min = timedelta(minutes=30)
    current_time_plus_30_minutes = current_time + thirty_min 
    time_token_expired_form = current_time_plus_30_minutes
    return time_token_expired_form

Thanks for the help.

[proud nebula]

For anyone else: the above question is answered on SO

[jolly tangle]

But I am still confused on the solution

[proud nebula]

So you deleted the link with the answer? That will not help us explain it. Post the link again and ask a question about it.

[jolly tangle]

@proud nebula Did I really delete the link with the answer? I am genuinely asking because I have no memory of doing it

[jolly tangle]

I solved it sorry this was a really dumb question in retrospect I was a little distracted before

[proud nebula]

Oh sorry. You posted another message and it had "edited" on it and I got confused. My bad.

[proud nebula]

No question is stupid. But a question can be badly asked ;)

[jolly tangle]

Don't worry it was a really dumb question like stated

it was dumb because I wasn't paying attention

I was distracted that day

[spring brook]

so i've been banging my head against the wall for like 10 hours and a few hundred lines of code on this:
how would i inject a fixture (pytest-benchmark's benchmark fixture) into a function that doesn't currently take the fixture as an argument? tests look like this github link, and we're using pytest to run tests. i've tried various things in the pytest_generate_tests, pytest_collection_modifyitems, and pytest_runtest_call hooks, such as the following:

def pytest_generate_tests(metafunc):
    if 'benchmark' not in metafunc.fixturenames:
        benchmark = metafunc.config.pluginmanager.get_plugin('benchmark')

        if benchmark is not None:
            original_func = metafunc.function

            def wrapper(benchmark, *args, **kwargs):
                return benchmark.pedantic(original_func, *args, **kwargs)

            metafunc.function = wrapper
        else:
            print("a")
            1 / 0
    else:
        print("b")
        1 / 0

but all it does is end up warning me like follows:

  tests/jsonpickle_test.py:236: PytestBenchmarkWarning: Benchmark fixture was not used at all in this test!
    def test_boolean(self):

(please ping me if you have any ideas)

[raw tundra]

can't you just do something like

@pytest.fixture(autouse=True)
def autouse_benchmark(benchmark):
    ...

(I don't know what the benchmark fixture does, so this might not be what you're looking for)

Oh I see

yeah this approach won't work sorry for the ping

out of curiosity, what if you try modifying the signature of wrapper to match that of original_func?

[next sun]

would you guys suggest pytest-mock in 2024 for unit testing?

[maiden pawn]

i just use std library for that and that's it
https://docs.python.org/3/library/unittest.mock.html

not feeling need to install extra lib for that

[next sun]

TBH using unittest.mock is solving a problem that i am having with pytest_mock, just wanted to confirm how other python devs handle it?

[maiden pawn]

Bringing extra third party library if it is not justified, then it is bad. 😛

Quality is achieved through having minimum necessary amount of third party libs.

Although to be fair... when i checked pytest-mock and discovered they offer patching, mocking without need for "nested" code. I became very captivated to install it :/ i don't like code more nested than it is necessary

default unittest.mock asks to use nested with constructions, so the resource is "released" for a next test

may be i should refactor to pytest-mock my mocks

[next sun]

Meaning instead of giving blanket statement at first, you should get some more context 😛

[maiden pawn]

on another hand i have low enough level of nestiness appeared from unittest.mock, so not needing that

Also, if u have too many mocks/nests, then may be smth is wrong with code 😁

[next sun]

My problem around this lib is when I am trying to mock unit test the following function. I have deliberately removed error handling, logging etc for the sake of example.

def _download_files(url: str, file_path: pathlib.Path) -> None:
    with httpx.stream("GET", url) as response, open(file_path, "wb") as fp:
        for chunk in response.iter_bytes():
            fp.write(chunk)

I am able to implement a unit test that uses pytest_mock very trivially along with unittest.mock to mock this, but it requires me to define 2 fixtures, 1 for mocking httpx.stream and 1 for mocking builtins.open

I wanted to see if I can shift my unit test to use any one of unittest.mock or pytest_mock

[next sun]

Here are my unit tests and fixtures, so you see how blindly following a blanket statement will leave me to manage more code eventually which could have been avoided with the use of pytest_mock
https://pastebin.com/WeBN7is4

[maiden pawn]

but it requires me to define 2 fixtures, 1 for mocking httpx.stream and 1 for mocking builtins.open
...

i think u have just wrong approach what to mock.

httpx.stream is custom third party library. it is hard and not very beneficial to mock (because it is already tested and has its own difficult custom structure)

def _download_files(url: str, file_path: pathlib.Path) -> None:

your function on another hand has simple type variables as input and output

therefore u are supposed to mock _download_files itself, if u use it in tests of other code

[next sun]

@maiden pawn my bad, I don't want to mock _download_files instead I want to write unit - tests for the same, which will require me to mock the 2 dependencies

[maiden pawn]

and that's my answer i think. U should want to mock _download_files itself instead

and not writing tests for details of it, to avoid interacting with httpx in your unit tests

i mean... u can write interacting with url if u wish, not really a problem to do. Raising some endpoint for testing and using it for testing

otherwise i see no benefit mocking httpx itself

mock _download_files itself, and/or raise testing endpoint and test _download_files without mocks

[maiden pawn]

The point why i say it, according to code logic of your function, httpx usage makes 80%+ of code logic.
Therefore if u mock it.... u receive code that is not testing anything. Also you make your mocking code too fragile depended on third party code, unnecessary complexity.
Therefore better mocking _download_files for receiving easy mock to use in other code parts.
Or not mocking httpx and "integration" testing against some endpoint, then 100% of your function logic will be validated in a reliable way

[next sun]

@maiden pawn no, this is not a sound advice according to me.
What you are eventually saying is that _download_files function should not be unit tested because

1. interacting with httpx is too cumbersome (Skill issue)
2. this can be integ tested.

Again this is a very simple version that I pasted to specifically talk about the use case of mocking httpx.stream and open and pytest_mock vs unittest.mock, there are a lot of error handling that I skipped which needs to be unit tested.
Also I might or might not have the ability to raise testing endpoints for integ tests, based on my conditions

[maiden pawn]

my point stands. mocking httpx is pointless. you receive code that is not testing anything besides your mock then, which will be working always as it is.
At least according to your provided code example

[next sun]

In my code example, if you check the 1st line, it does say that I have removed some parts.
It is done so that i can show the diff b/w pytest_mock vs unittest.mock usage.

[spring brook]

yeah i tried that, didn't work sadly :(

[molten fog]

any opinion on writing tests to see if element exists? How do you approach this?

for selenium

[molten fog]

I created a function since I did not want to keep writing this out

def find_element(driver, by, element, name):
    try:
        elem = driver.find_element(by, element)
        return elem
    except NoSuchElementException:
        pytest.fail(f"Element '{name}' with locator '{by}={element}' could not be found")

I have another function to check the value which is why i return elem

def check_content(element, expected_value, use_value=True):
     try:
        # Determine whether to get 'value' attribute or text content
        if use_value:
            result = element.get_attribute("value")
        else:
            result = element.text
        
        # Compare the actual value with the expected value
        if result != expected_value:
            raise ValueError(f"Expected value '{expected_value}', but got '{result}'")
    except Exception as e:
        raise ValueError(f"An error occurred while checking the content: {str(e)}")```

[jolly tangle]

In flask-sqlachemy and pytest I am trying to use the time-machine package. The problem is I don't know how to use it it in the function check_expired_token without getting an error. Below contains the error and a link to the time-machine package. Can someone help make a working example with pytest and using the function check_expired_token ?
https://pypi.org/project/time-machine/

Conftest.py

https://paste.pythondiscord.com/O3TQ

test_functions.py
https://paste.pythondiscord.com/FLLA

models.py
https://paste.pythondiscord.com/HMDQ

Here is the error
https://paste.pythondiscord.com/Y7AA

[jolly tangle]

I may have found an answer

https://stackoverflow.com/questions/57579659/make-pytest-use-a-fixed-date-for-the-database, i just need to adapt this to time-machine

[unique spade]

Hey I’m having some issues with using Pytest. My flask API uses pyodbc to talk to a MSSQL server db.

I have a test for a healthcheck (root) route that is as follows:

from app import app

def test_healthcheck(self):
    """
    Test the healthcheck endpoint
    EXPECTS: A 200 status code and the string "Healthcheck"
    """
    with app.test_client() as test_client:
        response = self.app.get("/")
        assert response.status_code == 200
        assert response.data, b"Healthcheck"

And my project is structured as follows:

myproject
|
| — api/
            | — routes/
            | — utils/
| — tests/
                | test_health.py

When I run python3 -m pytest, I get the following error:

ERROR tests/functional/test_health.py - pyodbc.Error: ('01000', "[01000] [unixODBC][Driver Manager]Can't open lib 'ODBC Driver 18 for SQL Server' : file not found (0) (SQLDriverConnect)")

But I have pyodbc installed.

The api itself runs in a docker container, and I’m using a Mac btw

[tall brook]

did you check posts like these? https://stackoverflow.com/questions/44527452/

[unique spade]

yeah. Tried odbcinst-j and the got the following:

mymac@IT-MB-002 ~ % odbcinst -j
unixODBC 2.3.12
DRIVERS............: /opt/homebrew/etc/odbcinst.ini
SYSTEM DATA SOURCES: /opt/homebrew/etc/odbc.ini
FILE DATA SOURCES..: /opt/homebrew/etc/ODBCDataSources
USER DATA SOURCES..: /Users/mymac/.odbc.ini
SQLULEN Size.......: 8
SQLLEN Size........: 8
SQLSETPOSIROW Size.: 8

[tall brook]

when you say the API run in a container. That API is the one that connects to the database?

[unique spade]

Yeah. The API has an endpoint that connects to an MSSQL server db

[tall brook]

and you installed MSSQL driver in Mac? shouldn't you install it in the container?

[unique spade]

It’s also in the container

When I run the API I run it via docker compose up —build

when I run the unit tests I’m doing it with python3 -m pytest

…do I have to run the unit tests within docker?

And if so how do I do that?

[tall brook]

I usually run tests from containers if I want to test the code is in it. To do that just ensure your tests are also in container file system (either use COPY or VOLUME), then you run pytest command using something like docker run --rm -it python pytest .

[unique spade]

ahh

I’m not at my desk rn but I can share my docker-compose in about an hr

[whole orchid]

Wait how are you supposed to use mock data factories to test? Like if I generate random data from a factory, for my input, and the function I'm testing is meant to transform that data, how do I procure the expected test data from that initial input mock data?

llike u can't really connect them right?

so the only way is to use fixtures?

[maiden pawn]

Check factory-boy

It offer factories to generate random data, yes
But u can specify for certain fields their fixed value.

That is enough to expect specific enough data

[manic belfry]

is there a way to make this a simple and re-usable fixture so that i can use it across multiple test files?

files = (
    Path("tests/test_data/source_BEST.rar"),
    Path("tests/test_data/source_BZIP2.7z"),
    Path("tests/test_data/source_BZIP2.zip"),
    Path("tests/test_data/source_DEFLATE.zip"),
    Path("tests/test_data/source_GNU.tar"),
    # Alot more, I've truncated them for this example
)

@pytest.mark.parametrize("file", files)
def test_extract(file: Path, tmp_path: Path) -> None:
    with ArchiveFile(file) as archive:
        member = archive.extract("README.md", destination=tmp_path)
        assert member.is_file()

[whole orchid]

at that point whats the reason for creating a factory instead of just inlining the mock data

[torpid osprey]

Could someone review the tests I made for my own concurrent event loop?: https://github.com/bendeez/AsyncLoop/tree/main/tests

[obsidian swift]

Is comparing test execution (how efficient/performant are the tests vs. the application code) across packages a fair comparison?

My test suite code is being compared to larger modules. "Tornado runs 12140 tests, 1214 per ten Python versions, in 3 minutes. Anything over 60 minutes is what I would expect from a much larger and comprehensive test suite as seen in electron or sqlalchemy where there are thousands of tests on dozens of environments"

My tests are executing a series of REST API calls against an implementation of a web framework, connected to a live database. They represent end to end integration tests for the web framework, the ORM, and the database, and the business logic built around them.

I imagine the tests for the modules I am being compared against might be simple unit tests with very little scale or latency to worry about?

I can't help but feel like we are comparing apples to orchards, but not sure how to push back on that comparison.

The REST API calls of the test suite are often dependent tests, and they overlap in namespace in the sense that the database identifiers are identical in the reused fixtures. (Parallelization of tests breaks as teardown/mutation could impact other tests happening in client side Python)

[maiden pawn]

i have reread your text three times and did not understand you compare your executions in comparison with what

packages word is having multiple possible meanings within python programming, so i was not able extract precise meaning out of your text

[obsidian swift]

I have a project that implements business logic around a Python web framework. My tests use Python requests to test the business logic built on the framework.

The tests make a request to the REST API like POST /API/create_records. The test then waits for a response from the server. The server takes the request data and operates against SQL Alchemy and the database. We make assertions around the response.

[maiden pawn]

okay. for now i am with you

[obsidian swift]

My test suite timing is being compared to the duration of the tests that SQL Alchemy module, or Tornado module run on their respective repositories

[maiden pawn]

My test suite timing is being compared to the duration of the tests that SQL Alchemy module, or Tornado module run on their respective repositories
your Test suite timing is compared to the duration of the tests that SQLAlchemy or Tornado module run on their respective repositories...
...Your application business code tests are compared with dev test time execution for SQLAlchemy and Tornado? For which reason?

Those tests have completely different meaning 🤔 SQLalchemy tests its SQLALchemy internal code right? And Tornado tests its own internal code? And you test your business logic while using those libraries.

why to compare those completely different sets of tests

[obsidian swift]

I agree. I think we are comparing unit tests to integration or end to end tests. It's very difficult to make a blanket statement and say that "these tests are fast, therefore yours should also be fast". I don't think it's a fair comparison, but I am trying to support the argument that it is unfair.

[maiden pawn]

https://www.sqlite.org/testing.html you can be using for example with SQLAlchemy with engine SQLite3 (lets supposed u use it)
SQlite3 has 155k lines of code
And 92053k lines of testing code 😁

so... what would be the point comparing your business app testing code with how 92053k code lines of SQlite3 tests are executed

for very extreme example, hehe

[obsidian swift]

I don't see the point, other than trying to pressure folks to make things run faster. It's like counting developer productivity in lines of code.

I think the point they are trying to make is that their developer tests are taking too long to execute. 40 minutes versus 5 minutes or less for some of the automated tests for libraries.

They are testing full coverage of the project, when their features touch an isolated module. They are also testing it at scale as well (more reads/writes). They are mixing testing types which is leading to longer execution.

[proud nebula]

I think you forgot the question.

[obsidian swift]

The question I am trying to answer is: why are a project's tests (project built on Tornado, SQLa) so much longer than the test executions for the libraries they are built on.

I don't think it's an easy or sensible comparison to make. But trying to determine how I push back and make that case.

[maiden pawn]

btw, 40 minutes is kind of very long i agree

and at this point u need to invest time into speeding up tests

for example through parallelism

rapid test response is important for dev comfort and speed

if u can run tests in 5 minutes it is great

[maiden pawn]

The question I am trying to answer is: why are a project's tests (project built on Tornado, SQLa) so much longer than the test executions for the libraries they are built on.
shrugs. devs invested time to speed them up

parallelized

mocked heavy time executions where they are not important

or wrote real unit tests and minimized integration testing code interacting with sql

you as business app, having most of fragility due to sql code is needing to test a lot of networked stuff

your choice to help yourself here is...

• fixing sql to be faster (async or just optimizations to rewrite them)
• or using parallelism. (like pytest-xdist)
• Or mocking database calls where they are not important.
  • Optionally more writing real unit testing code with separation from database calls if u can.

[proud nebula]

How long is a piece of string?

[brazen onyx]

Have you or anyone tried profiling the tests? Can they or you reduce it to one environment to see if adding environments has non linear slowdown? 1st thing to come to mind is a polling interval may be adding time if polling for results. 2nd thing might be pool size of connections that would restrict speed overall. At least this is how I would approach the isssue. If you can use decorators to time your functions that log the result then you can estimate how much overhead the test rig has compared to actual entry points in code being tested.

[obsidian swift]

My general suggestions were to run this particular set of tests either at a fixed interval (daily on stage) or on particular tags (alpha/beta/RC).

I also suggested they reduce their scale as they were testing the insert of 96 relationships (involving many SQL Alchemy models) and were in essence running a benchmark/scale test rather than a purely functional test.

[obsidian swift]

I also pointed out parallelism. They had tested parallelism but got hung up on fixtures. The fixtures used in overlapping tests create problems when one test modifies or tears down a fixture used by another test.

The fixture identifier is relevant to the database layer, so there is no way to isolate the tests without ensuring unique identifiers for every test.

"How long is a piece of string?" - not quite sure how to interpret your comment here, but I assume this is an analogy or sarcasm. As far as I can tell, what the other developer is suggesting (why I asked the question) is a nonsensical comparison. But they are likely just trying to say "help, my tests are too long".

I think improving the performance on individual tests is out of scope for the time being. They either need to scale back their testing scope or throw more hardware at it.

Refactoring SQL Alchemy usage (where most of the test time is likely coming from) is a long term effort and not a trivial fix.

[obsidian swift]

That's a good thought. I don't think we are constrained by connections and polling is not a factor at the moment. Most of the test time is request-response to a single server that can pick up multiple requests. The testing scripts run a series of REST API calls to that server, runs through business logic and SQL Alchemy, hands off to DB for data access, then circles back to response.

Effectively simulating a single user making (dependent) REST API calls sequentially.

[obsidian swift]

Never heard that expression before, learned something new! Guess it's common in the UK?

[manic belfry]

Am i missing something?

@pytest.mark.parametrize(
    "file,compression_type,compression_level,adapter",
    [
        (Path("tests/test_data/source_GNU.tar"), None, None, "TarFileAdapter"),
        (Path("tests/test_data/source_STORE.7z"), None, None, "SevenZipFileAdapter"),
        (Path("tests/test_data/source_STORE.rar"), None, None, "RarFileAdapter"),
        (Path("tests/test_data/source_STORE.zip"), CompressionType.STORED, None, "ZipFileAdapter"),
    ],
    ids=lambda tup: tup[0].name
)

I was under the impression that the ids=func will receive the full tuple

but that does not happen, instead I get this error:

tests\test_property.py:21: in <lambda>
    ids=lambda tup: tup[0].name
E   TypeError: 'WindowsPath' object is not subscriptable

so it receives the first element?

but nope

if i now change it to lambda path: path.name it errors that NoneType has no .name

so does that mean it's now moving over to the second param?

@pytest.mark.parametrize(
    "file,compression_type,compression_level,adapter",
    [
        (Path("tests/test_data/source_GNU.tar"), None, None, "TarFileAdapter"),
        (Path("tests/test_data/source_STORE.7z"), None, None, "SevenZipFileAdapter"),
        (Path("tests/test_data/source_STORE.rar"), None, None, "RarFileAdapter"),
        (Path("tests/test_data/source_STORE.zip"), CompressionType.STORED, None, "ZipFileAdapter"),
    ],
    ids=lambda x: x.name if isinstance(x, Path) else x
)

this is what I came up with, it works but since I don't have all the 4 args I can't customize it further

i.e something like lambda tup: f"{tup[0].name}-{tup[4]}"

looks like it's not possible https://github.com/pytest-dev/pytest/issues/8283

guess I'll do with my workaround

[proud nebula]

The tests travel across the network to a remote machine?! Yea that seems pretty obviously super slow.

[obsidian swift]

It would only be localhost latency as the test DB is located on the same machine as the app. But it's a large factor. Controlling SQL Alchemy query complexity and reducing N+1s are the main application factors.

Deconstructing the tests or designing as unit tests would go a long way. But there are certain aspects that are difficult to test in isolation. Many things are configuration driven and the relationships between the configurations are AFAIK not really unit testable.

Thanks for the insights

[proud nebula]

I would just start by picking the slowest test, running a profiler on it and check what can be done. I've cut an enormous amount of time from huge test suites in a bunch of different ways. I had to fix pytest once for example. Other times I joined 10 tests that asserted 10 different things on the same tested code path (because someone has cargo culted the "one assert per test" rule which is an idiotic idea)

[obsidian swift]

To give you an idea, 96 rows x maybe 20 columns are written/ updated. Changes are tracked in a change log table, so additional records there. Integrations with other systems add on additional rows to insert/update.

There are some server-side calculations we then need to return to user to update the user interface.

The primary API call they are executing takes ~15 seconds. They execute 1000 such API calls. ☠️

[proud nebula]

I have written a bit on this topic here: https://kodare.net/2020/05/19/python-is-slow-does-not-have-to-be.html

[proud nebula]

Jesus.

[obsidian swift]

I know my way around cprofiler, so likely I can track down some long hanging fruit. Most of the test code is making REST API calls against the running server and database. Increasing performance there has been ongoing, but ripping out SQLA/upgrading to more efficient ORM patterns is nontrivial

[proud nebula]

15s for a single api calls is crazy.

[obsidian swift]

It's more of an analytical app vs a public web app.

[proud nebula]

Long hanging? You mean low hanging hanging 🤣

[obsidian swift]

10s of users, maybe 100.

Typo on mobile ha.

[proud nebula]

Still. 15s for a view is a LOT

[obsidian swift]

Usually it gets dropped into Celery.
It's a workflow engine that spans a relatively long running laboratory process.

Lots of looking backwards to calculate values for each step.

It also has to maintain an audit log of changes.

[obsidian swift]

How long is a string?

If you insert\update\sort enough rows in an ORM transaction...

[proud nebula]

15s on a modern machine is a crazy amount though. Like having a string measured in light seconds ;)

[proud nebula]

Oooh. Maybe you can disable the audit trail in the tests?

Except in the audit trail tests obviously.

[obsidian swift]

It wouldn't fly for B2B/e-commerce.
But folks are doing sequencing. Sequencing takes much longer than 15sec. But yeah would love to make it faster, and enjoy doing it.

Disabling the audit trail is not a supported/optional feature at the moment. Would have to start inserting testing specific code paths in the REST endpoints.

If it was a separate service or endpoint, that would also be feasible.

But currently it's part of the same transaction. If you make a change, the transaction also ensures to persist the audit log rows.

[proud nebula]

Why in the endpoints? Surely the audit trail functions pass through one function and the tests run in the same process?

[obsidian swift]

The audit trail is part of the platform within the request-response bound db transaction

Test process is simply walking the endpoints roughly equivalent to a user in the browser.

Create a record POST /API/record
Audit log rolled up in that record creation/update/deletion

The endpoint can trigger custom functions or standard functions offer by the platform. Ideally they can focus on unit testing the custom functions more. But testing their usage of the functions (as part of the integrated platform) is also generally important, but quite expensive.

[proud nebula]

That was word salad I'm afraid

[river pilot]

@obsidian swift i've missed much of this discussion. Are you trying to make tests faster, or trying to convince someone they don't have to be faster?

[obsidian swift]

A little bit of both. They are running end to end / integration tests for every PR. Trying to push them towards trimming down the tests (less scale / records) or using a different set of tests that are more like unit tests.

[river pilot]

that sounds like a good thing to push for.

though: what's wrong with e2e tests for every PR?

[obsidian swift]

Sorry about that. Basically it is difficult to decouple functionality to speed up the tests.

The end to end tests take 40 minutes to execute. They run through Tornado, ORM, and database. I would also say they retest the same code paths more than once by running many records through the test.

[river pilot]

40 min is a long time, i understand

[obsidian swift]

Part of the barrier to suggesting changes (and thereby reducing time) is that we are being compared to the unit test performance for our component libraries (i.e. Tornado) instead of considering the many layers involved.

[proud nebula]

That comparison seems weird. Is that some manager making the comparison who knows nothing about code? Sounds like it.

[proud nebula]

You didn't respond at all to this comment.

[river pilot]

that comparison is just silly. Either use the library as inspiration to trim your tests, or admit that your code is not the library.

[obsidian swift]

The audit trail functions are part of the same database transaction as the changes they are auditing. The test cannot skip that functionality without changing the code that is being tested. It's a good thought though.

[obsidian swift]

My intuition has been that it is silly as well. But they are a paying customer (one of their developers extending our platform with their own configurations, Python, JavaScript). So it's a challenging conversation and they seem strongly opinionated on how Python and code built on it ought to be. 😅

I'm trying to treat their concern as sincere and point out why their comparison does not hold up in a polite way.

[dreamy agate]

@modern gull di sabling the audit trail is not a supported/optional feature the momen. Would have to start interseting if is make a change

[proud nebula]

I find the purity thought of never modifying the code under test to make testing better to be a bad idea. Just adding an if+return to check some global variable in one function to massively increase speed of tests is a great idea

[modern gull]

??

[solid garden]

Hi guys, I have a theory question - can a mock contain more methods (private) than the original object it is mocking? So methods that help with the mock setup itself.

[river pilot]

definitely. mocks already have more methods, like assert_called, etc: https://docs.python.org/3/library/unittest.mock.html#unittest.mock.Mock.assert_called

[solid garden]

isn't it a warning that the mock keeps too much state?

[river pilot]

it's important that you be able to ask the mock what happened to it.

[solid garden]

I am talking about methods from the business logic that help with the testing

[river pilot]

keep in mind: mocks are one kind of test double, and people say "mock" when they might mean other kinds of test doubles.

[river pilot]

that might be a fake or a stub rather than a mock.

[proud nebula]

Remember the first rule of mocking: avoid it if you can

[solid garden]

I already have a mock with the exact same methods as the original object, now the question is if I should add an extra method from the business logic domain that just will help me with testing, i.e. there's no such method in the real object

[river pilot]

touching back on what i was saying about test doubles. Your object is not a mock. It is a fake, I think. You've implemented a simple version of the API in a new object, right? I think it is fine to add methods to help with testing.

[delicate lynx]

I thinking to create a package to automate testing of rest API

[proud nebula]

What would that do? Testing json APIs is pretty simple already.

[sleek python]

For a simple web based CRUD application, with very little logic, what kinds of tests should I be doing? The app mostly presents data from a database in column/row form with a few forms for adding/updating data.

[proud nebula]

That isn't really answerable. You have to decide yourself. But some kind of check that the views work at all seems like a good start.

[sleek python]

@proud nebula I have at least that much...at least for the static views, any that require an ID of any type are still unaccounted for, but I suspect that may add a bit more complexitiy. I also have some that accomplish some minimal model testing

Inexperience is telling me I need more, but with no real internal logic, its hard to know what tests are actually needed

running the coverage app tells me I only have 51% of my code tested

[river pilot]

the code marked as uncovered in the coverage reports are clear candidates for more tests. What does that code do? How can you test it?

[runic jolt]

whenever you’re hand is itchy that means money is coming towards you.

[sleek python]

Its essentially a fairly simple internal ticketing system/work board with comments...nothing overly fancy

its intended to replace a locally created AccessDB that hasn't aged well...like most M$ products and its reached its size limit

[river pilot]

Look at your functions. Decide what they do. Ask yourself how you could check that they do that.

[sleek python]

pretty much all functions pull from DB, then send the data to a template...while the code is a bit more complicated then that, thats all that is really happening

[river pilot]

You seem to be saying you don't need tests

[sleek python]

thats my gut feeling, but am looking for confirmation

[river pilot]

the usual thing is to mock the db and see that the functions return the right data

[sleek python]

okay, can you point me to some code examples? im tseting the model and static views, thats about it

[river pilot]

i don't have a good example to point to, sorry

[sleek python]

okay, thanks, I will see what I can find

[balmy pebble]

any good learning materials for unit testing and testing with 3rd party api service?

[manic belfry]

How would I go about testing my flask webapp that uses mariadb via SQLAlchemy in CI?

Locally the tests pass because I have a full blown db setup

But I would need to somehow setup mariadb on the ci

mariadb-secure-installation (what I used locally) also seems to be interactive

I'm using pytest

[simple musk]

udemy or cursera

a really good book is "API Testing and Development with Postman" by Dave Westerveld. This book provides a practical approach to API testing using Postman.

[proud nebula]

I guess that would depend hugely on the CI. I use Github Actions at work and it's no problem to set up a postgres instance. You just do it in the yml file with a little bit of conf.

[maiden pawn]

the book name hints it is bad tbh. Testing by Postman? That's some kind of a joke.

while the tool is useful in its applications, teaching for testing with it is bad approach. Book should be teaching unit testing through auto testing means

with pytest or smth

[maiden pawn]

https://darklab8.github.io/blog/favourite.html#UnitTestingPrinciplesPracticesandPatterns this book is about theory of testing, including third party api services
utilmately though desired by you subject is in books about Clean Architecture https://darklab8.github.io/blog/favourite.html#CleanArchitectureACraftsmansGuide actually

As you need to "adjust" your code architecture to make it testable preferably.
mocks and patches can do a lot, but best designing it in the way it would be easier to test

[manic belfry]

Do you have any idea how I can setup mariadb in GitHub actions

I thought I could just run mysql_secure_installation in the ci but that's interactive

[proud nebula]

Did you google this? I found this as the first hit https://github.com/marketplace/actions/start-mariadb

[manic belfry]

Yep, I did see this but I haven't tried it yet

[calm juniper]

Hey everyone, please suggest a good book on unit testing which covers mindset as well, like philosophy behind unit testing and what exactly does it bring to the table. I am making a lot of mistakes at work, Looks like there is a gap in mindset for unit testing, sometimes these are regression but I still want to master over unit testing.

[proud nebula]

It's probably a better idea to ask specifically about those mistakes and we can see if tests would help or if there is something else that might be better to look at.

[dense bough]

The steps you keep repeating to see if your changes are working? For me, unit test is for simplifying that. Modify; run pytest; not okay yet? Repeat. So it is about automating checks for you expectations, and saving that for the future.

[calm juniper]

Okay, so we have a wrapper over Git, and I am adding a feature to automatically remove stale submodules.

Here is the scenario:

Suppose your HEAD is at a commit after a submodule has been added, and suddenly you want to check out a commit earlier than the one when the submodule was added. You will see stale submodules, and even git clean -fxd won't work as your .git/config file has been updated.

So, I added a feature to clean and remove the submodule from the index, and then I accidentally added git clean -fxd as well in that particular command, which cleaned untracked files.

This is not the first time something like this has happened. During a few releases earlier, one change was stuck in a recursive loop because it involved recursively updating submodules.

Is it that I am not able to think and reason about multiple parameters at a single point in time due to lower IQ, or is my brain occupied, or do I not have the proper mindset for unit testing? I don’t know.

[calm juniper]

I wanted to know what exactly unit test brings to the table. Is it just to check if every line is doing what it is expected to do or does it have the potential to flag issues like mine?

[proud nebula]

ok, there's nothing unit testing will do that is relevant to that as far as I can see. The problem here is that git submodules is not something you should be using probably. It's a quite bad system for... well.. anything. It's notorious for being hard to use and surprising.

We had submodules at a previous job. ENDLESS problems. We removed them and were much happier.

[jolly moat]

Did you replace git submodules with something else?

[dense bough]

Feels like a problem to be solved with code review rather than unit tests.

[proud nebula]

Copy pasted the relevant code into the source tree.

[calm juniper]

@proud nebula please suggest a book on unit test , maybe a philosophical book

[proud nebula]

(which meant the two products using the shared code got forks and diverged over time, but that was for the better really)

[jolly moat]

I see, thank you

[proud nebula]

I'm more of the "bitter personal experience" type of person :P

There's a lot of books on this topic that are... let's say.. religious, or fanatic

[calm juniper]

Not a good idea, a different team is working on those submodules and there is a massive influx of changes.

Btw submodules approach is okay i guess, as we don't really have other full proof dependency management systems for cpp code i don't know and I don't want to know.

[calm juniper]

🙈🙈

[calm juniper]

Okay, no problem, If other senior folks read this message, please do suggest

[proud nebula]

Sounds like a dangerous situation honestly. External dependencies should be managed as any other dependency: like a versioned package system that you upgrade in a controlled manner.

[calm juniper]

Ohh, these are calls made by seniors, I am a junior engineer. 🙈 Please suggest resources and books 😔

Btw, I countered your approach of copying pasting code.

I am not against dependency management systems

[dense bough]

Do they check your code before merging?

Not sure reading helps here. Like, what you needed in that scenario was experience to know what was related. Can't account for things you don't know about yet.

[proud nebula]

Maybe your problem was just that you didn't ask for help/feedback. I've seen that a LOT in juniors.

Many jr devs are afraid of looking stupid more than they are afraid of not learning stuff.

https://www.zainrizvi.io/blog/the-impostors-advantage/ should be mandatory reading before you can do any programming :P

[calm juniper]

Thanks for this 🤗

[calm juniper]

Yeah, but I don't want to blame anyone

[calm juniper]

I knew it right, git clean -fxd will clean untraced files.

[calm juniper]

As people are going to wake up after 2-3 hours, do suggest resources and books.

[calm juniper]

Thanks again sir🧑‍🎓

[fickle mural]

I might have just come up with a ... very good idea for mocking in general to avoid the "where to patch" problem...

def patchable(f):
  @wraps(f)
  def wrapped(*args, **kwargs):
    return wrapped.inner(*args, **kwargs)
  wrapped.inner = f
  return wrapped

Idea being that then you could patch foo.bar.inner and then even if foo.bar is imported into some other modules, the name patching "works" because the function would have been transformed into just calling the inner val.

Is there something like this in the standard library anywhere?

[hexed cloak]

@wraps already sets __wrapped__

[proud nebula]

That doesn't really seem workable. You would have to convince 100% of all maintainers of all packages that you might want to patch to add that decorator. Anything less means the people writing tests needs to learn how to do it the current way anyway.

[fickle mural]

My thing isn’t really about mocking third party stuff so much as mocking my own stuff without doing an annoying “_inner” auxiliary definition myself.

[elder lodge]

Are you a test develop of software

[fickle mural]

Like I’m not trying to absolve people of the need to “learn how to patch things” (though I think it’s hard to say the current mocker is ideal… obviously import style differences are incidental in almost all cases we’re talking about mocking). Just most of the time I’m mocking code I control

My thing is function-specific though… I probably just want a generalized proxy object or something

[proud nebula]

what?

[proud nebula]

Hmm.. maybe if it's your own stuff you could just not mock and instead do dependency injection. My rule for mocking is to avoid it at almost all costs.

[fickle mural]

I suppose so yeah, mocking is nice for ad-hoc DI when I don’t exactly need DI outside of tests

[buoyant valley]

Hi All, I'm learning Unit testing in python. I have a testable code in src directory, and i have created another directory tests . when i import function from src directory, and run the tests, its giving me ModuleNotFoundError. I have init.py files in both src and tests directories. I couldnt find anything online. Can anybody please help me with this?

[proud nebula]

You need to make sure the src dir is in the python path. Personally I don't like having a src dir for this reason.

[buoyant valley]

should we define the python path at the project level?

[proud nebula]

I don't remember how people who use src dirs do this since I don't. You could look at some project that does this and see how they do it.

[buoyant valley]

Thanks!!

[maiden pawn]

the laziest approach is using flat module structure approach

u just import code in tests as from app_name.module_name import your thing

treat app_name as module too

for that u just need having folder structure like

app_name/
   __init__.py
tests
   __init__.py

NO_INIT_AT_ROOT

and then u call tests with command like pytest or whatever unittest autodiscovery is called

app_name can be src i think too (at least if it is not reserved name 🤔 not sure if it is not reserved )

[proud nebula]

👆

[uneven beacon]

this might not be relevant to you but if you're developing an installable package, you'd normally do an editable install of your project during development to add your source code to sys.path, for example: py project/ ├── .venv/ ├── src/ │ └── mypackage/ │ ├── __init__.py │ └── ... ├── tests/ │ └── test_xyz.py │ from mypackage import xyz └── pyproject.toml bare minimum for pyproject.toml: ```toml
[build-system]
requires = ["setuptools"]
build-backend = "setuptools.build_meta"

[project]
name = "my-project"
version = "1.0.0"usage:sh
(.venv) /project $ pip install -e .
(.venv) /project $ pytest # or whatever you want to run```

[drifting sorrel]

Do you use Poetry, Hatch or any package & dependency management tool? Usually, these things with pythonpath are resolved by the tools.

Otherwise, you might need to tell pytest about where the top namespace is (a pytest configuration).

https://docs.pytest.org/en/stable/reference/customize.html

I think using a src is fine, and isn’t usually a problem for test tools. There are scenarios where having code in folders like this makes sense, such as monorepos.

[proud nebula]

<@&831776746206265384>

[split field]

they install the module before running the tests, with a pip install .

The point of the src/ layout is to make it so that the module won't work unless it's been installed, which in turn means that your tests are testing the installation configuration (albeit indirectly) and ensuring you haven't failed to include some important files in the package

[split field]

in other words, the fact that a package's test suite won't find files in src/ isn't some shortcoming of the src/ layout that users need to work around, it's the entire motivation for choosing to use the src/ layout

[proud nebula]

Thanks. One day I hope I will remember this 🤣

[fiery arrow]

I've seen people doing some cargo-culting and calling their top-level package src 💀 (and possibly using it as a package, sometimes a pep420 one)

from src.foo import bar

[split field]

yeah, I've definitely seen that particular cargo cult 😔

[drifting sorrel]

Unless the organization name in fact is src, it’s likely something wrong with the Python path setup 😄

[hollow galleon]

thoughts on function names?

[jolly moat]

It's not obvious to me what does "with_valid_params" mean here in the first test function, in all the other functions it's very obvious just from the names
(I'm a noob so take this with a grain of salt)

[glacial hollow]

"with valid parameters" I guess

[proper wind]

Hello, I am curious about testing methods that are fetching data from external api's. Let's say i have method that fetches some data from api. Should I mock return value? Or make real network call ?

[proud nebula]

Maybe both. It will depend on the situation. You could have a test you run rarely to check that the API doesn't change, and then mocked (or better yet dependency injection) test that assumes the response so that it's fast and you can run it in CI etc.

[split field]

If you only do mocks, you never actually test that you're using the network library correctly, and can successfully make network calls. And if you never do mocks and only make real network calls, it's very tough to exercise error handling paths, because you can't reasonably make network calls fail. You almost certainly need some tests that mock out the network call ("unit tests") and some tests that make the network call ("integration tests") in order to do a good job of exercising the code.

[hexed cloak]

You might like pytest-recording

[sudden mauve]

Unit test library can also be used

[cloud folio]

I am using pytest-django and django for the first time. Pytest-django is installed via poetry.
This is the config of pytest in the ,toml file

[tool.pytest.ini_options]
DJANGO_SETTINGS_MODULE = "arboretum.settings"
# -- recommended but optional:
python_files = ["test_*.py", "*_test.py", "testing/python/*.py"]

the first app of arboretum is "data_import/tests/i;porters/address_csv_test.py"

import pytest

from data_import.importers.address_csv import AddressCsv


def fake_test():
    assert True

when I run pytest it immediatly ends with "no tests ran''

How do I make it work ?

It's a django apps

[river pilot]

change the name of the function to test_fake

[cloud folio]

that's it. Did not realized the test itself had this constraint. Thx !

[analog basin]

So, where do y'all stand on "mock everything" vs "mock only expensive methods"?

It seems there are myriad opinions on this and I'm having trouble picking a path forward

[maiden pawn]

i stand on => by default we don't mock anything when we unit test backend services
We raise local postgresql instance and run all queries in tests against it.
We mock/patch only third party api services which we can't raise in docker containers locally, reproduce locally somehow at all

if i have an option available to run my tests using a "testing api client" i could be considering to run them by default because i am lazy
and inject this key into CI to run from it.
If i am slightly less crazy i could be hiding those "integration" tests behind @pytest.mark.integration and calling from CI only after i already injected once again my testing api key

TLDR:... we need to mock by default only third party services that do n't have "testing" mode somehow

but even in this case i could consider abusal to use key in read only access (if it exists) and run tests anyway...

[analog basin]

I feel like hitting a test database would push a test into integration territory rather than unit

[maiden pawn]

if i plan the app being very large...

...i would consider using unit test parallel run from an early start (adopting it earlier should be easier)

...and may be even configuring visual profiler to eliminate most expensive methods at some point

[maiden pawn]

as long as database is runnable locally, it is controlled dependency
Considering how relational db can plague more than half of backend logic, mocking is kind of absurd due to creating kind of useless testing that misses most important code checking (fragile code of interacting with db)

if u want to make bad unit tests for backend => mock db or don't test bd at all :/

[proud nebula]

My opinion is to avoid mocking as far as possible. It is very easy to fool yourself with mocks, and produce brittle and useless tests.

[weary quarry]

Another way to look at mocking is governed by a simple guide: Don't mock what you don't own.

For the database example, it should be common enough to have the crud operations to your database isolated into their own provider module. That module can't mock the database when writing tests, you don't own the databases's behavior. So a lightweight test database is needed to assert the behavior of your crud operations. Bonus if you also cover known constraints, though make those their own test as constraints can change over time and the next dev will thank you for small, easily understood tests.

For HTTP work, I actually follow the same pattern. The http provider holds all the base methods (get, post, put, delete, etc). Tests will either make actual calls to smoke test endpoints if they are available or we'll throw a lightweight flask app up next to the tests and make some calls. We're, again, testing behavior of these methods. How do you handle a success, how do you handle a failure, does back-off logic work, and does retry logic work the way we expect?

Once outside of those modules the situation can change. We have the option of mocking our own database provider with an expected response from the database in order to test a controller doing some work. We own the provider, and we've tested the provider, so it's safer to mock. Although, we also have the framework for a test database already so sometimes it's just as easy to use that and skip the mock fixture. ;)

For the http calls, I almost always mock the response once outside of the provider. My tests will never be hitting the production endpoint so at some level or another I'm giving the test a fabricated, expected response body. But, again, this is supported by the fact that I'm not actually mocking the http calls at all. I'm mocking something I own that, itself, has been fully tested.

[manic belfry]

Instead of writing a fixture per role, can I make a single fixture and call it as necessary?

@pytest.fixture()
def viewer_client():
    user = User.query.filter_by(role="viewer").first()
    with app.test_client(user=user) as client:
        return client


@pytest.fixture()
def mod_client():
    user = User.query.filter_by(role="moderator").first()
    with app.test_client(user=user) as client:
        return client


@pytest.fixture()
def admin_client():
    user = User.query.filter_by(role="admin").first()
    with app.test_client(user=user) as client:
        return client

Can I instead do it in a way that avoids duplicating code?

@pytest.fixture()
def dynamic_client(role):
    user = User.query.filter_by(role=role).first()
    with app.test_client(user=user) as client:
        return client

then do something like this in test_something.py?

def some_test(dynamic_client):
    client = dynamic_client("viewer")
    ...

[haughty pine]

you might wanna use markers or factories

[manic belfry]

Okay, factory seems pretty simple but I cannot understand how to apply the marker pattern

[haughty pine]

you'd decorate the tests with some marker with the type of user you want, and then, in the fixture code, retrieve that requested type of user from the marker

[river pilot]

do you need fixtures at all? You can write functions and call them from your tests.

[manic belfry]

I did not think about that 🤔

I'll try that

[calm juniper]

no we have to mock for unittest , you can do it with integration test. and 2 people have upvoted why? why did he ask this question in first place(I am not being rude this is the place to ask these question i know ), I guess unittest simply means testing your code flow that's it and mock is very light way to counter those expensive operations as we are testing the code flow.

[proud nebula]

"Need" is a strong word imo. It does also depend on what you mean by "mock".

[calm juniper]

peace out , your opinion seems tangential to me. 🙊 😨

[proud nebula]

You can pass an argument into a function instead of using the mocking library for example. That's what I mean by it depends on what you mean by "mocking".

Mocking with the mocking lib is rather magical and is action at a distance. If you do it to mock a requests call and the implementation changes to httpx maybe your tests now hit production or something nasty like that.

[calm juniper]

got your point, understood, sorry,but these problem are very subjective, if you're porting things from requests to httpx port your test as well , its pretty common to change your unittest according to code. Mocking is very handy and amazing way to test code flow, ask people who are doing integration test in their CI, I guess intergation tests are not for code verification, it's should be after merge. post merge, again mocking is very very very handy tool. and with due respect you have to do mocking for unittest if you want to spawn test db and all do it in intergation test. Btw the guy who has defended you have profile picture from preety famous book , i don't know why he is against mocking.

btw, i have recently explored these topics so i am open to any kind of feedback

[maiden pawn]

Consider if you are very interested regarding this topic in general reading Best Practices for unit testing 😉
https://darklab8.github.io/blog/favourite.html#UnitTestingPrinciplesPracticesandPatterns

i liked this book

they touch the topic of mocking in intensive way of course too, when it is justified and when it is not

[calm juniper]

Sorry for bringing my reddit mindset here, he said you should avoid mocking as much as possible. 😭😭Why people give unsolicited advice in tech world. 😭😭

[calm juniper]

Thanks, I will surely read it

Actually he was the one who countered my submodules approach with copy pasting the code. Really 😨 this is how you are teaching people.

[maiden pawn]

and i agree with this opinion too 😉 Mocking should be avoided for the tests to be more useful.
Or at least Mocking should be avoided as long as dependency you test is locally raisable under your own control, and tests you write can be written in predictable way where all dependencies are frozen and the same test invocation will be able to pass the tests a year later with very high chance or minimum effort.
TLDR: As long as dependencies are under your full control and you can rewipe their state cleanly between tests at a zero money cost and they do not depend on some cloud remote functionality, it should not be mocked

[calm juniper]

Then, don't write a unit test, simply write the integration test . Btw, maybe there is a fine line between these two i don't know(in current tech world )

[maiden pawn]

ergh, definition of what is integration test is not very strict.
For me integration testing begins at the level where u use stuff like Robot Framework in python to raise application with lets say wsgi server and have veeeeeery hard time to mock things / replacing things on a run in memory.
Or when u run tests against remote cloud located usually single dependency (quering AWS objects states for example, or invoking endpoints of a payment gateway provider)

[calm juniper]

maybe we should pin this message, please pin it.

[proud nebula]

I think people are here to learn. I want "unsolicited" advice. Because it's not unsolicited. I'm here, which implies I want it.

Anyway. I didn't say never use it. I said avoid it if possible. And there are many ways to avoid it. Making cleaner code that has more logic in pure functions for example. I think you agree with that.

[proud nebula]

Context matters. I don't remember the context so I can't remember what I said and why :)

[calm juniper]

No problem, you shared an amazing article, i am not being grateful here.

[calm juniper]

I can totally understand.

[proud nebula]

You mean ungrateful?

[calm juniper]

yes

grateful to you

leave it man

[drifting sorrel]

There's also a light weight approach to mocking/using fakes: monkeypatching. This is my preferred way when there is a need to control any I/O and when not using dependency injection and such.

example:

# this is the production code
# wrapping the I/O call in a function
def call_the_external_service(data) -> dict:
    ...
    # doing an http call with my favorite library, returning the json/dict response

# this is the test
def test_something(monkeypatch):
    # the code to be tested is using the call_service function.
    monkeypatch.setattr(the_module, "call_the_external_service", lambda *args, **kwargs: the_fake_response)
    ...
    # assert the things

[proud nebula]

I'm pretty sure mocking is implemented with monkey patching

[drifting sorrel]

Probably!

I see the monkeypatch as a better way than patching code "manually" in a test, because it is context aware.

[proud nebula]

The way you showed above doesn't clean up the monkey patch though.. that seems very bad?

[drifting sorrel]

pytest does that automagically.

[proud nebula]

ah, my bad

[torpid osprey]

If any of you people are doing tests and have one db connection for all the tests and you're sending requests to a FastAPI app locally and you delete an object with an http request and then query for the deleted object and suddenly find that an object is returned, it means that you have to refresh your session because it's not in sync with the changes made by a concurrent request by the db session within your fastapi app.

you can do that by calling: py await db.close() await db.connection()

It took me hours to figure that out and I want to save you all some time.

[drifting sorrel]

Maybe it is done when closing, but isn’t there a db.commit() (or a with session) missing in the production code?

[torpid osprey]

There was. The overall solution was to override the fastapi db dependency wtih a function that returns the test db so everything will be in sync.

[crimson frigate]

Hey, in my project I have a module constants.py this is part of it:

load_dotenv(override=True)

FOCUSTUI_DEBUG: bool = os.getenv("FOCUSTUI_DEBUG") == "True"

_minute = os.getenv("FOCUSTUI_DEBUG_MINUTE")
is_custom = FOCUSTUI_DEBUG and _minute is not None
MINUTE: int = int(_minute) if is_custom else 60

When the user runs the app, everything is set to default. But when I work on project debug and other settings are custom. For testing purpose, I wanted to set up debug to False automatically. Because when I import some constants they are already calculated, so I have to change debug before running the test.

[proud nebula]

You forgot a question

[crimson frigate]

XD How to change var env before initialization of module that I import?

[proud nebula]

Shouldn't you change the debug flag after? Like in the settings for tests?

[drifting sorrel]

Yes, it is "too late" to change the env variable in the test because the module has already loaded it. If it were loaded in a function and not executed before the changes in the test, then it would work, though. As @proud nebula says, the solution is to change the actual debug-flag variable.

[crimson frigate]

Do you mean turn the variables into functions?

[crimson frigate]

Do you mean before testing, go to .env change to false and run test?

[proud nebula]

er, no, absolutely never do that. I mean in your settings.py that you have for tests, you set DEBUG=False. You do have a specific settings file for tests right?

[crimson frigate]

what do you mean exactly

[proud nebula]

I mean, that when I run prod it points to dryft.settings, but when I run tests it points to tests.settings. Those are different settings.py files.

(dryft being the name of the company)

[crimson frigate]

It has a lot of sense. Now it reminds Django cookie cutter doing it, for example. How should I structure it?

Simplified tree

.
├── pyproject.toml
├── src
│   └── focustui
│       ├── config_manager.py
│       ├── constants.py
└── tests
    ├── __init__.py
    ├── test_config_manager.py

[proud nebula]

Don't you already have this? Surely you do!

[crimson frigate]

What do you mean 😅 I do no have more than this because it is a Textual project built from scratch. I guess I just should create constants.py inside tests and only write their strict values.

[proud nebula]

No I mean, do you have any tests at all right now?

[crimson frigate]

I do, this is why I asked because when I turn on debug some of them do not pass

[proud nebula]

So surely you already have a test settings file? Otherwise your tests would destroy your real db.

[crimson frigate]

Now I see your point. I didn't have because I didn't start testing DB yet. Sounds are just files on the disk and I was not touching real ones just Mocking etc

[proud nebula]

aha

[crimson frigate]

😅 do you have any advice on how to approach this problem?

[proud nebula]

I realized I just assumed this was a Django project. I forgot which discord I was answering this on. Is it Django?

[crimson frigate]

No problem, I am building UI app with Textual. In constants.py I have those who depend on Debug to make some parts of manual UI testing much faster.

[drifting sorrel]

I meant that it looks like the debug flag is set immediately when the module is loaded, and then it is probably too late to change the actual environment variable in the test. Pytest, as an example, has the monkeypatch feature that can .setenv("the env", "a value") but that won't work if the flag already has read from the existing environment.

A solution could be to patch the debug flag, or use a test-specific environment/settings file that the module will read from.

[crimson frigate]

I studied what you have said in part 1 yesterday and realized that you can reinitialize python module which would solve the problem.

How would you patch the flag?
I thought there is to set PyTest to test variable before running the test to override what is in .env file.

[drifting sorrel]

Maybe pytest-env (a pytest plugin) could help? With it, you can set test-specific environment variables in the config: https://github.com/pytest-dev/pytest-env

[crimson frigate]

I have already set it and started reading docs but I still can figure it out

[tool.pytest.ini_options]
python_files = "test_*.py"
env = [
    "FOCUSTUI_DEBUG=False"
]

is it not correct?

[drifting sorrel]

If that is from the pyproject.toml I think you need to prefix the first table with [tool.pytest]

What happens if you prefix the pytest command? FOCUSTUI_DEBUG=False pytest

[crimson frigate]

Any configuration like

FOCUSTUI_DEBUG = "False"

does not work...

[crimson frigate]

fails

[drifting sorrel]

Is it a "bool from string" problem?

the os.getenv in your code doesn't convert the data into a bool as it looks like. So anything in the string would evaluate to True in Python.

[crimson frigate]

Nope - part from code where it is setup

load_dotenv(override=True)
FOCUSTUI_DEBUG: bool = os.getenv("FOCUSTUI_DEBUG") == "True"

[drifting sorrel]

How does your production code environment file look like?

Do you mean that the flag is always set to True regardless of your pytest config?

[crimson frigate]

I am blind I have it!

load_dotenv(override=True) - override=True - I setup it before and in this case this is the problem - docs: Whether to override the system environment variables with the variables from the .env file.

But thank you @drifting sorrel and @proud nebula for the support!

[sick dust]

how do people typically set up their tests folder structure? Do you go for (a) creating new test files for each "topic" or (b) get the structure of the tests folder to match the package structure? I was looking at the code for xarray (a) and pandas (b'ish, but not sure if its 100%).

[maiden pawn]

i think it is good default... not creating separate tests folder.
when u have file package/code.py
create package/code_test.py to test it 😁

in python cases it can be also okay package/sub_package/stuff , having tested by package/sub_package/tests.py

than closer tests to the code they test, then they are easier to find them and structure imo

[proud nebula]

I think you should start with a tests folder.

With iommi we've gone a bit crazy and put the tests for foo.py in foo__tests.py in the same folder. It makes mutation testing WAY easier and makes it easy to find tests.. but it's very non-standard and a bit weird honestly.

[drifting sorrel]

I would go for having tests in a separate test folder. It can be one global test at the root, or if you devlop smaller components it could be something like:

the_feature
  /src
  /test

The above is less common than having the tests "globally":

the_code/
test/

# or plural: tests/

[maiden pawn]

@proud nebula i would say it a bit depends on a type of application.
foo.py and foo_test.py is very good for libraries at least.

for regular backend apps in Django way, obviously good way per Django sub app to put tests
for flask is obviously per blueprint 🤔

[proud nebula]

Yea. I also wanted to do mutation testing, so it was important for me to be able to cleanly and easily select smaller amounts of tests to run for each mutant.

[spare flax]

Hi there,

I'm new to testing and looking for advice on testing frameworks. I'm working with an embedded system that interfaces with Python to read/write registers. The system has additional complexity due to multiple revisions of firmware and devices. Meaning the device is read first to know how to interface with it and what it is capable of.

Setup:
Host (where the Python script runs) <--SPI/I2C--> MCU

My current testing script I wrote feels cumbersome for what I'm trying to achieve. Here's a sample of my current setup:

run_test_case("HIGH", test_logic_high, "GPIO set to HIGH", "piGPIO HIGH")

def test_logic_high(rpi_input_state, rpi_pin):
    return rpi_input_state == 1

u48_case_results_all = []

def run_test_case(function_name, test_logic, action_description, test_description, setup=None, **kwargs):
    u48_case_results = []
    device_gpios = u48_GPIOControls_shadow.set_gpios_to_function(function_name)
    
    if setup is not None:
        setup()

    for device_gpio in device_gpios:
        u48_case_result = {}
        rpi_pin = test_pins[device_gpio]
        GPIO.setup(rpi_pin, GPIO.IN, pull_up_down=GPIO.PUD_DOWN)
        rpi_input_state = GPIO.input(rpi_pin)

        # Specify the action taken
        u48_case_result["Action"] = action_description
        u48_case_result["Expected"] = test_description
        u48_case_result["axGPIO"] = device_gpio

        # Conditions to check and determine pass/fail
        case_result = '[PASS]' if test_logic(rpi_input_state, rpi_pin, **kwargs) else '[FAIL]'
        u48_case_result["Result"] = case_result
        u48_case_results.append(u48_case_result)
    u48_case_results_all.append(u48_case_results)
    return u48_case_results```

Pytests seems to offer a lot of features that I like however I'm struggling with getting a starting point. 

Any guidance or recommendations would be greatly appreciated!

[analog basin]

So the first thing is, I would run these with pytest rather than running them as a regular python script. Are you using an IDE?

For checking multiple test conditions under the same logic, you can parameterize your test to check different combinations of input/output
https://docs.pytest.org/en/7.1.x/how-to/parametrize.html#parametrize-basics

[spare flax]

Thank you for your reply, pytest does seem like the better approach. I am developing on VS code. I think I am confusing myself with how to implement the parametrization.

So in my code I have device_gpios = u48_GPIOControls_shadow.set_gpios_to_function(function_name)
which returns a list of device gpios with the specific function capability. So I don't have to manually say that there are x gpios.

With pytest I would do something like
@pytest.mark.parametrize("gpio_state", ["HIGH"])
def test_if_high(gpio, gpio_state):

then some other code that checks / returns the list?

[analog basin]

Hmm, could you explain what it is you're testing?

[ruby bobcat]

hey i need some help in debugging

can anyone help me?

[hexed cloak]

Paste your code into a #❓｜how-to-get-help channel

[proud nebula]

Ask the question

[ruby bobcat]

dms

messege me in my dms

[proud nebula]

No. Ask in public like everyone else.

[lilac ferry]

Hi everyone! I'm working on a personal project to learn sockets with python. At the moment I'm confident with the code I wrote for the server (game manager) and my client (players). However based on past experience confidence sometimes takes its toll when delivering projects and there's nothing better that an actual proof that tests can give you. Anyway, I've looked all around the web for integration tests but there are just few articles than mention it briefly. Why I'm looking for an integration test instead of unit tests? Well my code is based on the same server client connection you easily find in the internet, and I tested that manually to see the magic working on screen. However, I'd love to check game state changes, number of players connected and player turns to name a few to be sure that the game loop works beautifully.

A server and a client are instantiated in the following code. If I want to use setUp() I might get a server and a client running before each test which is good but both classes have an infinite loop that can be stopped with ctrl+C hotkey in the real world so I'm not able to shut them down with tearDown(). That's why I chose to instantiate them outside of the Testing class and run the specific couritines that wait for messages from either server or client. This is where I'm stuck, mocking? Not an option because again I want to have the whole flow running to test changes in the game state. I'd like to stop writing now to make this short and not a block of text. However, I'm always open to add details to this questions if you guys find it necessary.

server_client_integration

import unittest
import asyncio
from server.game_manager import GameManager
from client.player_controller import PlayerController
from server.server import Server
from client.client import Client


server = Server()
player_one = Client()
player_two = Client()

class TestSCIntegration(unittest.TestCase):
    def setUp(self):
        asyncio.run(server.server_loop())
        asyncio.run(player_one.game_loop())
    def test_number_connected_clients(self):
        self.assertEqual(server.thread_count,2)

server

import json
import asyncio 
import socket
from .game_manager import GameManager
from _thread import *

class Server:
    def __init__(self):
        self.host="127.0.0.1"
        self.port = 1233
        self.thread_count = 1
        self.socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
        self.game_manager =  GameManager()
        self.connect()

    def connect(self):
        try:
            self.socket.bind((self.host, self.port))
        except socket.error as e:
            print(str(e))

        print("Waiting for connection")
        self.socket.listen(2)

    def threaded(self, connection):
        if self.thread_count == 1:
            who = "player_one"
        else:
            who = "player_two"
        settings = {"who": who, "game_state": self.game_manager.get_game_state()}
        to_json = json.dumps(settings)
        connection.send(str.encode(to_json))
        while True:
            data = connection.recv(1024)
            if not data:
                print("Connection closed")
                break
            self.game_manager.set_game_state(data.decode("utf-8"))
            reply = self.game_manager.get_game_state()
            connection.send(str.encode(reply))
        connection.close()


    async def server_loop(self):
        while True:
            connection, addr = self.socket.accept()
            print("Connected to:", addr[0], ":", addr[1])
            start_new_thread(self.threaded, (connection, ))
            self.thread_count+=1
        self.socket.close()

if __name__=="__main__":
    server = Server()
    asyncio.run(server.server_loop())

client

import asyncio
import socket
import json
from client.player_controller import PlayerController

class Client:
    def __init__(self):
        self.host = "127.0.0.1"
        self.port = 1233
        self.socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
        self.connect()

    def connect(self):
        try:
            self.socket.connect((self.host, self.port))
        except socket.error as e:
            print(str(e))
        response = self.socket.recv(1024)
        data = response.decode("utf-8")
        settings = json.loads(data)
        self.id = settings["who"]
        self.player_controller = PlayerController(settings["game_state"],self.id) 

    async def game_loop(self):
        while True:
            await self.player_controller.end_turn.wait()
            state_to_server = str.encode(self.player_controller.get_game_state())
            self.socket.send(state_to_server)
            response = self.socket.recv(1024)
            data = response.decode("utf-8")
            new_state = json.loads(data)
            self.player_controller.set_game_state(new_state)
        self.socket.close()

if __name__=="__main__":
    client = Client()
    asyncio.run(client.game_loop())

[hexed cloak]

Why are you using blocking sockets with asyncio?

You should be using https://docs.python.org/3/library/asyncio-stream.html#asyncio.open_connection

And https://docs.python.org/3/library/asyncio-stream.html#asyncio.start_server

[lilac ferry]

Because I'm a beginner haha :), jocking! In a more serious tone, I've coded on python for just four months and read a couple of tutorials about sockets. Didn't know about streams

[hexed cloak]

Then you can run a server and client concurrently using a TaskGroup

[lilac ferry]

I've seen something about TaskGroup and coroutines

[hexed cloak]

You should decide if you're using threads and blocking sockets or asyncio, not both at the same time like this

[lilac ferry]

Hmmm can I ask you why? I'd like to learn

[hexed cloak]

Because blocking sockets like this prevent you from running your client and server concurrently using a TaskGroup

[lilac ferry]

Right, I think I understand you

[hexed cloak]

Also don't use import *

[lilac ferry]

Question though: Can I implement game states changes events like I did to wait for end turn?

[lilac ferry]

I found it silly but the tutorial pointed that out and I wanted something working to start with

[hexed cloak]

And don't use _thread either

Use threading.Thread

[lilac ferry]

Why? I'd like to learn 🙂

[hexed cloak]

_thread is low level API for managing threads and shouldn't be used by end user applications

[lilac ferry]

Alright, that makes sense

[lilac ferry]

Thanks a lot for taking time to read my question. Based on my code and the idea behind the project, do you feel there's something else I should check out/review?

[hexed cloak]

Probably anyio's pytest plug-in

It will let you use an async fixture for your server

[lilac ferry]

Cool, I'll try to stick with unittest cause I'm importing libraries as few as possible (currently just relying on textual for the UI) but if I'm really stuck with testing I'll try your suggestion

[hexed cloak]

Also the way you're using socket.send and socket.recv is wrong and doesn't handle fragmentation

For blocking sockets you need to use https://docs.python.org/3/library/socket.html#socket.socket.makefile

[lilac ferry]

Hang on, if I wanted to transfer my game state back and forth, should I make I file from that dictionary (game state stored in a dictionary of course)?

[hexed cloak]

Back and forth?

[lilac ferry]

Yep, client (player) makes a decision hence changes the game state and the server (game manager) determines if he/she is the winner. It's a black jack game

[lilac ferry]

Alright @hexed cloak, thanks again for your help. I'll spend some time reading before changing my code. Cheers!

[hexed cloak]

Probably you don't want a file

[lilac ferry]

Not really 😉

[manic belfry]

How would I go about loading project specific env vars for pytest?

Some of my tests need to authenticate so I need to keep the credentials out of git

[proud nebula]

I guess this would depend on the situation. Is the question about how to do this on Github Actions for example? Or just locally?

[manic belfry]

It would be nice if I could securely do this on GitHub actions too, but the question was solely in local context

[proud nebula]

Locally I'd probably recommend an .env file and having code to load from that

[manic belfry]

So python-dotenv I guess?

[proud nebula]

Don't know, I've personally never needed this :P

[manic belfry]

me neither, but now I do lol

Following on the previous question

How would I do the same but for GitHub Actions?

Certainly don't want to leak credentials

[proud nebula]

They have some system where you can encrypt secrets with githubs public key so GHA can decrypt it, but no one else can.

[manic belfry]

that sounds fancy

Will have to do a bit of reading to figure this out

[proud nebula]

A bit. Standard public key cryptography, but certainly handy. They also have a secrets system, where you get stuff as environment variables.

Maybe that's more reasonable here actually, since it's not that big.

[manic belfry]

Are the secrets accessible as env vars?

I see GitHub has both "secrets" and "environment variables" as two seperate things

[proud nebula]

I think so. Was a while ago I did anything with that stuff.

In their GUI the secrets are write only. Once saved you can't read them back.

[manic belfry]

makes sense

Thank you, I'll be reading up on these and hopefully figure something out

[hexed cloak]

That's how Travis secrets worked. With GHA you just add them in settings

[manic belfry]

I have a matrix of tests for 3.10-3.12, but there's an integration test that locks the endpoint it's connecting for it's duration

this means if 3.10 job is running said integration test, the other jobs will fail trying to run the same test until 3.10 is done

I can think of a few solutions here:

• Run 1 job at a time (slow)
• Randomize the order (small chance of still happening?)

[pearl cliff]

Is this a test server you are starting for the test suite run?

[manic belfry]

I'm connecting to the prod server for this test

[pearl cliff]

Oh my

It doesn't lock it for actual prod users right?

[manic belfry]

prod users cannot access the endpoint yea

so it's internal only

[drifting sorrel]

I'm curious about the setup: is it an API endpoint that is tested? How come there are tests for different python versions, do you have several endpoints running (with different Python versions)?

[proud nebula]

My first instincts here would be to not do that. Why are you doing something so seemingly dangerous in a test?

[manic belfry]

I plan to replace it with a test server soon enough but that's still a WIP so I started on the tests while that gets done. It's not a permanent setup

I'll just skip these tests till I get the test server up

Thanks for the insight 🙂

[proud nebula]

A test server also seems like a mistake. Why do you need to go over the network at all for a test?

[manic belfry]

hmm, how else should I test an API end point? Mock it?

[proud nebula]

It's an external system? Not part of the product?

[manic belfry]

yes, another system that's supposed to interact with the product via it's api

[pearl cliff]

They said above that this is an integration test

[proud nebula]

Terms are hard. I believe this might be people using different definitions.

[manic belfry]

English is not my native language so I apologise for any confusion caused by inappropriate usage of technical terms

[proud nebula]

In that case I would think of such a cross-product test as a smoketest and something that runs on a schedule separate from the normal tests. And yes, the product you are working on might be better off with mocking. The smoketest would then verify that the mock and reality are the same.

[proud nebula]

No problem. Teams use different definitions for these terms even if they are native English speakers. They are just not well defined, nor universally agreed upon.

Just be aware that using a term might not be enough. You might need to make sure both sides think they mean the same thing when using a term.

This problem never goes away :)

[manic belfry]

That sounds like an excellent way to go about it, thank you!

[manic belfry]

haha yea, terms are hard

[proud nebula]

Just make super super sure that they are the same. They should not be copy pastes, or written the same. They must use the exact same data structure so it's not possible to have them out of sync.

[sick dust]

Does anyone have advice on how to best create utils for your test suite when working with Pytest and shipping a package? Any advice would be appreciated (I created a help thread) https://discord.com/channels/267624335836053506/1284125260857475164 🙂

[valid belfry]

I’m a long time ruby developer, and I’ve been working in a large python codebase the last couple years. One major difference (and to me, smell), is an over reliance on mocks to test. I’ve often seen asserting the number of mock calls, mocking out other method calls to “unit” test a specific method. All of this feels brittle to me, and not all that useful.

I can’t tell if this is a python thing or just a bad practices in this codebase thing.

But my general sense is mocking is more judiciously used in python than it is in ruby. In the ruby world, mocks are mostly used to isolate external dependencies, especially external apis.

Curious on people’s perspective on this, and if I’m reading this correctly. Thanks in advance.

[proud nebula]

I would agree mocking is a last resort type of thing. I would question any mocking strongly before accepting it in a PR.

[valid belfry]

Thank you, this is helpful perspective.

[proud nebula]

I've been on a course on testing where the speaker suggested this mock-everything approach. It sounded great in the room, but an hour later not so much. To me, if you whitebox tests by mocking everything inside a function, you should be doing mutation testing instead.

[valid belfry]

Yea. Over-mocking tends to lead to brittle, ineffective tests. Brittle bc you're always hunting down why some mock broke or didn't get called in the way the test setup thought it should, and ineffective bc you can easily end up situations where the tests should be failing bc you changed something in the underlying behavior but you've got a mock returning the wrong value that makes the test pass

[proud nebula]

Yep. 100% agreed. And you can't even machine verify that you've mocked everything anyway.

If you do mock something, you should have some code that verifies that the mock is correct.

[pearl cliff]

The name "smoke test" is more about just turning the thing on and checking to see if smoke comes out -- I was thinking more of an "end to end" test where you are actually checking functionality in a live system, although ideally it would be a staging environment and not production

[proud nebula]

Yea. You can do that too. But I was thinking doing just the API calls you need and verify that the API response is not changed. That could be radically smaller.

[pearl cliff]

Right

[lusty shadow]

can i test api for fastapi like normal function using pytest ?

[hexed cloak]

Yes

[pearl cliff]

there is an entire page on the fastapi docs site about testing

[lusty shadow]

thank you

[lusty shadow]

thank you

[torpid osprey]

When testing, should business logic be tested separately from application logic since testing application logic indirectly tests business logic?

[river pilot]

i don't understand the difference between the two kinds of logic.

[torpid osprey]

Business logic is how your application acts no matter the environment whether it be a desktop or website. For example, a transaction of money takes the money from a sender's account and puts it in the receiver's account. That's business logic. The application logic is coupled to an external service/environment which uses the business logic to make decisions.

[river pilot]

i would test the business logic separate from any application logic, yes.

[torpid osprey]

Okay. Thanks for the clarification.

[proud nebula]

I think both are business logic :)

[maiden pawn]

the answer depends on what u understand as external service/environment.

your wording presumes that if we use this business logic in Backend framework like Django, it is already coupled means

then the answer in this definition, i think it depends on your code quality standards and even programming language capabilities

and also... if there is a point to make it separated like this, sounds like overengineering for simple app

separating business logic like this to be splitted from backend framework / relational db stuff, sounds like a lot of extra work to me, i would not go unless there was a clear point why

and u would have hard time also splitting it may be if u will be doing it in python, because this asks for a lot of extra... effort, with things like interfaces and validating correct code connection usage to business logic through static typing.

in language like golang, java, C#, it would be reasonably possible to separate logic like, but in python made for quick doing job done, it would be in 98% cases better going without it, just because u have no language level auto validating to support such level of custom code stuff automatically