#How do I filter properly?

55 messages · Page 1 of 1 (latest)

terse sierra
#

Hi all, I'm trying to filter out intentional 503 errors emitted in one of my views (https://github.com/shipperstack/shipper/blob/master/server/downloads/views.py#L86). Unfortunately I'm still getting error emails sent with this filtering config (https://github.com/shipperstack/shipper/blob/master/server/config/settings.py#L170 and https://github.com/shipperstack/shipper/blob/master/server/config/filters.py#L4).

I tried printing out the record parameter but it doesn't show up in the debug console. I know just filtering out all 503 errors is not ideal, but I can't figure out how to filter by the message only and how to get it to actually work for the emails. Any ideas?

limber kindle
#

are you saying that you've got django set up to send you an email whenever the web site returns a 4xx or 5xx, but you'd like to not send them when it's specifically a 503?

If so, I don't know how, but it's interesting and I'll take a look anyway. Good job that your code is on github or else it'd be too messy to get the code to me

#

hmph, I've cloned it, it's running, and I can't figure out where you told it "yo send emails when there are errors" 😐

terse sierra
#

@limber kindle thanks for looking into it! More specifically, Django has default loggers configured to send emails when the server is running in production mode (DEBUG set to false). I'm trying to set up filters so that it ignores 503 errors, or more specifically 503 errors that are emitted by that download_check endpoint. (I'm just trying to get to the first part first!)

I'm surprised the code ran as is, as it requires some environment variables in order to run fully.

limber kindle
#

It was reasonably clear about which environment variables I needed, so I just created them with dummy values

#

SHIPPER_CSRF_TRUSTED_ORIGINS=https://localhost SHIPPER_ALLOWED_HOSTS=localhost SHIPPER_SECRET_KEY=sekrit poetry run python server/manage.py runserver

#

heh, I see you trying ```py
"filters": {
"ignore_missing_build_503_errors": {
"()": "config.filters.IgnoreMissingBuild503Errors",
}
},
"handlers": {
"mail_admins": {
"level": "ERROR",
"filters": ["ignore_missing_build_503_errors"],
"class": "django.utils.log.AdminEmailHandler",
},
},

terse sierra
#

Yup, those were the ones I tested out. I thought it needed a total path for "()" and changed it, but it didn't take effect.

limber kindle
#

try changing "()": "config.filters.IgnoreMissingBuild503Errors" to "()": config.filters.IgnoreMissingBuild503Errors -- i.e., lose the second set of quotes

#

I say this because I've got a callable filter in my thingy at work, and that's the only difference between what I did and what you're doing

terse sierra
#

Hmm, I already tried importing the filter at the top with from .filter import IgnoreMissingBuild503Errors and then putting just IgnoreMissingBuild503Errors, but I'll try that and see if it works

limber kindle
#

in the meantime: change ```py
class IgnoreMissingBuild503Errors(logging.Filter):
"""
Filters out 503 errors for missing builds.
"""

def filter(self, record):
    status_code = getattr(record, "status_code", None)
    return status_code != 503

topy
class IgnoreMissingBuild503Errors(logging.Filter):
"""
Filters out 503 errors for missing builds.
"""

def filter(self, record):
    5/0 # Bwahahahah!!!
#

hmm, if it's being called, I'd stick a print(vars(record)) in there; I'm not 100% certain the attribute you want is called status_code

#

that attribute is not conventionally present on typical python log records, although I suppose some django middleware might be sticking it in there

terse sierra
#

Yeah, I assumed the filter code only got evaluated once or something because I could never get it to print out the record. On second thought I don't think the filter code is getting called at all :/

limber kindle
#

that'd argue for my "remove the quote marks" advice

terse sierra
#

I tried the invalid code trick and it's not panicking, so I'm guessing something is going wrong with the logging config

limber kindle
#

although ... I'd sort of expect it to work even if it's a string and not a callable

terse sierra
#

Oh, I just tested with the quotes removed, but I did have to import it with the from .filter import line

limber kindle
#

what does "base it off" mean?

terse sierra
#

More specifically adding those lines got it to trigger the bugged code:

    "loggers": {
        "django": {
            "handlers": ["console", "mail_admins"],
            "level": "INFO",
        },
        "django.server": {
            "handlers": ["django.server"],
            "level": "INFO",
            "propagate": False,
        },
    },
terse sierra
#

based off of the docs

#

That Django does some merging of configs when I declare a LOGGING = {} in my own settings.py file

#

By default, the LOGGING setting is merged with Django’s default logging configuration using the following scheme.

If the disable_existing_loggers key in the LOGGING dictConfig is set to True (which is the dictConfig default if the key is missing) then all loggers from the default configuration will be disabled. Disabled loggers are not the same as removed; the logger will still exist, but will silently discard anything logged to it, not even propagating entries to a parent logger. Thus you should be very careful using 'disable_existing_loggers': True; it’s probably not what you want. Instead, you can set disable_existing_loggers to False and redefine some or all of the default loggers; or you can set LOGGING_CONFIG to None and handle logging config yourself.

limber kindle
#

🤔

terse sierra
#

But for some reason, despite not overwriting loggers it was getting removed

#

When I copied it over from the default config then it started to evaluate the bugged filter

limber kindle
#

you know what's awesome?

terse sierra
#

oh cool, what does it do?

limber kindle
#

you just import it, and stick a single call in your code, and it displays the entire logging hierarchy. Helps you figure out this kinda thing

terse sierra
#

oh snap, that looks pretty handy! Will definitely keep it in mind

#

Thank you so much for the troubleshooting help and guidance!

limber kindle
#

if you ever understand why your fix works -- I wouldn't have thought it'd be necessary -- I'd love to hear it

terse sierra
#

I'll push a commit soon that fixed it and link it here

limber kindle
#

yeah I got the email about the commit

#

so what did I actually contribute -- was it removing the quote marks?

#

Or was it just moral support? 🤣

#

ah, it was "status_code"

terse sierra
#

Honestly I was unsure whether the logging configs were evaluated once on-launch or continuously, so you helped a lot in me understanding how Django logging settings work!

limber kindle
#

did you discover that via "print" like I suggested?

#

print is the best debugger

terse sierra
#

Oh, I didn't use status_code, it's not in the LogRecord

#

Apparently LogRecord is just one giant string when I call record.getMessages()

limber kindle
#

that doesn't seem right

terse sierra
#

So I just check whether the URL endpoint and the 503 status code is in there with Python string in

limber kindle
#

that's probably good enough

#

but if it were me I'd poke further: a logging Record is, you know, a record. Pretty much like a dataclass instance -- an object with attributes

limber kindle
terse sierra
terse sierra