#🔒 Need help connecting airflow with slack

35 messages · Page 1 of 1 (latest)

surreal lodge
#

i tried everything as per the documentation and tried out other tutorials also, did everything with the webhooks, i got no error, everything works fine, except i dont get an message on slack, please helps stuck on this for 1 week

covert harnessBOT
#

@surreal lodge

Python help channel opened

Remember to:

  • Ask your Python question, not if you can ask or if there's an expert who can help.
  • Show a code sample as text (rather than a screenshot) and the error message, if you've got one.
  • Explain what you expect to happen and what actually happens.

:warning: Do not pip install anything that isn't related to your question, especially if asked to over DMs.

Closes after a period of inactivity, or when you send !close.
neon cosmos
#

share the details of your code please

surreal lodge
#

the code is quite big how to share?

neon cosmos
#

!code

covert harnessBOT
#
Formatting code on Discord

Here's how to format Python code on Discord:

```py
print('Hello world!')
```

These are backticks, not quotes. Check this out if you can't find the backtick key.

For long code samples, you can use our pastebin.

neon cosmos
#

or

#

!paste

covert harnessBOT
#
Pasting large amounts of code

If your code is too long to fit in a codeblock in Discord, you can paste your code here:
https://paste.pythondiscord.com/

After pasting your code, save it by clicking the Paste! button in the bottom left, or by pressing CTRL + S. After doing that, you will be navigated to the new paste's page. Copy the URL and post it here so others can see it.

neon cosmos
#

but probably only need to share the slack part

#

let me just pull my airflow repo so i can compare

#

i have not written the slack parts of this repo, but it does work

surreal lodge
neon cosmos
#

so i see a huge difference

#

we use SlackAPIPostOperator

#

and not the webhook

surreal lodge
#

can you share some of your code or anyother reference pls

neon cosmos
#

i can share it all, even though im not sure it will help you out

#

do you want to see it?

surreal lodge
#

yea

neon cosmos
# 
#!/usr/bin/python3

from airflow.providers.slack.operators.slack import SlackAPIPostOperator
import os, time

DEFAULT_CHANNELS = ["#airflow-alerts"]


def notify_failure(ctx):
    post_alert(ctx, ":jenkins_red: Failure!", failure_attachment(ctx))

def notify_success(ctx):
    post_alert(ctx, ":jenkins_green: Success!", success_attachment(ctx))

def notify_sla_miss(dag, task_list, blocking_task_list, slas, blocking_tis):
    # Since the sla callback is a bit different, we manually create the context obj here
    # See https://airflow.apache.org/docs/apache-airflow/stable/concepts/tasks.html#sla-miss-callback
    ctx = {
        "dag": dag,
    }
    post_alert(ctx, ":clock2: SLA miss!", sla_miss_attachment(dag, task_list))


def post_alert(ctx, message, attachment):
    try:
        channels = ctx["params"].get("slack_channels", DEFAULT_CHANNELS)
    except AttributeError:
        channels = DEFAULT_CHANNELS
    for i, channel in enumerate(channels):
        SlackAPIPostOperator(
            slack_conn_id='slack_default',
            channel=channel,
            username="Composer",
            task_id=f"send_slack_msg_{i}",
            text=message,
            attachments=[attachment],
        ).execute(ctx)


def sla_task_fields(dag, task_list):

    return [
        {"title": "DAG", "value": dag.dag_id, "short": True},
        {"title": "Tasks", "value": task_list, "short" : True},
    ]

def task_fields(ctx):
    return [
        {"title": "DAG", "value": ctx["dag"].dag_id, "short": True},
        {"title": "Task", "value": ctx["task"].task_id, "short": True},
        {
            "title": "Execution date",
            "value": ctx["data_interval_start"].strftime("%d.%m.%Y %H:%M:%S UTC"),
            "short": True,
        },
        {"title": "Owner", "value": ctx["task"].owner, "short": True},
    ]


def task_log_url(ctx):
    return ctx.get("task_instance").log_url


def failure_attachment(ctx):
    return {
        "fallback": "Task failed! " + task_log_url(ctx),
        "color": "danger",
        "title": "Task failed! [view log]",
        "title_link": task_log_url(ctx),
        "text": "A task was unable to complete its run successfully.",
        "fields": task_fields(ctx),
        "ts": int(time.time()),
    }


def success_attachment(ctx):
    return {
        "fallback": "Task completed! " + task_log_url(ctx),
        "color": "good",
        "title": "Task completed",
        "title_link": task_log_url(ctx),
        "text": "A task completed its run successfully.",
        "fields": task_fields(ctx),
        "ts": int(time.time()),
    }

def sla_miss_attachment(ctx):
    return {
        "fallback": "SLA miss! " + task_log_url(ctx),
        "color": "warning",
        "title": "SLA miss",
        "title_link": task_log_url(ctx),
        "text": "A task missed its SLA.",
        "fields": task_fields(ctx),
        "ts": int(time.time()),
    }
#

the SlackAPIPostOperator is the important part though

surreal lodge
#

ok let me try that

neon cosmos
#

but you use webhook.. is it setup correctly?

#

have you tested the webhook outside airflow?

surreal lodge
#

yes saw an youtube video for that, used postman to test, it works fine on postman

neon cosmos
#

i see.. thats good to know

#

the docs says This operator allows you to post messages to Slack using incoming webhooks. Takes both Slack webhook token directly and connection that has Slack webhook token. If both supplied, http_conn_id will be used as base_url, and webhook_token will be taken as endpoint, the relative path of the url

surreal lodge
#

i am trying for almost one week and found no solution, i will try using APIPost

neon cosmos
#

do that, i have no experience with the webhook, but i cant see anything incorrect with your code.

#

i run airflow in composer, so i cant easily test out webhooks to see either

#

I have a pressing matter to attend, I hope you get something out of my wall of text.. good luck

surreal lodge
#

ok thanks

covert harnessBOT
#
Python help channel closed

This help channel has been closed and it's no longer possible to send messages here. If your question wasn't answered, feel free to create a new post in #1035199133436354600. To maximize your chances of getting a response, check out this guide on asking good questions.