Setting Up Gmail/Google Auth in Self Hosted Twenty (Docker Compose) | Twenty | Page 1

spare bluff May 5, 2024, 7:11 PM

#

Hi everyone,

I've recently set up Twenty on Elest.io and have encountered some difficulties integrating the Gmail API. The setup mostly went smoothly, but I'm having trouble connecting and configuring the Gmail API specifically. I've set up the necessary configurations within the Google Cloud Console and enabled the Gmail and Calendar APIs.

Here’s the main challenge: configuring the environment variables properly in the Docker Compose setup.

Below are the current settings from my Docker Compose configuration:

MESSAGING_PROVIDER_GMAIL_ENABLED: "true"
CALENDAR_PROVIDER_GMAIL_ENABLED: "true"
AUTH_GOOGLE_CALLBACK_URL: ${SERVER_URL}/google-callback
AUTH_GOOGLE_ENABLED: "true"
AUTH_GOOGLE_CLIENT_ID: ${AUTH_GOOGLE_CLIENT_ID}
AUTH_GOOGLE_CLIENT_SECRET: ${AUTH_GOOGLE_CLIENT_SECRET}
AUTH_GOOGLE_APIS_CALLBACK_URL: ${SERVER_URL}/google-callback

The Server URL is set to https://twenty.regh.io.
I have generated a Google Cloud Consol App and have set the Client ID and Client Secret in the ENV.

Specific Questions:

What should be the authorized redirect URI in the Google Cloud Console for the app?
What is the proper setup for the callback URLs for APIs and Google?
Are there any additional APIs I need to enable or specific settings I might be missing for Docker Compose?

Any guidance or suggestions would be greatly appreciated. Hoping to get this integration smooth and operational!

#

I was able to get it fixed.

Here was the proper Callback URL Configuration for me:

MESSAGING_PROVIDER_GMAIL_ENABLED: "true"
CALENDAR_PROVIDER_GMAIL_ENABLED: "true"
AUTH_GOOGLE_CALLBACK_URL: ${SERVER_URL}/auth/google/redirect
AUTH_GOOGLE_ENABLED: "true"
AUTH_GOOGLE_CLIENT_ID: ${AUTH_GOOGLE_CLIENT_ID}
AUTH_GOOGLE_CLIENT_SECRET: ${AUTH_GOOGLE_CLIENT_SECRET}
AUTH_GOOGLE_APIS_CALLBACK_URL: ${SERVER_URL}/auth/google-apis/get-access-token

#

I also added the correct URIs to the google cloud console under the OAuth Credentials for authorizes redirect URIs

spare bluff May 5, 2024, 11:54 PM

#

Maybe I don't have it setup right though...

I'm able to log in via OAuth with Google, and I was able to connect an email account, but it's sitting there in pending and doesn't seem to be syncing contacts or anything

vivid carbon May 5, 2024, 11:59 PM

#

This is how i've set it up. make sure your pass the right callbacks in google cloud and example the APIs

MESSAGING_PROVIDER_GMAIL_ENABLED=true
CALENDAR_PROVIDER_GMAIL_ENABLED=true
MESSAGING_PROVIDER_GMAIL_CALLBACK_URL=https://crm.yourdomain.com/auth/google-gmail/get-access-token
AUTH_GOOGLE_APIS_CALLBACK_URL=https://crm.yourdomain.com/auth/google-apis/get-access-token
AUTH_GOOGLE_ENABLED=true
AUTH_GOOGLE_CLIENT_ID=1234567890abcdegfhijklmnopqrstuvwxyz.apps.googleusercontent.com
AUTH_GOOGLE_CLIENT_SECRET=1234567890abcdegfhijklmnopqrstuvwxyz
AUTH_GOOGLE_CALLBACK_URL=https://crm.yourdomain.com/auth/google/redirect
FRONT_AUTH_CALLBACK_URL=https:/crm.yourdomain.com/verify
IS_SIGN_UP_DISABLED=false
PASSWORD_RESET_TOKEN_EXPIRES_IN=5m

spare bluff May 6, 2024, 2:12 PM

#

Thanks - is it normal that the account under "emails" just sits staying in "pending"?

sand mulch May 7, 2024, 1:18 AM

#

You need to have the worker running. I found the command by searching the discord for this. You may also need to kick off the cron jobs (commands also found in the discord)

worn sage May 7, 2024, 12:40 PM

#

yes @unique garden is writing doc about it. In a nutshell:

env variable: MESSAGE_QUEUE_TYPE=pg-boss
env variable: CALENDAR_PROVIDER_GOOGLE_ENABLED=true
env variable: MESSAGING_PROVIDER_GMAIL_ENABLED=true
env variable: AUTH_GOOGLE_CLIENT_ID, AUTH_GOOGLE_CLIENT_SECRET, AUTH_GOOGLE_APIS_CALLBACK_URL

#

Then:

run the worker in a separate process: yarn worker:prod (if you are using a dockerize environment, you might want to run the regular twenty container but only override run command from yarn start:prod to yarn worker:prod)

#

activate mail cron: yarn command:prod cron:messaging:gmail-fetch-messages-from-cache

#

Note that MESSAGE_QUEUE_TYPE can also be set to bull-mq if you want to use redis (it's what we use for the cloud) but pg-boss (postgres) should be totally fine

spare bluff May 8, 2024, 1:58 AM

#

Well, we have progress, but now it's just showing "not synced"

Here's my Docker Compose:

version: "3.8"

services:
  server:
    image: twentycrm/twenty:${SOFTWARE_VERSION_TAG}
    restart: always
    ports:
      - "172.17.0.1:39973:3000"
    environment:
      PORT: 3000
      PG_DATABASE_URL: postgres://${POSTGRES_USER}:${POSTGRES_PASSWORD}@db:5432/${POSTGRES_DB}
      SERVER_URL: ${SERVER_URL}
      FRONT_BASE_URL: ${SERVER_URL}

      ENABLE_DB_MIGRATIONS: "true"

      SIGN_IN_PREFILLED: "false"
      STORAGE_TYPE: local
      STORAGE_LOCAL_PATH: .local-storage
      ACCESS_TOKEN_SECRET: ${ACCESS_TOKEN_SECRET}
      LOGIN_TOKEN_SECRET: ${LOGIN_TOKEN_SECRET}
      REFRESH_TOKEN_SECRET: ${REFRESH_TOKEN_SECRET}
      FILE_TOKEN_SECRET: ${FILE_TOKEN_SECRET}

      EMAIL_FROM_ADDRESS: ${EMAIL_FROM_ADDRESS}
      EMAIL_SYSTEM_ADDRESS: ${EMAIL_FROM_ADDRESS}
      EMAIL_FROM_NAME: "Twenty"
      EMAIL_DRIVER: smtp
      EMAIL_SMTP_HOST: ${EMAIL_SMTP_HOST}
      EMAIL_SMTP_PORT: ${EMAIL_SMTP_PORT}
      PASSWORD_RESET_TOKEN_EXPIRES_IN: 5m
      FRONT_AUTH_CALLBACK_URL: ${SERVER_URL}/verify
      IS_SIGN_UP_DISABLED: "true"
      
      MESSAGE_QUEUE_TYPE: "pg-boss"
      MESSAGING_PROVIDER_GMAIL_ENABLED: "true"
      CALENDAR_PROVIDER_GMAIL_ENABLED: "true"
      AUTH_GOOGLE_CALLBACK_URL: ${SERVER_URL}/auth/google/redirect
      AUTH_GOOGLE_ENABLED: "true"
      AUTH_GOOGLE_CLIENT_ID: ${AUTH_GOOGLE_CLIENT_ID}
      AUTH_GOOGLE_CLIENT_SECRET: ${AUTH_GOOGLE_CLIENT_SECRET}
      AUTH_GOOGLE_APIS_CALLBACK_URL: ${SERVER_URL}/auth/google-apis/get-access-token
      MESSAGING_PROVIDER_GMAIL_CALLBACK_URL: ${SERVER_URL}/auth/google-gmail/get-access-token
      
    depends_on:
      - db
    volumes:
      - ./server-local-data:/app/.local-storage

  worker:
    image: twentycrm/twenty:${SOFTWARE_VERSION_TAG}
    restart: always
    environment:
      PORT: 3000
      PG_DATABASE_URL: postgres://${POSTGRES_USER}:${POSTGRES_PASSWORD}@db:5432/${POSTGRES_DB}
      SERVER_URL: ${SERVER_URL}
      MESSAGE_QUEUE_TYPE: "pg-boss"
    command: yarn worker:prod
    depends_on:
      - db

  cron:
    image: twentycrm/twenty:${SOFTWARE_VERSION_TAG}
    restart: always
    environment:
      PORT: 3000
      PG_DATABASE_URL: postgres://${POSTGRES_USER}:${POSTGRES_PASSWORD}@db:5432/${POSTGRES_DB}
      SERVER_URL: ${SERVER_URL}
      MESSAGE_QUEUE_TYPE: "pg-boss"
    command: yarn command:prod cron:messaging:gmail-fetch-messages-from-cache
    depends_on:
      - db
  
  db:
    image: twentycrm/twenty-postgres:${SOFTWARE_VERSION_TAG}
    restart: always
    volumes:
      - ./db-data:/bitnami/postgresql
    environment:
      POSTGRES_PASSWORD: ${POSTGRES_PASSWORD}
      POSTGRES_USER: ${POSTGRES_USER}
      POSTGRES_DB: ${POSTGRES_DB}
    ports:
      - 172.17.0.1:51175:5432

  pgadmin4:
    image: elestio/pgadmin:latest
    restart: always
    environment:
      PGADMIN_DEFAULT_EMAIL: ${ADMIN_EMAIL}
      PGADMIN_DEFAULT_PASSWORD: ${ADMIN_PASSWORD}
      PGADMIN_LISTEN_PORT: 8080
    ports:
      - "172.17.0.1:26565:8080"
    volumes:
      - ./servers.json:/pgadmin4/servers.json

I'm a bit of a newb with this learning as I go - typically use ChatGPT to help me try an move forward, so feel free to help me correct any AI errors

worn sage May 8, 2024, 6:42 AM

#

Hi:

Your worker needs exactly the same env variable as your server
no need to have a dedicated container for the cron, you can just run the command one time from server container

#

Then look at your worker logs when you connect your account to see how the email sync is doing

spare bluff May 8, 2024, 2:59 PM

#

@worn sage I'm sorry - I don't know what this means:

Your worker needs exactly the same env variable as your server

Do you mean that I need to have this both for the server and for the worker? or am I misunderstanding?

      PORT: 3000
      PG_DATABASE_URL: postgres://${POSTGRES_USER}:${POSTGRES_PASSWORD}@db:5432/${POSTGRES_DB}
      SERVER_URL: ${SERVER_URL}
      FRONT_BASE_URL: ${SERVER_URL}

      ENABLE_DB_MIGRATIONS: "true"

      SIGN_IN_PREFILLED: "false"
      STORAGE_TYPE: local
      STORAGE_LOCAL_PATH: .local-storage
      ACCESS_TOKEN_SECRET: ${ACCESS_TOKEN_SECRET}
      LOGIN_TOKEN_SECRET: ${LOGIN_TOKEN_SECRET}
      REFRESH_TOKEN_SECRET: ${REFRESH_TOKEN_SECRET}
      FILE_TOKEN_SECRET: ${FILE_TOKEN_SECRET}

      EMAIL_FROM_ADDRESS: ${EMAIL_FROM_ADDRESS}
      EMAIL_SYSTEM_ADDRESS: ${EMAIL_FROM_ADDRESS}
      EMAIL_FROM_NAME: "Twenty"
      EMAIL_DRIVER: smtp
      EMAIL_SMTP_HOST: ${EMAIL_SMTP_HOST}
      EMAIL_SMTP_PORT: ${EMAIL_SMTP_PORT}
      PASSWORD_RESET_TOKEN_EXPIRES_IN: 5m
      FRONT_AUTH_CALLBACK_URL: ${SERVER_URL}/verify
      IS_SIGN_UP_DISABLED: "true"
      
      MESSAGE_QUEUE_TYPE: "pg-boss"
      MESSAGING_PROVIDER_GMAIL_ENABLED: "true"
      CALENDAR_PROVIDER_GMAIL_ENABLED: "true"
      AUTH_GOOGLE_CALLBACK_URL: ${SERVER_URL}/auth/google/redirect
      AUTH_GOOGLE_ENABLED: "true"
      AUTH_GOOGLE_CLIENT_ID: ${AUTH_GOOGLE_CLIENT_ID}
      AUTH_GOOGLE_CLIENT_SECRET: ${AUTH_GOOGLE_CLIENT_SECRET}
      AUTH_GOOGLE_APIS_CALLBACK_URL: ${SERVER_URL}/auth/google-apis/get-access-token
      MESSAGING_PROVIDER_GMAIL_CALLBACK_URL: ${SERVER_URL}/auth/google-gmail/get-access-token

crystal granite May 9, 2024, 10:40 AM

#

spare bluff I also added the correct URIs to the google cloud console under the OAuth Creden...

Which are ?

spare bluff May 9, 2024, 3:02 PM

#

crystal granite Which are ?

Since it said "authorized" URLs, I just added every one of them that I saw in the docker compose - so I added these for mine:

https://twenty.regh.io/auth/google/redirect
https://twenty.regh.io/auth/google-apis/get-access-token
https://twenty.regh.io/auth/google-gmail/get-access-token
https://twenty.regh.io/verify

worn sage May 14, 2024, 1:28 PM

#

Is this issue solved? 🙂

hazy dagger Aug 17, 2024, 4:24 AM

#

spare bluff Since it said "authorized" URLs, I just added every one of them that I saw in th...

I believe I have everything set up properly, but I'm not sure I'm running the cron properly. I don't see anything in crontab -l that would indicated that the processes are kicking off.

#

# from your worker container yarn command:prod cron:messaging:messages-import yarn command:prod cron:messaging:message-list-fetch yarn command:prod cron📆calendar-event-list-fetch

#

Any help you could provide would be much appreciated. Thanks!

worn sage Aug 22, 2024, 5:10 PM

#

Hi @hazy dagger, sorry for the slow response

#

the crons are actually not saved in crontab but on the worker

#

what's your setup? and are you using redis?

hazy dagger Aug 22, 2024, 5:12 PM

#

Hey @worn sage no worries. I ended up figuring it out by setting:

docker exec -it app-server-name yarn command:prod cron:messaging:messages-import docker exec -it app-server-name yarn command:prod cron:messaging:message-list-fetch docker exec -it app-server-name yarn command:prod cron📆calendar-event-list-fetch

worn sage Aug 22, 2024, 5:12 PM

#

wonderful!

#

is it working?

hazy dagger Aug 22, 2024, 5:13 PM

#

The other area that I got hung up on was the calendar integration. I think I was missing a scope, but everything is working as it should thus far.

#

Yes, everything is working. Thank you! 🙏

worn sage Aug 22, 2024, 5:13 PM

#

wonderful!

#

Enjoy using Twenty

hazy dagger Aug 22, 2024, 5:14 PM

#

The cal.com guide for integrations is a great resource:

https://github.com/calcom/cal.com

GitHub

GitHub - calcom/cal.com: Scheduling infrastructure for absolutely e...

Scheduling infrastructure for absolutely everyone. - calcom/cal.com

#

Thanks, @worn sage !

hazy dagger Aug 22, 2024, 6:11 PM

#

@worn sage after another look. It appears that the calendar functionality isn't sticking. I think I'm asking for all of the right scopes from the app in Google Console.

The auth process works and the emails come in, but:

Calendar doesn't seem to stick.

#

worn sage Aug 23, 2024, 10:00 AM

#

Weird, even if the scopes are not right you should see a calendarChannel here

#

could you show me the content of your calendarChannel table (in your postgres workspaceSchema?)

hazy dagger Aug 23, 2024, 4:27 PM

#

@worn sage will DM you. It's empty though.

worn sage Aug 23, 2024, 4:49 PM

#

We've found the issue:
turns out that there was an error in env variables
CALENDAR_PROVIDER_GOOGLE_ENABLED=true is correct
CALENDAR_PROVIDER_GMAIL_ENABLED=trueis not correct

#Setting Up Gmail/Google Auth in Self Hosted Twenty (Docker Compose)