Can't host Vapor Backend on Elastic Beanstalk | Vapor | Page 1

elfin hollow Jul 29, 2023, 9:56 AM

#

Hi, I have run "Swift build --configuration release" and I have the release version of my backend. Maybe I'm totally misunderstanding but after this but what I do next is to copy the Dockerfile and the contents of the .build/release folder into a Package/ folder so I can then zip it and then upload it when creating a new environment.
Can someone help me through the steps of properly deploying a Vapor backend to Elastic beanstalk? I'd even be open to alternatives to EB

dapper onyx Jul 29, 2023, 10:58 AM

#

I think AppRunner is probably the easiest way on AWS

#

However EB should support docker containers

#

What steps did you follow to set it up?

elfin hollow Jul 29, 2023, 4:49 PM

#

I really couldn't find steps to deploy on Beanstalk to be honest. I had to ask ChatGPT. This is just the backend for an iOS Application too.

#

I create the Vapor app template, set everything up
2.I ran Swift build --configuration release to get the release version, keeping in mind I should my dockerfile and image already created from what I understand. This is where my mind started to go to different places because I know that a container should be hosted in a container registry so I followed the steps to upload my container onto ECR, thinking I could use the ECR URL to point to my container when creating an environment...it did not work
My next thoughts are to just do what ChatGPT says in the screenshot I have attached. So I've built a release version of the backend, now I just have to package it and compress it to a Zip and upload it to the environment..which is what I tried.

elfin hollow Jul 29, 2023, 6:40 PM

#

I'm trying to use AppRunner, I've built the container using docker buildx --platform linux/arm64 .....I keep getting this error regardless of whether I specify the CPU architecture or not : 07-29-2023 11:31:16 AM exec ./App: exec format error

native panther Jul 29, 2023, 9:45 PM

#

@elfin hollow AppRunner is intel/amd64 only (last I checked)

#

What machine are you building docker images on? You'll need to build intel docker images.

elfin hollow Jul 29, 2023, 10:00 PM

#

native panther What machine are you building docker images on? You'll need to build intel docke...

I'm building on Apple Silicon

#

Ohhh so is "docker buildx --platform linux/amd64 what I'd be looking for"? I'm assuming by default I've been building for apple silicon with or without the additional flags.

native panther Jul 29, 2023, 10:06 PM

#

I'm not familiar with building intel images from an arm machine. I believe you're supposed to be able to do that, but I've heard it's slow. It might have gotten better with Docker's experimental (?) support for Rosetta.

#

I'm still running an intel machine, but a while ago I switched to building my docker images on GH runners in preparation for eventually switching to an M2/M3 machine in the future. You may consider trying that.

elfin hollow Jul 29, 2023, 10:08 PM

#

native panther I'm still running an intel machine, but a while ago I switched to building my do...

I saw this as an option and I think I may have to try it.

#

Heroku for now tho

#

FeelsBadMan

#

nothing against Heroku though butI'd rather have done this the first time with AppRunner

native panther Jul 29, 2023, 10:09 PM

#

I believe AppRunner will build your code for you, but I don't know how it works and if it support swift. You may want to check that out.

#

You should also confirm that they haven't added arm support.

elfin hollow Jul 29, 2023, 10:09 PM

#

native panther I believe AppRunner will build your code for you, but I don't know how it works ...

it doesn't that's the thing, I think

#

It has support for Python, NodeJS...etc

#

and you even still need the apprunner.yaml file too

native panther Jul 29, 2023, 10:10 PM

#

If you need a GH runner .yml file I can probably dig one up for you.

elfin hollow Jul 29, 2023, 10:10 PM

#

That'd actually be great. Thank you

native panther Jul 29, 2023, 10:29 PM

#

Ok, @elfin hollow , this is my GH action file that has been sanitized and cleaned up a bit.

A couple of notes:

This builds and then pushes the images to both Docker.io and AWS ECR
There is a little extra complication added because I like to grab the git branch, tag and hash information and send it through as env variable to my Vapor app so that I can confirm what version is running live
This triggers on tags matching "**devar" which is what I tag my commits with if I want to run a build. You can modify this to be tag or branch based.
You'll need to modify the AWS regions if your ECR is held somewhere other than us-west-2
This pulls secrets setup in GH, if you need those portions you'll have to set them up for yourself in GH
You'll need to configure the path to your dockerfile correctly at the end of the docker build call
IIRC uses: aws-actions/configure-aws-credentials@v1-node16 or aws-actions/configure-aws-credentials@v1-node16 relies on a connection between GH and AWS ECR that has to be configured independently, but GH/AWS had decent instructions when I set it up.

📎 devar-tag-build-clean.yml

elfin hollow Jul 29, 2023, 10:31 PM

#

native panther Ok, <@232278124920963073> , this is my GH action file that has been sanitized an...

Thank you! I'm going to try this tonight! Fingers crossed I can get a working build haha

timber flareBOT Jul 29, 2023, 10:31 PM

#

elfin hollow Thank you! I'm going to try this tonight! Fingers crossed I can get a working bu...

@native panther now has 13 coin !

elfin hollow Jul 30, 2023, 2:57 PM

#

@native panther Thanks so much! It took me a while but it worked . It's able to successfully deploy now!

timber flareBOT Jul 30, 2023, 2:57 PM

#

elfin hollow <@441408658484625409> Thanks so much! It took me a while but it worked . It's ab...

@native panther now has 14 coin !

native panther Jul 30, 2023, 3:42 PM

#

elfin hollow <@441408658484625409> Thanks so much! It took me a while but it worked . It's ab...

Glad to hear it!

elfin hollow Jul 30, 2023, 3:43 PM

#

It deploys but I get a bunch of errors hahaha

#

07-30-2023 01:18:28 AM Swift/ErrorType.swift:200: Fatal error: Error raised at top level: Error Domain=NSURLErrorDomain Code=-1000 "(null)"
07-30-2023 01:18:28 AM [ WARNING ] Error Domain=NSURLErrorDomain Code=-1000 "(null)"

native panther Jul 30, 2023, 3:43 PM

#

Was the GitHub to AWS connector the hard part?

native panther Jul 30, 2023, 3:44 PM

#

elfin hollow 07-30-2023 01:18:28 AM Swift/ErrorType.swift:200: Fatal error: Error raised at t...

Where are you using URL? For your DB configuration?

elfin hollow Jul 30, 2023, 3:46 PM

#

native panther Was the GitHub to AWS connector the hard part?

learning the format of the workflow file and plugging my own stuff in as well as connecting GitHub to AWS were all hard but I was able to fix it. I had to remove the part where it is logging into Docker Hub, that was a blocker

#

#

In all honesty I'm not even sure how the environment get works but I've changed the fields in the Dockerfile

native panther Jul 30, 2023, 3:50 PM

#

Based on your error, it looks like there’s some sort of error related to the URL. I’ve been trying to work through that new URL api just over the last 24h, so I don’t have it all figured out yet.

#

Oh, I see it

#

let databaseURL = Environment .get ("DATABASE_PORT") probably needs to be something different.

elfin hollow Jul 30, 2023, 3:53 PM

#

Oh!

#

changed it

#

I was following the deployment guide for Heroku, if the first if let needed for AWS?

native panther Jul 30, 2023, 3:54 PM

#

You'll need a full URL for the subsequent SQLPostgresConfiguration (url: databaseURL) call to work.

#

The way that if let block is supposed to work is to choose between the different code paths for configuring the database.

elfin hollow Jul 30, 2023, 3:55 PM

#

ohh so with the https://

native panther Jul 30, 2023, 3:56 PM

#

The first part is if you're using a URL for the configuration vs. all of that information broken out into parameters.

elfin hollow Jul 30, 2023, 3:56 PM

#

Ohh

#

I don't think I'm doing the first one.

native panther Jul 30, 2023, 3:57 PM

#

The URL would need to look something like postgres://username:password@localhost:5432/DatabaseName?sslmode=prefer

elfin hollow Jul 30, 2023, 3:57 PM

#

interesting, my database is hosted on RDS

#

so I've been using the endpoint

native panther Jul 30, 2023, 3:58 PM

#

If you're not using the URL configuration then you can probably remove the whole thing. The alternative would be make let databaseURL = Environment .get ("DATABASE_URL") and DO NOT include a DATABASE_URL parameter in your environment.

elfin hollow Jul 30, 2023, 3:59 PM

#

native panther If you're not using the URL configuration then you can probably remove the whole...

this is what I was diong before actually. I didn't hve the DATABASE_URL parameter set up but I had a DATABASE_HOST parameter set up

#

even though they will be the same thing right?

native panther Jul 30, 2023, 4:00 PM

#

No, you don't want them to be the same thing in this case. Not including the Database URL param in your env setup will cause you to go through the second code path, which sounds like how your env file is setup.

#

How are you loading env variables? with an .env file or configuring it in a UI?

#

Here, to make the point clear. I'd just replace the whole if/else block with the code from the else portion.

#

That'll simplify things for you.

elfin hollow Jul 30, 2023, 4:07 PM

#

native panther Here, to make the point clear. I'd just replace the whole if/else block with the...

I think this is the best bet

elfin hollow Jul 30, 2023, 4:08 PM

#

native panther How are you loading env variables? with an .env file or configuring it in a UI?

I'm passing them through when I create the App in AppRunner

#

but I also see them set up in the docker-compose file too

native panther Jul 30, 2023, 4:08 PM

#

Why are you using a compose file?

#

What are you configuring with your compose file?

elfin hollow Jul 30, 2023, 4:10 PM

#

the compose file came with the project I think. So I've just been going along with it

native panther Jul 30, 2023, 4:11 PM

#

The compose file won't come into play if you're deploying on apprunner

elfin hollow Jul 30, 2023, 4:11 PM

#

ahhhh

#

ok that clears things up for me a lot. So that means Passing it in through the UI or an ENV file would be better

native panther Jul 30, 2023, 4:12 PM

#

Yea, for apprunner that's how you'll want to do it.

elfin hollow Jul 30, 2023, 4:12 PM

#

okok, well I've good thing i've already been doing that haha

native panther Jul 30, 2023, 4:12 PM

#

depending on how you're deploying and configuring your app runner that's probably the easiest way to get started.

elfin hollow Jul 30, 2023, 4:13 PM

#

definitely. I'm hoping this changes everything haha.

#

I was trying to make get/post requests with insomnia and I was getting a 500 error , which makes sense

#

I'm checking the Application logs and I'm getting this too

#

#

#

it's rollingback now

native panther Jul 30, 2023, 4:29 PM

#

yea, this is actually the same error I'm digging into right now.

elfin hollow Jul 30, 2023, 4:29 PM

#

OH :O

#

thank God I'm not the only one running into this 😭

native panther Jul 30, 2023, 5:35 PM

#

elfin hollow OH :O

I haven't gotten to confirm everything yet, but I think I have a path forward on fixing this. You can see the conversation with Gwynne over here: #fluent message

elfin hollow Jul 30, 2023, 5:36 PM

#

native panther I haven't gotten to confirm everything yet, but I think I have a path forward on...

Definitely! Will take a look when I arrive home

elfin hollow Jul 30, 2023, 6:53 PM

#

wow that's a lot hahaha @native panther I'm not even sure I understand what's going on there .

native panther Jul 30, 2023, 6:58 PM

#

elfin hollow wow that's a lot hahaha <@441408658484625409> I'm not even sure I understand wha...

yea, it's a bit complicated. Long story short, encryption is complicated and you need to tell the SSL system to trust amazon's cert.

#

I'm trying to get a finalized implementation together in that thread for you to use. I'm currently stumbling on loading the file though the package.swift resources system, but I've got a backup plan if necessary (just load it as base64 data from a string in the env file).

elfin hollow Jul 30, 2023, 7:01 PM

#

native panther I'm trying to get a finalized implementation together in that thread for you to ...

Thank you so much 🙏 I really appreciate it!

timber flareBOT Jul 30, 2023, 7:01 PM

#

elfin hollow Thank you so much 🙏 I really appreciate it!

@native panther now has 15 coin !

native panther Jul 30, 2023, 11:36 PM

#

Ok, I think I got it figured out (deploying now to confirm on apprunner. My solution looks like this:

/// Setup database
var sqlPsqlConfig = try SQLPostgresConfiguration(url: psqlUrl)
// setup TLS trust store to trust amazon CA cert
let localPath = "/Users/js/code/PSVapor/PSVaporV4/Sources/App/TLSSupport/global-bundle.pem"
let bundlePath = "./TLSSupport/global-bundle.pem"
var tlsConfig = TLSConfiguration.makeClientConfiguration()
if let localRootCert = try? NIOSSLTrustRoots.certificates(NIOSSLCertificate.fromPEMFile(localPath)) {
    // only config tls/ssl when connecting to cloud database (has "amazonaws.com" in the url
    if let hostString = psqlUrl.host(), hostString.contains(#/amazonaws.com/#) {
        tlsConfig.trustRoots = localRootCert
        sqlPsqlConfig.coreConfiguration.tls = .require(try .init(configuration: tlsConfig))
    }
}else {
    tlsConfig.trustRoots = .certificates(try NIOSSLCertificate.fromPEMFile(bundlePath))
    sqlPsqlConfig.coreConfiguration.tls = .require(try .init(configuration: tlsConfig))
}
app.databases.use(DatabaseConfigurationFactory.postgres(configuration: sqlPsqlConfig),
                  as: .psql,
                  isDefault: true)

elfin hollow Jul 30, 2023, 11:38 PM

#

Sounds good I'm gonna try it as well

native panther Jul 30, 2023, 11:59 PM

#

I think yours will look like this:

/// Setup database
var sqlPsqlConfig = try SQLPostgresConfiguration(
  hostname: Environment.get("DATABASE_HOST' ) ?? "localhost",
  port: Environment .get ("DATABASE_PORT").flatMap(Int.init(_:)) ?? 5432,
  username: Environment.get("DATABASE_USERNAME") ?? "vapor_username",
  password: Environment.get("DATABASE_PASSWORD") ?? "vapor _password",
  database: Environment.get("DATABASE_NAME") ?? "vapor_database",
  tls: .disable)
// setup TLS trust store to trust amazon CA cert
let localPath = "/Users/js/code/PSVapor/PSVaporV4/Sources/App/TLSSupport/global-bundle.pem"
let bundlePath = "./TLSSupport/global-bundle.pem"
var tlsConfig = TLSConfiguration.makeClientConfiguration()
if let localRootCert = try? NIOSSLTrustRoots.certificates(NIOSSLCertificate.fromPEMFile(localPath)) {
    // only config tls/ssl when connecting to cloud database (has "amazonaws.com" in the url
    if let hostString = psqlUrl.host(), hostString.contains(#/amazonaws.com/#) {
        tlsConfig.trustRoots = localRootCert
        sqlPsqlConfig.coreConfiguration.tls = .require(try .init(configuration: tlsConfig))
    }
}else {
    tlsConfig.trustRoots = .certificates(try NIOSSLCertificate.fromPEMFile(bundlePath))
    sqlPsqlConfig.coreConfiguration.tls = .require(try .init(configuration: tlsConfig))
}
app.databases.use(.postgres(configuration: sqlPsqlConfig,
                  as: .psql,
                  isDefault: true)

elfin hollow Jul 31, 2023, 12:02 AM

#

native panther I think yours will look like this: ```swift /// Setup database var sqlPsqlConfi...

Ohh I see. The PEM file..... how would I obtain that?

native panther Jul 31, 2023, 12:08 AM

#

Yea, I was working my way that direction. You'll need to do the following:

download the .pem file from aws here: https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/UsingWithRDS.SSL.html (I used this one - https://truststore.pki.rds.amazonaws.com/global/global-bundle.pem)
add that file to your project under the Sources/App/ (App could be named different for you). I put mine in a new folder at Sources/App/TLSSupport/global-bundle.pem. [see image]
Add that path to your package.swift [see image]

Using SSL/TLS to encrypt a connection to a DB instance - Amazon Rel...

Create encrypted connections to your Amazon RDS DB instance using SSL/TLS.

#

#

elfin hollow Jul 31, 2023, 12:09 AM

#

Ohhhh okok. I'm going to try that now! I appreciate your help and supprt on this!!

native panther Jul 31, 2023, 12:10 AM

#

I'm happy to help. I'm trying to pay it forward after years of help around here.

#

(I started as a mechanical engineer hanging around here trying to build a passion project)

elfin hollow Jul 31, 2023, 12:13 AM

#

native panther (I started as a mechanical engineer hanging around here trying to build a passio...

That's amazing and really inspiring I'm also trying my bestto learn as much as I can so I can know enough to help others who might need help so Thank you!

timber flareBOT Jul 31, 2023, 12:13 AM

#

elfin hollow That's amazing and really inspiring I'm also trying my bestto learn as much as ...

@native panther now has 16 coin !

elfin hollow Jul 31, 2023, 12:22 AM

#

just wanted to confirm the psqlUrl should be the url for the database correct? @native panther

native panther Jul 31, 2023, 12:26 AM

#

If you want to use the URL approach, your URL will need to look like this: postgres://DATABASE_USERNAME:DATABASE_PASSWORD@DATABASE_HOST:DATABASE_PORT/DATABASE_NAME?sslmode=require

#

I kinda like doing it that way because I only have 1 env variable to manage in the env file and check for when I'm configuring the app.

elfin hollow Jul 31, 2023, 12:31 AM

#

ahh I see. I wanted to confirm that the comment is really just a check for if we're connecting to amazonaws.

elfin hollow Jul 31, 2023, 1:10 AM

#

Just tried it and it says failed to load the certificate for me. Were yuo able to deploy smoothly? @native panther

native panther Jul 31, 2023, 1:15 AM

#

I got an error for using the host() function on Linux, I’ll have to fix it when I get back to the house.

#

Give me ~15 min

elfin hollow Jul 31, 2023, 1:17 AM

#

oh okok sounds good. I'll keep at it

native panther Jul 31, 2023, 6:24 AM

#

I've validated this on docker locally, but pushing to aws to finalize the confirmation, but I think I got to the bottom of it.

The package.swift needed to be updated to load the directory and not only the single file like so:

resources: [
  .copy("TLSSupport/")
]

And the bundle path needed to be updated to this:

let bundlePath = "./PSVaporV4_App.resources/TLSSupport/global-bundle.pem"

elfin hollow Jul 31, 2023, 6:28 AM

#

oh , that's interesting. The previous way made a lot of sense to me too. Here's hoping that AWS will accept it haha

native panther Jul 31, 2023, 6:29 AM

#

It's all doing the same thing as before, just updating my understanding of how the package resources work.

native panther Jul 31, 2023, 6:49 AM

#

Ok, my GH build and apprunner deploy worked successfully! 🎉

elfin hollow Jul 31, 2023, 6:49 AM

#

native panther Ok, my GH build and apprunner deploy worked successfully! 🎉

🥳

#

and you can acces your endpoints?

native panther Jul 31, 2023, 6:50 AM

#

yep, access works

elfin hollow Jul 31, 2023, 6:51 AM

#

wow.... so those two changes did the trick then

#

I guess it makes sense though, Provide access to the folder with the resources. The bundle path it a bit weird to me though

#

so it would be ./App.resources/<anyother/folders>/global-bundle.pem ?

#

I'm assuming the _ is a naming convention of yours aha

native panther Jul 31, 2023, 7:05 AM

#

honestly, I'm not super sure how that part is built. I think it's done by apple as part of how the bundle is built.

elfin hollow Jul 31, 2023, 7:07 AM

#

my project is called Lip-Backend, would this be the corresponding bundlePath? :
let bundlePath = "./Lip-Backend_App.resources/TLSSupport/global-bundle.pem"

native panther Jul 31, 2023, 7:09 AM

#

I figured mine out by building the docker image and then digging through the image using $ docker image save psvapor:0.10.2.6.dev > suspect-image.tar and making sure that aligned with what I saw as output when running the below code as part of a local docker compose setup:

let fm = FileManager.default
let path = Bundle.main.resourcePath!
do {
    let items = try fm.contentsOfDirectory(atPath: path)
    for item in items {
        print("Found \(item)")
    }
} catch {
    // failed to read directory – bad permissions, perhaps?
}

#

I'm currently working on switch to the new @main template. I can let you know if that changes anything incase that's relevant.

elfin hollow Jul 31, 2023, 7:12 AM

#

yes definitley please do.

#

now I gotta file my file patch hahaha

native panther Jul 31, 2023, 7:19 AM

#

It doesn't seem to change anything switching. I believe the resource package name is derived using PACKAGE_NAME underscore TARGET_NAME dot "resources". You can see my package.swift in this image:

elfin hollow Jul 31, 2023, 7:20 AM

#

native panther It doesn't seem to change anything switching. I believe the resource package nam...

this is what I assumed as well

#

and I guess that format also makes sense in itself

elfin hollow Jul 31, 2023, 9:00 AM

#

public func configure(_ app: Application) throws {
    // uncomment to serve files from /Public folder
    // app.middleware.use(FileMiddleware(publicDirectory: app.directory.publicDirectory))
    /// Setup database
    var sqlPsqlConfig = SQLPostgresConfiguration(
        hostname: Environment.get("DATABASE_HOST" ) ?? "localhost",
        port: Environment .get ("DATABASE_PORT").flatMap(Int.init(_:)) ?? 5432,
        username: Environment.get("DATABASE_USERNAME") ?? "vapor_username",
        password: Environment.get("DATABASE_PASSWORD") ?? "vapor _password",
        database: Environment.get("DATABASE_NAME") ?? "vapor_database",
        tls: .disable)
    // setup TLS trust store to trust amazon CA cert
    let localPath = "/Users/dara/Documents/Development/Lip/Lip-Backend/Sources/App/TLSSupport/global-bundle.pem"
    //let bundlePath = "./Lip-Backend_App.resources/TLSSupport/global-bundle.pem"
    if let bundlePath = Bundle.main.url(forResource: "global-bundle", withExtension: "pem")?.absoluteString {
        var tlsConfig = TLSConfiguration.makeClientConfiguration()
        if let localRootCert = try? NIOSSLTrustRoots.certificates(NIOSSLCertificate.fromPEMFile(localPath)) {
            tlsConfig.trustRoots = localRootCert
            sqlPsqlConfig.coreConfiguration.tls = .require(try .init(configuration: tlsConfig))
            
        }else {
            tlsConfig.trustRoots = .certificates(try NIOSSLCertificate.fromPEMFile(bundlePath))
            sqlPsqlConfig.coreConfiguration.tls = .require(try .init(configuration: tlsConfig))
        }
    }
    
    app.databases.use(.postgres(configuration: sqlPsqlConfig), as: .psql, isDefault: true)
    
    
    app.migrations.add(CreateDeal())
    app.migrations.add(CreateUser())
    
    
    try app.autoMigrate().wait()
    
    // register routes
    try routes(app)
    
}

Tried this also...did not work hahaha

#

im gonna get some rest for now before trying again to find my globa-bundle.pem file

native panther Jul 31, 2023, 4:45 PM

#

elfin hollow ```swift public func configure(_ app: Application) throws { // uncomment to ...

if let bundlePath = Bundle.main.url(forResource: "global-bundle", withExtension: "pem")?.absoluteString looks funny.

#

If you want to use this approach, I think you'll need to use Bundle.module.

#

And, you'd probably need to use the .url(forResource: String, withExtension: String, subdirectory: String) method if you added the using .copy("TLSSupport/").

elfin hollow Aug 1, 2023, 3:30 AM

#

native panther And, you'd probably need to use the `.url(forResource: String, withExtension: St...

thats true , thank you!

timber flareBOT Aug 1, 2023, 3:30 AM

#

elfin hollow thats true , thank you!

@native panther now has 17 coin !

elfin hollow Aug 1, 2023, 3:31 AM

#

Running it locally for the first time in a while , I think I may see where some of my problems are coming from

#

it expects a .env file .

native panther Aug 1, 2023, 3:45 AM

#

Yea, you’ve got an error with your env file somewhere. Are you also configuring your DB connection bears on info in the .env file?

elfin hollow Aug 1, 2023, 3:45 AM

#

I never had an .env file 🥲

native panther Aug 1, 2023, 3:47 AM

#

If you can send your setup code I can try to help you though this. I take it your running this through docker compose, is that right?

elfin hollow Aug 1, 2023, 3:48 AM

#

correct

native panther Aug 1, 2023, 3:48 AM

#

Getting the env file setup with compose takes a little work, but it’s doable.

elfin hollow Aug 1, 2023, 3:49 AM

#

native panther Getting the env file setup with compose takes a little work, but it’s doable.

that's true. I'll make the latest push and link it now

#

[email protected]:DaraAdekore/Lip-Backend.git

#

ah, the .env files were rightfully ignored, wondered why I didn't see them for a sec

elfin hollow Aug 1, 2023, 5:15 AM

#

The local docker compose I"m running still seems ot have an issue finding the .env file

native panther Aug 1, 2023, 12:17 PM

#

elfin hollow The local docker compose I"m running still seems ot have an issue finding the .e...

Can you show the docker compose file and the dockerfile?

elfin hollow Aug 1, 2023, 1:37 PM

#

native panther Can you show the docker compose file and the dockerfile?

of course

#

# Docker Compose file for Vapor
#
# Install Docker on your system to run and test
# your Vapor app in a production-like environment.
#
# Note: This file is intended for testing and does not
# implement best practices for a production deployment.
#
# Learn more: https://docs.docker.com/compose/reference/
#
#   Build images: docker-compose build
#      Start app: docker-compose up app
# Start database: docker-compose up db
# Run migrations: docker-compose run migrate
#       Stop all: docker-compose down (add -v to wipe db)
#
version: '3.7'

volumes:
  db_data:

x-shared_environment: &shared_environment
  LOG_LEVEL: ${LOG_LEVEL:-debug}
  DATABASE_HOST: lip-database.ci59mhk01fma.us-west-2.rds.amazonaws.com
  DATABASE_NAME: lip-database
  DATABASE_USERNAME: masteruser
  DATABASE_PASSWORD: opensesame123

  DATABASE_PORT: 5432
  
services:
  app:
    image: lipcontainer:latest
    build:
      context: .
    environment:
      <<: *shared_environment
    depends_on:
      - db
    ports:
      - '8080:8080'
    # user: '0' # uncomment to run as root for testing purposes even though Dockerfile defines 'vapor' user.
    command: ["serve", "--env", "production", "--hostname", "0.0.0.0", "--port", "8080"]
  migrate:
    image: lipcontainer:latest
    build:
      context: .
    environment:
      <<: *shared_environment
    depends_on:
      - db
    command: ["migrate", "--yes"]
    deploy:
      replicas: 0
  revert:
    image: lipcontainer:latest
    build:
      context: .
    environment:
      <<: *shared_environment
    depends_on:
      - db
    command: ["migrate", "--revert", "--yes"]
    deploy:
      replicas: 0
  db:
    image: postgres:15-alpine
    volumes:
      - db_data:/var/lib/postgresql/data/pgdata
    environment:
      PGDATA: /var/lib/postgresql/data/pgdata
      POSTGRES_USER: vapor_username
      POSTGRES_PASSWORD: vapor_password
      POSTGRES_DB: vapor_database
    ports:
      - '5432:5432'

This is the docker compose file @native panther

#

# ================================
# Build image
# ================================
FROM swift:5.8-jammy as build

# Install OS updates and, if needed, sqlite3
RUN export DEBIAN_FRONTEND=noninteractive DEBCONF_NONINTERACTIVE_SEEN=true \
    && apt-get -q update \
    && apt-get -q dist-upgrade -y\
    && rm -rf /var/lib/apt/lists/*

# Set up a build area
WORKDIR /build

# First just resolve dependencies.
# This creates a cached layer that can be reused
# as long as your Package.swift/Package.resolved
# files do not change.
COPY ./Package.* ./
RUN swift package resolve

# Copy entire repo into container
COPY . .

# Build everything, with optimizations
RUN swift build -c release --static-swift-stdlib

# Switch to the staging area
WORKDIR /staging

# Copy main executable to staging area
RUN cp "$(swift build --package-path /build -c release --show-bin-path)/App" ./

# Copy resources bundled by SPM to staging area
RUN find -L "$(swift build --package-path /build -c release --show-bin-path)/" -regex '.*\.resources$' -exec cp -Ra {} ./ \;

# Copy any resources from the public directory and views directory if the directories exist
# Ensure that by default, neither the directory nor any of its contents are writable.
RUN [ -d /build/Public ] && { mv /build/Public ./Public && chmod -R a-w ./Public; } || true
RUN [ -d /build/Resources ] && { mv /build/Resources ./Resources && chmod -R a-w ./Resources; } || true

# ================================
# Run image
# ================================
FROM ubuntu:jammy

# Make sure all system packages are up to date, and install only essential packages.
RUN export DEBIAN_FRONTEND=noninteractive DEBCONF_NONINTERACTIVE_SEEN=true \
    && apt-get -q update \
    && apt-get -q dist-upgrade -y \
    && apt-get -q install -y \
      ca-certificates \
      tzdata \
# If your app or its dependencies import FoundationNetworking, also install `libcurl4`.
      # libcurl4 \
# If your app or its dependencies import FoundationXML, also install `libxml2`.
      # libxml2 \
    && rm -r /var/lib/apt/lists/*

# Create a vapor user and group with /app as its home directory
RUN useradd --user-group --create-home --system --skel /dev/null --home-dir /app vapor

# Switch to the new home directory
WORKDIR /app

# Copy built executable and any staged resources from builder
COPY --from=build --chown=vapor:vapor /staging /app

# Ensure all further commands run as the vapor user
USER vapor:vapor

# Let Docker bind to port 8080
EXPOSE 8080

# Start the Vapor service when the image is run, default to listening on 8080 in production environment
ENTRYPOINT ["./App"]
CMD ["serve", "--env", "production", "--hostname", "0.0.0.0", "--port", "8080"]

#

This is the dockerfile

native panther Aug 1, 2023, 9:23 PM

#

I used to use the x-shared_environment: &shared_environment method, but I found it frustrating. That was so long ago that I don't know why though. You might just want to try loading an .env file.

#

This is what mine looks like:

version: "3.7"
services:
  api:
    #build:
      #context: .
      #dockerfile: Dockerfile-dev
    #image: api:dev
    image: daskier/psvapor:latest.dev #alt: daskier/psvapor:latest.dev
    env_file: 
      - ../.env.development
    ports:
      - "8080:8080"

#

I believe my dockerfile is functionally the same as yours.

#

BTW - double check how vapor loads the environment variables. I don't remember if those warnings/debug messages always come up if you have your logging levels set super low. The easiest thing to do is try to access your env vars with a guard statement and log the error if the item you're looking for isn't available and you can throw a fatalError("ENV var not found")

elfin hollow Aug 2, 2023, 4:09 AM

#

native panther This is what mine looks like: ```docker version: "3.7" services: api: #bu...

I tried using the env_file but I may have used it wrong

native panther Aug 2, 2023, 4:17 AM

#

One thing that I found confusing (and I still don't recall the correct answer) is if the search path for the env file is with respect to where you are running the command from or where the dockerfile is stored.

elfin hollow Aug 2, 2023, 4:21 AM

#

so relative to the dockerfile

#

that's what I"m thinking too

#

could it be because I've been trying to use .env.production?

#

instead of just .env?

native panther Aug 2, 2023, 4:53 AM

#

That .env.production is how vapor looks for the env files if it were searching on disk within the working directory, but in the case of docker it will look for the file itself and then load the variables into the ENV for you.

#

So, this is why when you startup vapor with the logging level set to a low threshold, you'll see vapor looking for the .env.production file and not finding it. But, that's ok, because Docker will load the ENV vars for you.

#

(My description may be slightly off, but the end outcome should be correct - when providing docker with your env file you don't need to worry about vapor's access to the .env.file)

elfin hollow Aug 2, 2023, 4:57 AM

#

OH

#

I've been stressing about that this whole time haha

native panther Aug 2, 2023, 4:59 AM

#

Yes. So, like I said, the best way I've found to be super sure the ENV has what I want accessible I just do a lot of log based debugging to ensure everything is correct the first run through.

#

You have to be careful about this with publicly available resources or executing those things on public cloud instances because it could end up logging sensitive information like logins/passwords/keys. I usually do this with dummy data to avoid that problem.

elfin hollow Aug 2, 2023, 5:04 AM

#

You're right! Logging it worked

native panther Aug 2, 2023, 5:10 AM

#

What is your error at this point?

elfin hollow Aug 2, 2023, 5:10 AM

#

when I run it in Xcode or try compose up app , I get this.

#

I'm deploying to AppRunner now to see what happens

native panther Aug 2, 2023, 5:18 AM

#

There's a chance that this has to do with the database config.

elfin hollow Aug 2, 2023, 5:18 AM

#

AppRunner will automatically rollback when I see this LUL

native panther Aug 2, 2023, 5:19 AM

#

Yea, that error will mean that your app never responds to the health check, so apprunner will automatically roll back to the last known good state.

#

Show me your db config code once more?

elfin hollow Aug 2, 2023, 5:21 AM

#

I just removed the if let for the global resource code I had earlier and this is what I have now :

import NIOSSL
import Fluent
import FluentPostgresDriver
import Vapor

// configures your application
public func configure(_ app: Application) throws {
    
    guard (Environment.get("DATABASE_HOST" ) != nil),
          (Environment.get("DATABASE_PORT").flatMap(Int.init(_:)) != nil),
          Environment.get("DATABASE_USERNAME") != nil,
          Environment.get("DATABASE_PASSWORD") != nil,
          Environment.get("DATABASE_NAME") != nil else {
        fatalError("Error ENV Variable not found")
    }
    
    print("host : \(Environment.get("DATABASE_HOST" ))")
    print("port: \(Environment.get("DATABASE_PORT"))")
    print("username: \(Environment.get("DATABASE_USERNAME"))")
    print("password : \(Environment.get("DATABASE_PASSWORD"))")
    print("\(Environment.get("DATABASE_NAME"))")
    // uncomment to serve files from /Public folder
    // app.middleware.use(FileMiddleware(publicDirectory: app.directory.publicDirectory))
    /// Setup database
    var sqlPsqlConfig = SQLPostgresConfiguration(
        hostname: Environment.get("DATABASE_HOST" ) ?? "localhost",
        port: Environment.get("DATABASE_PORT").flatMap(Int.init(_:)) ?? 5432,
        username: Environment.get("DATABASE_USERNAME") ?? "vapor_username",
        password: Environment.get("DATABASE_PASSWORD") ?? "vapor _password",
        database: Environment.get("DATABASE_NAME") ?? "vapor_database",
        tls: .disable)
    // setup TLS trust store to trust amazon CA cert
    let localPath = "/Users/dara/Documents/Development/Lip/Lip-Backend/Sources/App/TLSSupport/global-bundle.pem"
    let bundlePath = "./Lip-Backend_App.resources/TLSSupport/global-bundle.pem"
    var tlsConfig = TLSConfiguration.makeClientConfiguration()
    if let localRootCert = try? NIOSSLTrustRoots.certificates(NIOSSLCertificate.fromPEMFile(localPath)) {
        tlsConfig.trustRoots = localRootCert
        sqlPsqlConfig.coreConfiguration.tls = .require(try .init(configuration: tlsConfig))
        
    }else {
        tlsConfig.trustRoots = .certificates(try NIOSSLCertificate.fromPEMFile(bundlePath))
        sqlPsqlConfig.coreConfiguration.tls = .require(try .init(configuration: tlsConfig))
    }
    app.databases.use(.postgres(configuration: sqlPsqlConfig), as: .psql, isDefault: true)
    
    
    app.migrations.add(CreateDeal())
    app.migrations.add(CreateUser())
    
    
    try app.autoMigrate().wait()
    
    // register routes
    try routes(app)
    
}

native panther Aug 2, 2023, 5:29 AM

#

That line message: no pg-hba.conf entry for host "54.70.103.169", user "masteruser", database "Lip-database", no encryption, gives me pause. I'm not familiar with that.

#

That looks like a DB configuration issue. I'm not sure how that arises in an RDS DB though.

#

It might be worth posting that in #fluent

#

It'd probably be helpful to read this too: https://www.postgresql.org/docs/current/auth-pg-hba-conf.html

PostgreSQL Documentation

21.1. The pg_hba.conf File

21.1. The pg_hba.conf File Client authentication is controlled by a configuration file, which traditionally is named pg_hba.conf and is stored in …

elfin hollow Aug 3, 2023, 1:54 AM

#

Yeah

#

Apparently RDS is meant to cover the whole configuration for the DB abstracting it to just the UI that we see

#

I get it when runnning locally too so that makes me think it's not an issue with RDS

#

I was forgetting to up the database hahaha

#

this is better!

elfin hollow Aug 3, 2023, 2:20 AM

#

these are my DB configurations on RDS actually

#

the global-bundle.pem should work with this right?

native panther Aug 3, 2023, 3:14 AM

#

Yea, the global-bundle.pem should work with that. I'm using the same Certificate Authority option.

#

What security group is your apprunner instance in?

elfin hollow Aug 3, 2023, 3:15 AM

#

how would I find that?

#

chekcing the configuration settings for it and I'm not sure i see it

#

But locally: I was not pointnig to my local database now I'm getting this error lip-backend-app-1 | [ WARNING ] PSQLError(code: sslUnsupported)
lip-backend-app-1 | Swift/ErrorType.swift:200: Fatal error: Error raised at top level: PSQLError(code: sslUnsupported)

native panther Aug 3, 2023, 3:18 AM

#

Under apprunner configuration:

#

Ok, sslUnsuported makes sense for your local. I was getting the same thing. That's why you see in my config.swift file that I don't set the tls settings to require if the localRootCert exists AND the database url doesn't include amazonaws.com.

#

So, when connecting to my local database ssl isn't required and when connecting to RDS it will require the ssl.

#

#

Are you able to directly connect to your RDS instance from your local machine? (I setup a security group for my local ip address to access my RDS instance)

#

Because if you can, you can test this whole setup from your local machine.

elfin hollow Aug 3, 2023, 3:25 AM

#

I believe I can connect to it from Azure Data Studio

#

#

no tables or anything yet tho I think so there's nothing to query

native panther Aug 3, 2023, 3:28 AM

#

If that's a local app then you can probably connect. You could try with an app like postico or TablePlus or similar.

#

Or just psql command line.

#

If that works then you could feel confident testing your app from xcode with the connection parameters set for RDS.

elfin hollow Aug 3, 2023, 3:30 AM

#

okI see, I'll try that now

#

oh

#

it iddn't work

#

I get this

Password for user masteruser: 
psql: error: connection to server at "lip-database.ci59mhk01fma.us-west-2.rds.amazonaws.com" (44.235.170.101), port 5432 failed: FATAL:  database "lip-database" does not exist

native panther Aug 3, 2023, 3:51 AM

#

This is the command I used to verify my connection: psql -h hostname.us-west-2.rds.amazonaws.com -p 5432 "dbname=dbName user=userName sslrootcert=/Users/js/code/PSVapor/PSVaporV4/Sources/App/TLSSupport/global-bundle.pem sslmode=verify-full"

elfin hollow Aug 3, 2023, 3:55 AM

#

Lip-Backend % psql -h lip-database.ci59mhk01fma.us-west-2.rds.amazonaws.com -p 5432 "dbname=lip-database user=masteruser sslrootcert=/Users/dara/Documents/Development/Lip/Lip-Backend/Sources/App/TLSSupport/global-bundle.pem sslmode=verify-full"
Password for user masteruser: 
psql: error: connection to server at "lip-database.ci59mhk01fma.us-west-2.rds.amazonaws.com" (44.235.170.101), port 5432 failed: FATAL:  database "lip-database" does not exist

Still the same thingit seems

#

#

it verifiably exists too

native panther Aug 3, 2023, 4:03 AM

#

Can you try with sslmode=allow?

elfin hollow Aug 3, 2023, 4:06 AM

#

I can try that

#

but what theheck

#

DB name is blacnk

#

are the instance ID and the DB name not the same?

#

it's modifying right nwo so I can't check the settings but I could have sworn I set that.

native panther Aug 3, 2023, 4:14 AM

#

Mine does have a name:

#

They don't have to be the same:

#

Database name is also part of my terraform config for creating the RDS instance:

#

You may want to create a new RDS DB just to see if that changes anything for you.

elfin hollow Aug 3, 2023, 4:30 AM

#

I can't see it in the UI for the RDS instance :O

#

I think I may have to delete and recreate it

#

#

Ok I figured it out. I think it's because I'm using the free tear?

 psql -h lip-database.ci59mhk01fma.us-west-2.rds.amazonaws.com -p 5432 "dbname=postgres user=masteruser sslrootcert=/Users/dara/Documents/Development/Lip/Lip-Backend/Sources/App/TLSSupport/global-bundle.pem sslmode=verify-full"
Password for user masteruser: 
psql (15.3)
SSL connection (protocol: TLSv1.2, cipher: ECDHE-RSA-AES256-GCM-SHA384, compression: off)
Type "help" for help.

``` this is what it should look like if you've succesfully connected right?

native panther Aug 3, 2023, 4:51 AM

#

Yea, this is what mine looks like with a successful conection:

elfin hollow Aug 3, 2023, 4:52 AM

#

perfect!

#

so...my database name is 'postgres'

#

supposedly?

native panther Aug 3, 2023, 4:55 AM

#

I think that is the default DB name.

#

You can probably create another database with a different name through the PSQL tool if you want.

#

But, for now you can probably test these configuration values with your project.

elfin hollow Aug 3, 2023, 4:56 AM

#

native panther But, for now you can probably test these configuration values with your project.

exactly

#

#

works in Xcode

elfin hollow Aug 3, 2023, 6:19 AM

#

We got it 😎 @native panther

native panther Aug 3, 2023, 12:42 PM

#

elfin hollow We got it 😎 <@441408658484625409>

Congratulations!

elfin hollow Aug 3, 2023, 1:39 PM

#

native panther Congratulations!

You did this !

#

I feel like I should make a post on medium about thisbhahaha

#

Just to help others who might be in the same situation

native panther Aug 3, 2023, 2:03 PM

#

Yea, I’ve been thinking the same.

#

I think there’s a post in the tls setup

#

There’s another part in there about the dynamic config for local vs. cloud

#

I’ve been trying to use the new App Store Connect server library from Apple recently and there’s a post in there too.

elfin hollow Aug 4, 2023, 4:08 AM

#

Ohh ok could you link it if possible?

#

I'm currently trying to figure out Relations. From it seems they're useful for querying from a database and not creating or updating.

native panther Aug 4, 2023, 4:19 AM

#

yea, relations are more for querying.

#

The vapor docs are a good starting point.

#

https://docs.vapor.codes/fluent/overview/

Vapor: Fluent → Overview

Vapors documentation (web framework for Swift).

#Can't host Vapor Backend on Elastic Beanstalk