Suggestions on limiting memory consumption, we have Swagger , a cache
#memory consumption
placid shoal
placid shoal
Nestjs in docker container is killed off at the build stage itseld
fierce yew
Can you send us the Dockerfile ?
steep tree
Killed at the build stage? Do you mean the bootstrap stage, whne the application is starting? Or are you actually building (i.e. compiling typescript) inside a container?
fierce yew
that's what I was trying to find out XD
placid shoal
FROM node:20-alpine
RUN mkdir -p /usr/src/app/node_modules
RUN chown -R node:node /usr/src/app
USER node
WORKDIR /usr/src/app
COPY --chown=node:node package*.json ./
RUN npm ci
RUN npm cache clean --force
RUN mv node_modules ../
COPY --chown=node:node . ./
RUN ls -l
RUN npm run build
EXPOSE 8083
CMD npm run migration:run && npm run seed && npm run start:prod
@steep tree @fierce yew
fierce yew
FROM node:20-alpine
RUN mkdir -p /usr/src/app/node_modules
RUN chown -R node:n...
is this image supposed to run in production ?
fierce yew
is this image supposed to run in production ?
It seems so, my bad its a bit hard to read because you didn't enclose it in backticks like so:
FROM...
alpine fableBOT
Please format your question or answer with Markdown formatting.
It leads to better readability and an easier time to spot problems.
For code blocks, you can wrap your block with three back ticks before and after the block, and after the first three back ticks you can add a language (like ts) to add syntax highlighting.
e.g.
```ts
@Injectable()
export class MySuperAwesomeService {
constructor(@Inject('InjectionToken') private readonly dep: SomeDependency) {}
getRandomNumber(): number {
return Math.round(Math.random() * 1000);
}
}
```
Becomes :point_down:
@Injectable()
export class MySuperAwesomeService {
constructor(@Inject('InjectionToken') private readonly dep: SomeDependency) {}
getRandomNumber(): number {
return Math.round(Math.random() * 1000);
}
}
Can you show us the scripts migration:run, seed and start:prod ? And the logs
placid shoal
is this image supposed to run in production ?
Yes
placid shoal
It seems so, my bad its a bit hard to read because you didn't enclose it in back...
My bad
FROM node:20-alpine
RUN mkdir -p /usr/src/app/node_modules
RUN chown -R node:node /usr/src/app
USER node
WORKDIR /usr/src/app
COPY --chown=node:node package*.json ./
RUN npm ci
RUN npm cache clean --force
# RUN mv node_modules ../
COPY --chown=node:node . ./
RUN ls -l
RUN npm run build
EXPOSE 8083
CMD npm run migration:run && npm run seed && npm run start:prod
Docker logs just say
Killed after the npm run build command
placid shoal
Yup. I don't know what to move there
fierce yew
if you haven't done so
placid shoal
what if you remove the cache clean ?
Removed.
Still get the same
One question, if i remove the node_modules, how will it use node modules for building the docker image and how will those libraries be used
fierce yew
well npm ci is supposed to do that
it adds a layer with the node_modules to the docker image
placid shoal
Oh got it
fierce yew
so node_modules should be in .dockerignore that's for sure
placid shoal
Assumed that npm ci would only install all dependencies except devDependencies
fierce yew
Assumed that npm ci would only install all dependencies except devDependencies
nah it doesn't do that and shouldn't you need the devDepds to build your image
fierce yew
Yes
fierce yew
You don't need version unless your docker / docker compose is very old, like 1.17
And yes it is fine for a start, but if you want https at some point you will need to mix it up
so you can get rid of the first line
placid shoal
so you can get rid of the first line
Oh. Our server is https
Does that mean version has been conflicting
placid shoal
Oh. Our server is https
Only difference with our production docker-compose is specifying the ports
fierce yew
Yeah you might have to do that
placid shoal
We haven't got any issues so far. Can you point me to the docs of this
steep tree
Docker logs just say
Killed after the npm run build command
So it crashes on CI?
fierce yew
We haven't got any issues so far. Can you point me to the docs of this
the version key is useless https://docs.docker.com/compose/compose-file/04-version-and-name/
dependingo n the version of docker and docker compose again
placid shoal
So it crashes on CI?
Nope. After it's on npm run build
fierce yew
is it still crashing after what I told you ?
steep tree
Nope. After it's on npm run build
By "on CI" I meant on your continuous integration platform
placid shoal
Oh yes. My bad😁
steep tree
the version key is useless https://docs.docker.com/compose/compose-file/04-versi...
I didn't even know they deprecated version. It seems like a very unwise decision to me. They basically decided that all future versions of compose will be backwards compatible, which is a noble idea, but I'm skeptical about it holding in practise.
steep tree
Oh yes. My bad😁
What's the specs of the CI runner? You might need at least 1 gig of RAM for building a non trivial Nest app. Maybe even 2 to be safe.
Especially if you use some nest-cli plugins (like swagger) that add a noticeable overhead to the build process
placid shoal
is it still crashing after what I told you ?
Got an access denied: rmdir /use/app/dist/src
fierce yew
Got an access denied: rmdir /use/app/dist/src
what does the dockerfile look like right now ^^'
placid shoal
FROM node:20-alpine
RUN mkdir -p /usr/src/app/node_modules
RUN chown -R node:node /usr/src/app
WORKDIR /usr/src/app
COPY --chown=node:node package*.json ./
RUN npm ci
# RUN npm cache clean --force
# RUN mv node_modules ../
COPY --chown=node:node . ./
RUN ls -l
RUN npm run build
EXPOSE 8083
USER node
CMD npm run migration:run && npm run seed && npm run start:prod
last effort, try this:
FROM node:20-alpine
WORKDIR /usr/src/app
COPY package*.json ./
RUN npm ci
COPY . .
RUN npm run build
RUN chown -R node:node /usr/src/app
USER node
EXPOSE 8083
CMD npm run migration:run && npm run seed && npm run start:prod
first locally then in your CI
(with node_modules in dockerignore)
placid shoal
last effort, try this:
```docker
FROM node:20-alpine
WORKDIR /usr/src/app
COPY...
Looks like it's stuck in the step where rights are been assigned, RUN chown...
fierce yew
It takes a long time to run that’s normal
fierce yew
Looks like it's stuck in the step where rights are been assigned, RUN chown...
FROM node:20-alpine AS build
WORKDIR /usr/src/app
COPY package*.json ./
RUN npm ci
COPY . .
RUN npm run build
RUN npm ci --omit=dev
FROM node:20-alpine
WORKDIR /usr/src/app
COPY --from=build --chown=node:node /usr/src/app/dist ./dist
COPY --from=build --chown=node:node /usr/src/app/node_modules ./node_modules
COPY --from=build --chown=node:node /usr/src/app/package*.json ./
USER node
EXPOSE 8083
CMD npm run migration:run && npm run seed && npm run start:prod
might be faster like this
placid shoal
Should I make it multi-stage
Omit dev didn't work. Exit code: 127
Husky not found
> qts-be@0.0.1 migration:run
ticket-service-1 | > npm run typeorm migration:run -- -d ./src/config/typeorm.config.ts
ticket-service-1 |
ticket-service-1 |
ticket-service-1 | > qts-be@0.0.1 typeorm
ticket-service-1 | > ts-node ./node_modules/typeorm/cli migration:run -d ./src/config/typeorm.config.ts
ticket-service-1 |
ticket-service-1 | Error during migration run:
ticket-service-1 | Error: Unable to open file: "/usr/src/app/src/config/typeorm.config.ts". Cannot find module '/usr/src/app/src/config/typeorm.config.ts'
ticket-service-1 | Require stack:
ticket-service-1 | - /usr/src/app/node_modules/typeorm/util/ImportUtils.js
ticket-service-1 | - /usr/src/app/node_modules/typeorm/commands/CommandUtils.js
it is multi stage now 🙂
which is why it cannot find typeorm.config.ts, I didn't not copy over the src folder
placid shoal
Ah yes😭
I can move the migration and seed commands to first stage?
Done
That worked, now getting some issue of some hbs email templates I had, which did not get added to dist
Hmm. Fixed it by adding a COPY --from build
What is the point of adding node modules, dist, package.json in dockerignore, if you are going to copy it anyway
fierce yew
Well normally, you would not copy src to the final stage, however you do it because you have typeorm configured in some way. You should make it so typeorm has files separate for migration (sorry I have NO idea how typeorm works)
as to why we add node_modules/ and dist to .dockerignore (not package.json) it's to not pollute the docker image with your local node_modules and build files (which will be deleted anyways). Your node_modules might contain platform specific modules, also, since npm ci overwrites node_modules, you would have one node_modules on the layer COPY . . and another on the layer RUN npm ci which would result in an image twice as big for no reason
same for build
placid shoal
```docker
FROM node:20-alpine AS build
WORKDIR /usr/src/app
COPY package*.json...
Also two npm ci here?
Doesn't matter, right
Since already installed
fierce yew
Also two npm ci here?
The second one removes dev dependencies like typescript, prettier eslint
placid shoal
The second one removes dev dependencies like typescript, prettier eslint
When i omit dev dependencies, husky is creating a conflict