✅ - Issues using a Lambda Layer

[ashen python]

Hello, I am trying to use a Lambda Layer in my Lambda.
Here is my function definition:

export const onUploadHandler = defineFunction({
  name: "on-upload-handler",
  entry: "./handler.ts",
  layers: {
    sharp: "arn:aws:lambda:eu-west-1:[ACCOUNT_ID]:layer:sharp:3",
  },
});

Following this documentation, I was expecting that I just use import sharp from "sharp" in my Lambda function and then it will use the layer instead of a local installation. If sharp doesn't appear in my ./package.json my deployment fails with the error message: error TS2307: Cannot find module 'sharp' or its corresponding type declarations..

However, when I install sharp locally (even as a dev dependency), it seems to use the local installation which then lets the Lambda invocation fail as my local environment is mac OS and my Lambda runs on Linux.

Here is the invocation error message:

Error: Could not load the "sharp" module using the linux-x64 runtime
Possible solutions:
- Ensure optional dependencies can be installed:
    npm install --include=optional sharp
- Ensure your package manager supports multi-platform installation:
    See https://sharp.pixelplumbing.com/install#cross-platform
- Add platform-specific dependencies:
    npm install --os=linux --cpu=x64 sharp

I don't get my head around it. Any recommendation?
I know I could create a custom resource but then I loose all the benefits of Amplify's integration (e.g., permissions on storage and API resources) which is a pity. Also, I would need to have Docker installed locally, which I want to avoid.

The documentation on layers is very basic both in Amplify's documentation as well as in the Lambda documentation. I want to avoid sharp gets bundled in the deployment package but it doesn't 😦

[sonic steppe]

hey quickly tested using a sharp lambda layer and but did not run into this issue
refer to an example here
https://github.com/chrisbonifacio/amplify-gen2-samples/tree/main/sharp

[ashen python]

Thank you @sonic steppe . That looks actually pretty simple. I compared our settings (tsconfig etc.) and they all look very similar. I can only recognize your Lambda handler is a .js file where mine is a .ts file. Not sure if that makes any difference. One question, do you face any issues in your sandbox environment? I can see you do not have sharp in your ./package.json dependencies. If that is the case for me, the sandbox doesn't deploy as it claims the missing dependency.

[sonic steppe]

@ashen python hey you can just use ts file as file type. you would need to have sharp installed locally. but at runtime lambda will use this from the layer.
Thanks for pointing that out. using the js file skips checking the imports and types the js file gets deployed as-is. you can use the js file but you would miss out on the type safety goodness

[ashen python]

Makes sense for the production environment; for the sandbox environment I made the experience, defineFunction is also bundling the locally installed sharp and then the invocations run into issues. But maybe using .js for that one function is totally fine for me. I will test this and let you know

[sonic steppe]

https://github.com/ykethan/amplify-gen2-samples/tree/main/gen2-lambda-layer/layers
could you try deploying this x86 zip to a layer and replace the arn in defineFunction. wanted to quickly check if this new zip may fix the issue
this function example does use a ts file for the handler

[ashen python]

Oh wow, it finally works. I am super happy. I have 2 Lambdas, one using the Amplify defineFunction functionality which reacts on an image upload on S3 and creates a base64 image string (using sharp) which is stored in DynamoDB. As this Lambda is interacting with both storage and data backends I am happy I can use the defineFunction functionality as it makes permission management so much easier. My 2nd Lambda is a fallback origin for a cloudfront distribution. As CloudFront is not supported out of the box by Amplify I have to create a stack and deploy the distribution and some other stuff using CDK. This Lambda then takes the original image, creates an optimized image and stores it in a separate bucket which I create within the stack as well. This bucket is also the primary origin for the CF distribution. If it is a cache miss and CF doesn't find the expected file, the Lambda creates this optimized image and returns it to CF. This Lambda uses sharp as well for this operation. The architecture is based on this article.
Thanks to your guidance @sonic steppe I was able to implement this and I think it doesn't even need Docker anymore. I will verify this.
Again, I am super happy. I was creating an image gallery comparable to this one. However, with no image optimization and just getting the files from S3, the performance was really bad. Now, with this implementation the user experience will be much better. I got a lot of headaches, implementing this solution. It would be amazing if Amplify (and CloudFront as well) could actually do this out of the box. AWS is a little bit behind on that front compared to Cloudflare or Cloudinary. Thanks for your help on this matter

[sonic steppe]

@ashen python glad to hear you were able to get it working. to improve the S3 transfer performance, you could you try and see if enabling transfer acceleration could help with this.
https://docs.amplify.aws/vue/build-a-backend/storage/extend-s3-resources/#example---enable-transfer-acceleration
but if you are looking for a native support for a compression/image optimization in the Amplify library, could you file a feature request on the Amplify JS repository to track this.

[tight coyote]

✅ - Issues using a Lambda Layer