bug/Execution speed is very slow in AWS LAMBDA environment

[cds-code]

Describe the bug
Execution speed is very slow in AWS Lambda environment with extract text from txt,pdf,docx etc, but very fast in local windows environment.

[scanny]

@cds-code can you describe how you are running unstructured in AWS Lambda?

[cds-code]

Im running a docker image in AWS Lambda

FROM public.ecr.aws/docker/library/python:3.11.6-slim-bookworm
RUN apt-get update && apt-get install -y \
    # unstructured package requirements for file type detection
    libmagic-mgc libmagic1 \
RUN [ "python3", "-c", "import nltk; nltk.download('punkt', download_dir='/usr/local/nltk_data')" ]
RUN [ "python3", "-c", "import nltk; nltk.download('averaged_perceptron_tagger', download_dir='/usr/local/nltk_data')" ]

pip install 
unstructured
unstructured[pdf]
unstructured[docx]
unstructured[xlsx]
unstructured[pptx]
unstructured[md]

from unstructured.partition.auto import partition
partition(filename="XXXXX.pdf")

[scanny]

Have you accounted for spin-up (cold-start) time of the Lambda instance? Like only start timing after receiving the first response?

Also, can you provide some specific timings?

[scanny]

And how much memory is allocated to the Lambda instance?

[danielfornarini]

I have the same problem in AWS Batch running on fargate. I allocated 2 vCPUs and 4 GB of memory

[cds-code]

Have you accounted for spin-up (cold-start) time of the Lambda instance? Like only start timing after receiving the first response?

Also, can you provide some specific timings?

Does not contain Lambda instance cold-start time. just partition(filename="XXXXX.pdf") .
The memory of Lambda instance is 1024MB, 2048MB, 4096MB, and the execution time is the same.

When the program is executed three times in a loop. Only the first time was very long.
It seems that the initialization of unstructured takes a lot of time in AWS Lambda environment

 ```
| 2024-04-24T05:10:30.306Z | --1--- 2024-04-24 05:09:52.577591 ⇒loop 1
| 2024-04-24T05:10:30.306Z | --2--- 2024-04-24 05:10:07.032373 ⇒loop 1
| 2024-04-24T05:10:30.306Z | --1--- 2024-04-24 05:10:07.032521 ⇒loop 2
| 2024-04-24T05:10:30.306Z | --2--- 2024-04-24 05:10:08.106393 ⇒loop 2
| 2024-04-24T05:10:30.306Z | --1--- 2024-04-24 05:10:08.106490 ⇒loop 3
| 2024-04-24T05:10:30.306Z | --2--- 2024-04-24 05:10:08.840540 ⇒loop 3

[scanny]

@cds-code what "extras" are you installing with unstructured? Like where you do pip install unstructured[x], what x do you use? NM, I see you posted that above :)

[scanny]

Also, just out of curiosity, can you give me a sense of the cold-start times you've seen?

[adieuadieu]

We are able to run Unstructured in a container on AWS Lambda without issue (or, well, there are issues, but we can work around them.)

Things to consider (sorry that these points are a bit............unstructured):

• If your container image is large (e.g. 1GB or more), each time after you deploy your container (e.g. when you deploy your Lambda function), on the initial (first) invocation after, it can take quite a while (e.g. 30 seconds) for Lambda to download your container image from AWS ECR. This only happens on the first invocation after deployment of a new image. After that, Lambda will have cached the image. It looks like you install NLTK models and magick libs into the container, so your image is likely around 1 GB.
• • Since you mention that it's only the first invocation that's slow, this initial downloading/caching from ECR is probably what you're seeing
• • Consider using multi-stage builds to reduce the number of new layers from the container Lambda has to download from ECR.
• I noticed you're not pre-downloading/installing any of the models that Unstructured uses to partition/read/parse PDF files. Unless you're using the "fast" strategy, it's likely that Unstructured will try to use a model to parse PDF files. Unstructured lazy-loads these models, so they're not downloaded until Unstructured needs them, which will take some time the first time — That's assuming you've configured your environment to download these models into /tmp with enough disk space — the only writeable location in Lambda. Also, note that Unstructureds current default model detectron2onnx won't work in AWS Lambda because of an underlying issue in onnxruntime->pytorch->cpuinfo. Use the model-override parameter to specify chipper instead-->though this will further balloon your container image. Consider using only the fast strategy instead. e.g. partition(file, strategy="fast") — make sure to add tesseract to your container image.

If you're already accounting for the ECR download/caching time, one other thing you can try is to run a "fake" partition script during the build of your container image. This will help "warm up" any libraries/dependencies which may want to run some initial first-time setup tasks (like building/caching fonts, or downloading models). For example, in the same way you "warm up" the NLTK libraries, you could add a RUN step:

COPY XXXXX.pdf /tmp/XXXXX.pdf
RUN [ "python3", "-c", "from unstructured.partition.auto import partition; partition(filename='/tmp/XXXXX.pdf')" ]

But, this will potentially exacerbate the first point about the container image size.

[cds-code]

This works for me thanks.

COPY XXXXX.pdf /tmp/XXXXX.pdf
RUN [ "python3", "-c", "from unstructured.partition.auto import partition; partition(filename='/tmp/XXXXX.pdf')" ]

Does unstructured itself have an initial load method to Implement the above function?