Machine learning on CPUs without AVX: howto

[bertmelis]

Prepare

Create the working directories

mkdir -p build-tf
mkdir my-immich-ml
mkdir immich

Create the container for building Tensorflow

Containerfile in build-tf' directory:

FROM debian:bullseye-slim

RUN apt-get update && apt-get install -y curl git python3 python3-dev python3-pip apt-transport-https curl gnupg
RUN curl -fsSL https://bazel.build/bazel-release.pub.gpg | gpg --dearmor >bazel-archive-keyring.gpg && \
    mv bazel-archive-keyring.gpg /usr/share/keyrings && \
    echo "deb [arch=amd64 signed-by=/usr/share/keyrings/bazel-archive-keyring.gpg] https://storage.googleapis.com/bazel-apt stable jdk1.8" | tee /etc/apt/sources.list.d/bazel.list && \
    apt-get update && apt-get install -y bazel bazel-5.0.0

RUN pip3 install --upgrade pip && \
    pip install -U --user pip numpy wheel packaging requests opt_einsum && \
    pip install -U --user keras_preprocessing --no-deps

WORKDIR /tensorflow
VOLUME /tensorflow/output

COPY entrypoint.sh /tensorflow/entrypoint.sh
RUN chmod +x /tensorflow/entrypoint.sh

ENTRYPOINT ["/tensorflow/entrypoint.sh"]

CMD ["sh"]

entrypoint.sh in 'build-tf' directory

#!/bin/sh

cd output
git clone https://github.com/tensorflow/tensorflow.git
if cd tensorflow
  then git pull
else
  git clone https://github.com/tensorflow/tensorflow.git
  cd tensorflow
fi
git checkout v3.19.0
./configure
bazel build --config=mkl -c opt --copt=-march=native --config=monolithic //tensorflow/tools/lib_package:libtensorflow
cp bazel-bin/tensorflow/tools/lib_package/libtensorflow.tar.gz /tensorflow/output/libtensorflow.tar.gz

Containerfile in 'my-immich-ml'

FROM altran1502/immich-machine-learning:release

COPY libtensorflow.tar.gz node_modules/@tensorflow/tfjs-node/deps/libtensorflow.tar.gz

RUN apt-get update && \
    apt-get -y --no-install-recommends install libomp-dev && \
    apt-get clean && \
    cd node_modules/@tensorflow/tfjs-node/deps && \
    tar -xf libtensorflow.tar.gz && \
    rm libtensorflow.tar.gz

This is what my directory structure looks like:

home dir
  |
  +-- build-tf
  |     |
  |     +-- Containerfile
  |
  +-- my-immich-ml
  |     |
  |     +-- Containerfile
  |
  +-- immich
        |
        +-- podman-compose.yml

Start building!

Build Tensorflow

Use Docker or Podman, inside the 'build-tf' directory

podman build -t tfbuild .
podman run -d -v ../my-immich-ml:/tensorflow/output:z --name tfbuild tfbuild

Running the container takes a long time. You can check the progress from time to time with podman logs tfbuild.

When done, you will find a file called libtensorflow.tar.gz in the my-immich-ml directory.

Build the machine learning container

Use Docker or Podman, inside the 'my-immich-ml' directory

podman build -t my-immich-machine-learning .

Launch Immich with your custom ML container

adjust the docker-compose file:

  immich-machine-learning:
    # point to your own ML image
    #image: altran1502/immich-machine-learning:release
    image: localhost/my-immich-machine-learning:latest
    entrypoint: ["/bin/sh", "./entrypoint.sh"]
    volumes:
      - ${UPLOAD_LOCATION}:/usr/src/app/upload:z
    env_file:
      - .env
    environment:
      - NODE_ENV=production
    depends_on:
      - database
    restart: always

Launch as usual.

[bertmelis]

One line it the ML container's log is intriguing me though:

2022-07-04 09:05:08.932552: I tensorflow/core/platform/cpu_feature_guard.cc:193] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  SSE3 SSE4.1 SSE4.2

To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.

More to read here: https://www.intel.com/content/www/us/en/developer/articles/guide/optimization-for-tensorflow-installation-guide.html

I don't have a spare 6 hours at the moment to rebuild the while thing so... to be continued.

[alextran1502]

Hey, thank you, this implementation will make a lot of people happy.

[bertmelis]

Edited: made an easier version.

[bertmelis]

Guard the file libtensorflow.tar.gz with your life. Immich is under development and you will need the file every time you want to update the machine-learning image.

[bertmelis]

To avoid the warnings (and improve performance), I should add a few options to my build for my J4105 cpu:

--copt=-msse --copt=-msse2 --copt=-msse3 --copt=-msse4.1 --copt=-msse4.2

So entrypoint.sh for building Tensorflow should look like this:

#!/bin/sh

cd output
git clone https://github.com/tensorflow/tensorflow.git
if cd tensorflow
  then git pull
else
  git clone https://github.com/tensorflow/tensorflow.git
  cd tensorflow
fi
git checkout v2.9.1
./configure
bazel build --config=mkl -c opt --copt=-msse --copt=-msse2 --copt=-msse3 --copt=-msse4.1 --copt=-msse4.2 --copt=-march=native --config=monolithic //tensorflow/tools/lib_package:libtensorflow
cp bazel-bin/tensorflow/tools/lib_package/libtensorflow.tar.gz /tensorflow/output/libtensorflow.tar.gz

I don't know why -march=native doesn't pick up the CPUs features. Various sources on Stackoverflow and other websites are contradictory. 🤐

[bertmelis]

If there is interest, I might pursue crosscompiling for Raspberry Pi.

[DrMxrcy]

I think a few people run this on Pis, so I would be interested!

[untenops]

Sorry probably the wrong place for this, but couldn't find and answer searching online.
I get through all of the instructions

podman build -t my-immich-machine-learning .
gives me,

STEP 1/4: FROM altran1502/immich-machine-learning:release
STEP 2/4: COPY libtensorflow.tar.gz node_modules/@tensorflow/tfjs-node/deps/libtensorflow.tar.gz
--> Using cache d05709722d816c3c03fe25e09a6be3ff6ab0e261d6620f28d90c1bad773f4678
--> d05709722d8
STEP 3/4: RUN cd node_modules/@tensorflow/tfjs-node/deps && tar -xf libtensorflow.tar.gz
--> Using cache 574ac4b6c3e592c92bac61f50a3b77657ad87e19c880dd7d3572f98d301a6152
--> 574ac4b6c3e
STEP 4/4: RUN npm run build
--> Using cache 4a6ea59f80f54fdc74b29ac529fe1640e408c92b38bf5851c37f693141f30313
COMMIT my-immich-machine-learning
--> 4a6ea59f80f
Successfully tagged localhost/my-immich-machine-learning:latest

Looks good, I think, but when I try to start my docker compose, I get this

Error response from daemon: Get "http://localhost/v2/": dial tcp 127.0.0.1:80: connect: connection refused

Any thought on what happening, how to fix?

[untenops]

Ive never used podman before this. I installed podman just to follow the instructions above. Then I used docker-compose to try and start it.

[bertmelis]

Use Docker and docker-compose or podman and podman-compose.

On the surface they look the same but underneath they are different. Pick a flavour and stick with it.

You don't have to rebuild Tensorflow. Only build the ML container in Docker.

[untenops]

Ok I'll play with it some more. I've never used podman before this. But half way there already so finished with podman. It starts up but I get the same error I get, if just using straight docker without the custom ML container

In and browser
500

Converting circular structure to JSON
--> starting at object with constructor 'Socket'
| property '_httpMessage' -> object with constructor 'ClientRequest'
--- property 'socket' closes the circle

TypeError: Converting circular structure to JSON
--> starting at object with constructor 'Socket'
| property '_httpMessage' -> object with constructor 'ClientRequest'
--- property 'socket' closes the circle

[alextran1502]

What does your .env look like?

[untenops]

That was the problem. I had VITE_SERVER_ENDPOINT with out https://
Adding that got me up a running. Thank you so much. Can't wait to play around with Immich.

[sbkg0002]

Does anyone know what might be wrong? I tried to compile it (on the same CPU: J4105), but it keeps erroring:

[7,078 / 8,344] Compiling tensorflow/core/common_runtime/eager/shape_inference.cc; 14s local ... (4 actions running)
[7,153 / 8,344] Compiling tensorflow/compiler/mlir/tensorflow/ir/tf_ops.cc; 2031s local ... (4 actions running)
ERROR: /tensorflow/output/tensorflow/tensorflow/compiler/mlir/tensorflow/BUILD:574:11: Compiling tensorflow/compiler/mlir/tensorflow/ir/tf_ops.cc failed: (Exit 1): gcc failed: error executing command /usr/bin/gcc -U_FORTIFY_SOURCE -fstack-protector -Wall -Wunused-but-set-parameter -Wno-free-nonheap-object -fno-omit-frame-pointer -g0 -O2 '-D_FORTIFY_SOURCE=1' -DNDEBUG -ffunction-sections ... (remaining 201 arguments skipped)
gcc: fatal error: Killed signal terminated program cc1plus
compilation terminated.
Target //tensorflow/tools/lib_package:libtensorflow failed to build
Use --verbose_failures to see the command lines of failed build steps.
INFO: Elapsed time: 18628.538s, Critical Path: 3605.21s
INFO: 7157 processes: 537 internal, 6620 local.
FAILED: Build did NOT complete successfully
FAILED: Build did NOT complete successfully
cp: cannot stat 'bazel-bin/tensorflow/tools/lib_package/libtensorflow.tar.gz': No such file or directory

[bertmelis]

How much RAM (and swap) do you have? At least, lack of free memory is something various sources on the Internet point at for this error. You might want to run less compilation processes in parallel at the cost of having to wait longer.

[sbkg0002]

Thanks @bertmelis; I have 16GB of Ram, but it truned our to be not enough. Added 10GB Swap and now have my libtensorflow.tar.gz! Thanks again!

[bertmelis]

Hmmm, I also have 16GB. But I might have swap too, never looked at it. Maybe Rocky Linux enables swap out of the box.

Anyway, happy to help and happy it worked!

[sbkg0002]

For some reason my ml container fails to start and throws the following error:

Error: libiomp5.so: cannot open shared object file: No such file or directory
    at Object.Module._extensions..node (node:internal/modules/cjs/loader:1189:18)
    at Module.load (node:internal/modules/cjs/loader:981:32)
    at Function.Module._load (node:internal/modules/cjs/loader:822:12)
    at Module.require (node:internal/modules/cjs/loader:1005:19)
    at require (node:internal/modules/cjs/helpers:102:18)
    at Object.<anonymous> (/usr/src/app/node_modules/@tensorflow/tfjs-node/dist/index.js:60:16)
    at Module._compile (node:internal/modules/cjs/loader:1105:14)
    at Object.Module._extensions..js (node:internal/modules/cjs/loader:1159:10)
    at Module.load (node:internal/modules/cjs/loader:981:32)
    at Function.Module._load (node:internal/modules/cjs/loader:822:12) {
  code: 'ERR_DLOPEN_FAILED'
}

libiomp is not a package in the latest debian versions, so I have no clue how to add this dependency. Any ideas?

EDIT:
Got it to work; the Dockerfile in my-immich-ml needed one extra dependency:

FROM altran1502/immich-machine-learning:release

COPY libtensorflow.tar.gz node_modules/@tensorflow/tfjs-node/deps/libtensorflow.tar.gz

RUN --mount=target=/var/lib/apt/lists,type=cache,sharing=locked \
    --mount=target=/var/cache/apt,type=cache,sharing=locked \
    rm -f /etc/apt/apt.conf.d/docker-clean \
    && apt-get update \
    && apt-get -y --no-install-recommends install \
        libomp-dev

RUN cd node_modules/@tensorflow/tfjs-node/deps && \
    tar -xf libtensorflow.tar.gz

RUN npm run build

[bertmelis]

Updated instructions to match tfjs version to Immich and added libomp-dev dependency (thanks @sbkg0002 )

[JVT038]

It doesn't work for me, I got this error when running the container: standard_init_linux.go:230: exec user process caused: no such file or directory

My Dockerfile for the libtensorflow.tar.gz file:

FROM debian:bullseye-slim

RUN apt-get update && apt-get install -y curl git python3 python3-dev python3-pip apt-transport-https curl gnupg
RUN curl -fsSL https://bazel.build/bazel-release.pub.gpg | gpg --dearmor >bazel-archive-keyring.gpg && \
    mv bazel-archive-keyring.gpg /usr/share/keyrings && \
    echo "deb [arch=amd64 signed-by=/usr/share/keyrings/bazel-archive-keyring.gpg] https://storage.googleapis.com/bazel-apt stable jdk1.8" | tee /etc/apt/sources.list.d/bazel.list && \
    apt-get update && apt-get install -y bazel bazel-5.0.0

RUN pip3 install --upgrade pip && \
    pip install -U --user pip numpy wheel packaging requests opt_einsum && \
    pip install -U --user keras_preprocessing --no-deps

WORKDIR /tensorflow
VOLUME /tensorflow/output

COPY entrypoint.sh /tensorflow/entrypoint.sh
RUN chmod +x /tensorflow/entrypoint.sh

ENTRYPOINT ["/tensorflow/entrypoint.sh"]

CMD ["sh"]

entrypoint.sh:

#!/bin/sh

cd output
git clone https://github.com/tensorflow/tensorflow.git
if cd tensorflow
  then git pull
else
  git clone https://github.com/tensorflow/tensorflow.git
  cd tensorflow
fi
git checkout v3.19.0
./configure
bazel build --config=mkl -c opt --copt=-march=native --config=monolithic //tensorflow/tools/lib_package:libtensorflow
cp bazel-bin/tensorflow/tools/lib_package/libtensorflow.tar.gz /tensorflow/output/libtensorflow.tar.gz

Docker build command: docker build -t tfbuild .
Docker run command: docker run -v /path/to/immich/immich-ml:/tensorflow/output:z --name tfbuild tfbuild

[bertmelis]

Can't really tell where it fails from that message.
You might need to drop the :z in the docker run command. I needed to add this to keep SELinux happy.

Google tells me this is caused by CRLF instead of LF line endings.

[JVT038]

Thanks! Changing from CRLF to LF fixed it!

[JVT038]

So I got another error after letting the container run:

fatal: destination path 'tensorflow' already exists and is not an empty directory.
hint: Pulling without specifying how to reconcile divergent branches is
hint: discouraged. You can squelch this message by running one of the following
hint: commands sometime before your next pull:
hint:
hint:   git config pull.rebase false  # merge (the default strategy)
hint:   git config pull.rebase true   # rebase
hint:   git config pull.ff only       # fast-forward only
hint:
hint: You can replace "git config" with "git config --global" to set a default
hint: preference for all repositories. You can also pass --rebase, --no-rebase,
hint: or --ff-only on the command line to override the configured default per
hint: invocation.
Already up to date.
error: pathspec 'v3.19.0' did not match any file(s) known to git
You have bazel 5.3.0 installed.
Please specify the location of python. [Default is /usr/bin/python3]:

Found possible Python library paths:
  /usr/lib/python3/dist-packages
  /usr/local/lib/python3.9/dist-packages
Please input the desired Python library path to use.  Default is [/usr/lib/python3/dist-packages]
Do you wish to build TensorFlow with ROCm support? [y/N]: No ROCm support will be enabled for TensorFlow.

Extracting Bazel installation...
Do you wish to build TensorFlow with CUDA support? [y/N]: No CUDA support will be enabled for TensorFlow.

Do you wish to download a fresh release of clang? (Experimental) [y/N]: Clang will not be downloaded.

Please specify optimization flags to use during compilation when bazel option "--config=opt" is specified [Default is -Wno-sign-compare]:

Would you like to interactively configure ./WORKSPACE for Android builds? [y/N]: Not configuring the WORKSPACE for Android builds.

Preconfigured Bazel build configs. You can use any of the below by adding "--config=<>" to your build command. See .bazelrc for more details.
        --config=mkl            # Build with MKL support.
        --config=mkl_aarch64    # Build with oneDNN and Compute Library for the Arm Architecture (ACL).
        --config=monolithic     # Config for mostly static monolithic build.
        --config=numa           # Build with NUMA support.
        --config=dynamic_kernels        # (Experimental) Build kernels into separate shared objects.
        --config=v1             # Build with TensorFlow 1 API instead of TF 2 API.
Preconfigured Bazel build configs to DISABLE default on features:
        --config=nogcp          # Disable GCP support.
        --config=nonccl         # Disable NVIDIA NCCL support.
Configuration finished
Starting local Bazel server and connecting to it...
INFO: Options provided by the client:
  Inherited 'common' options: --isatty=0 --terminal_columns=80
INFO: Reading rc options for 'build' from /tensorflow/output/tensorflow/.bazelrc:
  Inherited 'common' options: --experimental_repo_remote_exec
INFO: Reading rc options for 'build' from /tensorflow/output/tensorflow/.bazelrc:
  'build' options: --define framework_shared_object=true --define=use_fast_cpp_protos=true --define=allow_oversize_protos=true --spawn_strategy=standalone -c opt --announce_rc --define=grpc_no_ares=true --noincompatible_remove_legacy_whole_archive --enable_platform_specific_config --define=with_xla_support=true --config=short_logs --config=v2 --define=no_aws_support=true --define=no_hdfs_support=true --experimental_cc_shared_library --experimental_link_static_libraries_once=false
INFO: Reading rc options for 'build' from /tensorflow/output/tensorflow/.tf_configure.bazelrc:
  'build' options: --action_env PYTHON_BIN_PATH=/usr/bin/python3 --action_env PYTHON_LIB_PATH=/usr/lib/python3/dist-packages --python_path=/usr/bin/python3
INFO: Reading rc options for 'build' from /tensorflow/output/tensorflow/.bazelrc:
  'build' options: --deleted_packages=tensorflow/compiler/mlir/tfrt,tensorflow/compiler/mlir/tfrt/benchmarks,tensorflow/compiler/mlir/tfrt/jit/python_binding,tensorflow/compiler/mlir/tfrt/jit/transforms,tensorflow/compiler/mlir/tfrt/python_tests,tensorflow/compiler/mlir/tfrt/tests,tensorflow/compiler/mlir/tfrt/tests/ir,tensorflow/compiler/mlir/tfrt/tests/analysis,tensorflow/compiler/mlir/tfrt/tests/jit,tensorflow/compiler/mlir/tfrt/tests/lhlo_to_tfrt,tensorflow/compiler/mlir/tfrt/tests/lhlo_to_jitrt,tensorflow/compiler/mlir/tfrt/tests/tf_to_corert,tensorflow/compiler/mlir/tfrt/tests/tf_to_tfrt_data,tensorflow/compiler/mlir/tfrt/tests/saved_model,tensorflow/compiler/mlir/tfrt/transforms/lhlo_gpu_to_tfrt_gpu,tensorflow/core/runtime_fallback,tensorflow/core/runtime_fallback/conversion,tensorflow/core/runtime_fallback/kernel,tensorflow/core/runtime_fallback/opdefs,tensorflow/core/runtime_fallback/runtime,tensorflow/core/runtime_fallback/util,tensorflow/core/tfrt/common,tensorflow/core/tfrt/eager,tensorflow/core/tfrt/eager/backends/cpu,tensorflow/core/tfrt/eager/backends/gpu,tensorflow/core/tfrt/eager/core_runtime,tensorflow/core/tfrt/eager/cpp_tests/core_runtime,tensorflow/core/tfrt/gpu,tensorflow/core/tfrt/run_handler_thread_pool,tensorflow/core/tfrt/runtime,tensorflow/core/tfrt/saved_model,tensorflow/core/tfrt/graph_executor,tensorflow/core/tfrt/saved_model/tests,tensorflow/core/tfrt/tpu,tensorflow/core/tfrt/utils
INFO: Found applicable config definition build:short_logs in file /tensorflow/output/tensorflow/.bazelrc: --output_filter=DONT_MATCH_ANYTHING
INFO: Found applicable config definition build:v2 in file /tensorflow/output/tensorflow/.bazelrc: --define=tf_api_version=2 --action_env=TF2_BEHAVIOR=1
INFO: Found applicable config definition build:mkl in file /tensorflow/output/tensorflow/.bazelrc: --define=build_with_mkl=true --define=enable_mkl=true --define=tensorflow_mkldnn_contraction_kernel=0 --define=build_with_openmp=true -c opt
INFO: Found applicable config definition build:monolithic in file /tensorflow/output/tensorflow/.bazelrc: --define framework_shared_object=false --experimental_link_static_libraries_once=false
INFO: Found applicable config definition build:linux in file /tensorflow/output/tensorflow/.bazelrc: --host_copt=-w --copt=-Wno-all --copt=-Wno-extra --copt=-Wno-deprecated --copt=-Wno-deprecated-declarations --copt=-Wno-ignored-attributes --copt=-Wunused-result --copt=-Werror=unused-result --define=PREFIX=/usr --define=LIBDIR=$(PREFIX)/lib --define=INCLUDEDIR=$(PREFIX)/include --define=PROTOBUF_INCLUDE_PATH=$(PREFIX)/include --cxxopt=-std=c++17 --host_cxxopt=-std=c++17 --config=dynamic_kernels --distinct_host_configuration=false --experimental_guard_against_concurrent_changes
INFO: Found applicable config definition build:dynamic_kernels in file /tensorflow/output/tensorflow/.bazelrc: --define=dynamic_loaded_kernels=true --copt=-DAUTOLOAD_DYNAMIC_KERNELS
Loading:
Loading: 0 packages loaded
Loading: 0 packages loaded
Loading: 0 packages loaded
Loading: 0 packages loaded
Loading: 0 packages loaded
Loading: 0 packages loaded
Loading: 0 packages loaded
Loading: 0 packages loaded
WARNING: Download from https://storage.googleapis.com/mirror.tensorflow.org/github.com/tensorflow/runtime/archive/08aff0bd38c71cdc8d95c05045f6d32e73125c80.tar.gz failed: class java.io.FileNotFoundException GET returned 404 Not Found
Loading: 0 packages loaded
Loading: 0 packages loaded
Loading: 0 packages loaded
Loading: 0 packages loaded
Loading: 0 packages loaded
Loading: 0 packages loaded
WARNING: Download from https://storage.googleapis.com/mirror.tensorflow.org/github.com/llvm/llvm-project/archive/b60f801607543f0c64dd99f9a5741ae9c69404ee.tar.gz failed: class java.io.FileNotFoundException GET returned 404 Not Found
Loading: 0 packages loaded
Loading: 0 packages loaded
Analyzing: target //tensorflow/tools/lib_package:libtensorflow (1 packages loaded, 0 targets configured)
Analyzing: target //tensorflow/tools/lib_package:libtensorflow (49 packages loaded, 65 targets configured)
WARNING: Download from https://mirror.bazel.build/github.com/bazelbuild/rules_cc/archive/081771d4a0e9d7d3aa0eed2ef389fa4700dfb23e.tar.gz failed: class java.io.FileNotFoundException GET returned 404 Not Found
Analyzing: target //tensorflow/tools/lib_package:libtensorflow (76 packages loaded, 65 targets configured)
Analyzing: target //tensorflow/tools/lib_package:libtensorflow (237 packages loaded, 8507 targets configured)
INFO: Analyzed target //tensorflow/tools/lib_package:libtensorflow (293 packages loaded, 21332 targets configured).
INFO: Found 1 target...
[0 / 9] [Prepa] Writing file tensorflow/tools/lib_package/libtensorflow.manifest
// more compiling lines...
[7,015 / 8,816] Compiling tensorflow/core/kernels/conv_ops.cc; 3551s local ... (4 actions running) 
ERROR: /tensorflow/output/tensorflow/tensorflow/core/kernels/BUILD:4118:18: Compiling tensorflow/core/kernels/conv_ops.cc failed: (Exit 1): gcc failed: error executing command /usr/bin/gcc -U_FORTIFY_SOURCE -fstack-protector -Wall -Wunused-but-set-parameter -Wno-free-nonheap-object -fno-omit-frame-pointer -g0 -O2 '-D_FORTIFY_SOURCE=1' -DNDEBUG -ffunction-sections ... (remaining 248 arguments skipped)
gcc: fatal error: Killed signal terminated program cc1plus
compilation terminated.
Target //tensorflow/tools/lib_package:libtensorflow failed to build
Use --verbose_failures to see the command lines of failed build steps.
INFO: Elapsed time: 25801.456s, Critical Path: 5608.61s
INFO: 7019 processes: 617 internal, 6402 local.
FAILED: Build did NOT complete successfully
FAILED: Build did NOT complete successfully
cp: cannot stat 'bazel-bin/tensorflow/tools/lib_package/libtensorflow.tar.gz': No such file or directory

Run command: docker run -d -v /path/to/immich/immich-ml:/tensorflow/output --name tfbuild tfbuild

[dorianim]

You can run free -h to check

[JVT038]

Result:

              total        used        free      shared  buff/cache   available
Mem:           11Gi       5.2Gi       301Mi       317Mi       6.1Gi       5.8Gi
Swap:         8.9Gi       6.0Gi       2.9Gi

So I guess I can't build Tensorflow, because I don't have enough memory?

[bertmelis]

Memory won't be the problem.

Maybe, as the log says you should add --verbose_failures to the build command.

[JVT038]

I did (I edited the entrypoint.sh file, added the parameter and rebuilt the image), and this is the complete output:

// some more compiling lines
[7,920 / 8,838] Compiling tensorflow/compiler/mlir/tensorflow/ir/tf_ops_n_z.cc; 2855s local ... (4 actions running)
ERROR: /tensorflow/output/tensorflow/tensorflow/compiler/mlir/tensorflow/BUILD:591:11: Compiling tensorflow/compiler/mlir/tensorflow/ir/tf_ops.cc failed: (Exit 1): gcc failed: error executing command
  (cd /root/.cache/bazel/_bazel_root/7f9fba3f745520adec472847d4c8aa03/execroot/org_tensorflow && \
  exec env - \
    PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin \
    PWD=/proc/self/cwd \
    PYTHON_BIN_PATH=/usr/bin/python3 \
    PYTHON_LIB_PATH=/usr/lib/python3/dist-packages \
    TF2_BEHAVIOR=1 \
  /usr/bin/gcc -U_FORTIFY_SOURCE -fstack-protector -Wall -Wunused-but-set-parameter -Wno-free-nonheap-object -fno-omit-frame-pointer -g0 -O2 '-D_FORTIFY_SOURCE=1' -DNDEBUG -ffunction-sections -fdata-sections '-std=c++0x' -MD -MF bazel-out/k8-opt/bin/tensorflow/compiler/mlir/tensorflow/_objs/tensorflow_ops/tf_ops.pic.d '-frandom-seed=bazel-out/k8-opt/bin/tensorflow/compiler/mlir/tensorflow/_objs/tensorflow_ops/tf_ops.pic.o' -fPIC -DEIGEN_MPL2_ONLY '-DEIGEN_MAX_ALIGN_BYTES=64' -DHAVE_SYS_UIO_H -DTF_USE_SNAPPY '-DLLVM_ON_UNIX=1' '-DHAVE_BACKTRACE=1' '-DBACKTRACE_HEADER=<execinfo.h>' '-DLTDL_SHLIB_EXT=".so"' '-DLLVM_PLUGIN_EXT=".so"' '-DLLVM_ENABLE_THREADS=1' '-DHAVE_DEREGISTER_FRAME=1' '-DHAVE_LIBPTHREAD=1' '-DHAVE_PTHREAD_GETNAME_NP=1' '-DHAVE_PTHREAD_H=1' '-DHAVE_PTHREAD_SETNAME_NP=1' '-DHAVE_REGISTER_FRAME=1' '-DHAVE_SETENV_R=1' '-DHAVE_STRERROR_R=1' '-DHAVE_SYSEXITS_H=1' '-DHAVE_UNISTD_H=1' -D_GNU_SOURCE '-DHAVE_LINK_H=1' '-DHAVE_LSEEK64=1' '-DHAVE_MALLINFO=1' '-DHAVE_SBRK=1' '-DHAVE_STRUCT_STAT_ST_MTIM_TV_NSEC=1' '-DLLVM_NATIVE_ARCH="X86"' '-DLLVM_NATIVE_ASMPARSER=LLVMInitializeX86AsmParser' '-DLLVM_NATIVE_ASMPRINTER=LLVMInitializeX86AsmPrinter' '-DLLVM_NATIVE_DISASSEMBLER=LLVMInitializeX86Disassembler' '-DLLVM_NATIVE_TARGET=LLVMInitializeX86Target' '-DLLVM_NATIVE_TARGETINFO=LLVMInitializeX86TargetInfo' '-DLLVM_NATIVE_TARGETMC=LLVMInitializeX86TargetMC' '-DLLVM_NATIVE_TARGETMCA=LLVMInitializeX86TargetMCA' '-DLLVM_HOST_TRIPLE="x86_64-unknown-linux-gnu"' '-DLLVM_DEFAULT_TARGET_TRIPLE="x86_64-unknown-linux-gnu"' -D__STDC_LIMIT_MACROS -D__STDC_CONSTANT_MACROS -D__STDC_FORMAT_MACROS '-DBLAKE3_USE_NEON=0' -DBLAKE3_NO_AVX2 -DBLAKE3_NO_AVX512 -DBLAKE3_NO_SSE2 -DBLAKE3_NO_SSE41 -iquote . -iquote bazel-out/k8-opt/bin -iquote external/com_google_absl -iquote bazel-out/k8-opt/bin/external/com_google_absl -iquote external/nsync -iquote bazel-out/k8-opt/bin/external/nsync -iquote external/eigen_archive -iquote bazel-out/k8-opt/bin/external/eigen_archive -iquote external/gif -iquote bazel-out/k8-opt/bin/external/gif -iquote external/libjpeg_turbo -iquote bazel-out/k8-opt/bin/external/libjpeg_turbo -iquote external/com_google_protobuf -iquote bazel-out/k8-opt/bin/external/com_google_protobuf -iquote external/zlib -iquote bazel-out/k8-opt/bin/external/zlib -iquote external/com_googlesource_code_re2 -iquote bazel-out/k8-opt/bin/external/com_googlesource_code_re2 -iquote external/farmhash_archive -iquote bazel-out/k8-opt/bin/external/farmhash_archive -iquote external/fft2d -iquote bazel-out/k8-opt/bin/external/fft2d -iquote external/highwayhash -iquote bazel-out/k8-opt/bin/external/highwayhash -iquote external/double_conversion -iquote bazel-out/k8-opt/bin/external/double_conversion -iquote external/snappy -iquote bazel-out/k8-opt/bin/external/snappy -iquote external/llvm-project -iquote bazel-out/k8-opt/bin/external/llvm-project -iquote external/llvm_terminfo -iquote bazel-out/k8-opt/bin/external/llvm_terminfo -iquote external/llvm_zlib -iquote bazel-out/k8-opt/bin/external/llvm_zlib -Ibazel-out/k8-opt/bin/external/llvm-project/mlir/_virtual_includes/BuiltinAttributeInterfacesIncGen -Ibazel-out/k8-opt/bin/external/llvm-project/mlir/_virtual_includes/BuiltinAttributesIncGen -Ibazel-out/k8-opt/bin/external/llvm-project/mlir/_virtual_includes/BuiltinDialectIncGen -Ibazel-out/k8-opt/bin/external/llvm-project/mlir/_virtual_includes/BuiltinLocationAttributesIncGen -Ibazel-out/k8-opt/bin/external/llvm-project/mlir/_virtual_includes/BuiltinOpsIncGen -Ibazel-out/k8-opt/bin/external/llvm-project/mlir/_virtual_includes/BuiltinTypeInterfacesIncGen -Ibazel-out/k8-opt/bin/external/llvm-project/mlir/_virtual_includes/BuiltinTypesIncGen -Ibazel-out/k8-opt/bin/external/llvm-project/mlir/_virtual_includes/CallOpInterfacesIncGen -Ibazel-out/k8-opt/bin/external/llvm-project/mlir/_virtual_includes/CastOpInterfacesIncGen -Ibazel-out/k8-opt/bin/external/llvm-project/mlir/_virtual_includes/FunctionInterfacesIncGen -Ibazel-out/k8-opt/bin/external/llvm-project/mlir/_virtual_includes/InferTypeOpInterfaceIncGen -Ibazel-out/k8-opt/bin/external/llvm-project/mlir/_virtual_includes/OpAsmInterfaceIncGen -Ibazel-out/k8-opt/bin/external/llvm-project/mlir/_virtual_includes/RegionKindInterfaceIncGen -Ibazel-out/k8-opt/bin/external/llvm-project/mlir/_virtual_includes/SideEffectInterfacesIncGen -Ibazel-out/k8-opt/bin/external/llvm-project/mlir/_virtual_includes/SubElementInterfacesIncGen -Ibazel-out/k8-opt/bin/external/llvm-project/mlir/_virtual_includes/SymbolInterfacesIncGen -Ibazel-out/k8-opt/bin/external/llvm-project/mlir/_virtual_includes/TensorEncodingIncGen -Ibazel-out/k8-opt/bin/external/llvm-project/mlir/_virtual_includes/AsmParserTokenKinds -Ibazel-out/k8-opt/bin/external/llvm-project/mlir/_virtual_includes/ArithmeticBaseIncGen -Ibazel-out/k8-opt/bin/external/llvm-project/mlir/_virtual_includes/ArithmeticCanonicalizationIncGen -Ibazel-out/k8-opt/bin/external/llvm-project/mlir/_virtual_includes/ArithmeticOpsIncGen -Ibazel-out/k8-opt/bin/external/llvm-project/mlir/_virtual_includes/InferIntRangeInterfaceIncGen -Ibazel-out/k8-opt/bin/external/llvm-project/mlir/_virtual_includes/VectorInterfacesIncGen -Ibazel-out/k8-opt/bin/external/llvm-project/mlir/_virtual_includes/ControlFlowInterfacesIncGen -Ibazel-out/k8-opt/bin/external/llvm-project/mlir/_virtual_includes/ControlFlowOpsIncGen -Ibazel-out/k8-opt/bin/external/llvm-project/mlir/_virtual_includes/FuncIncGen -Ibazel-out/k8-opt/bin/external/llvm-project/mlir/_virtual_includes/DerivedAttributeOpInterfaceIncGen -Ibazel-out/k8-opt/bin/external/llvm-project/mlir/_virtual_includes/LoopLikeInterfaceIncGen -isystem external/nsync/public -isystem bazel-out/k8-opt/bin/external/nsync/public -isystem third_party/eigen3/mkl_include -isystem bazel-out/k8-opt/bin/third_party/eigen3/mkl_include -isystem external/eigen_archive -isystem bazel-out/k8-opt/bin/external/eigen_archive -isystem external/gif -isystem bazel-out/k8-opt/bin/external/gif -isystem external/com_google_protobuf/src -isystem bazel-out/k8-opt/bin/external/com_google_protobuf/src -isystem external/zlib -isystem bazel-out/k8-opt/bin/external/zlib -isystem external/farmhash_archive/src -isystem bazel-out/k8-opt/bin/external/farmhash_archive/src -isystem external/llvm-project/mlir/include -isystem bazel-out/k8-opt/bin/external/llvm-project/mlir/include -isystem external/llvm-project/llvm/include -isystem bazel-out/k8-opt/bin/external/llvm-project/llvm/include -Wno-all -Wno-extra -Wno-deprecated -Wno-deprecated-declarations -Wno-ignored-attributes -Wunused-result '-Werror=unused-result' -DAUTOLOAD_DYNAMIC_KERNELS '-march=native' '-std=c++17' -fno-canonical-system-headers -Wno-builtin-macro-redefined '-D__DATE__="redacted"' '-D__TIMESTAMP__="redacted"' '-D__TIME__="redacted"' -c tensorflow/compiler/mlir/tensorflow/ir/tf_ops.cc -o bazel-out/k8-opt/bin/tensorflow/compiler/mlir/tensorflow/_objs/tensorflow_ops/tf_ops.pic.o)
# Configuration: ae1207a718faaafc59350d51115431e274e5ead6354542ebb9cdffb3eadc11c6
# Execution platform: @local_execution_config_platform//:platform
gcc: fatal error: Killed signal terminated program cc1plus
compilation terminated.
Target //tensorflow/tools/lib_package:libtensorflow failed to build
INFO: Elapsed time: 40991.269s, Critical Path: 2956.70s
INFO: 7924 processes: 618 internal, 7306 local.
FAILED: Build did NOT complete successfully
FAILED: Build did NOT complete successfully
cp: cannot stat 'bazel-bin/tensorflow/tools/lib_package/libtensorflow.tar.gz': No such file or directory

[bertmelis]

Maybe lower the amount of parallel jobs using bazel build --jobs 1. It's a trail and error path...

[dorianim]

@alextran1502 are you willing to officially provide this image to make it easier to use for everyone?

[dorianim]

@bertmelis thank you so much for this guide, it worked perfectly for me! However, I#d suggest changing the seconf Dockerfile to this:

FROM altran1502/immich-machine-learning:release

COPY libtensorflow.tar.gz node_modules/@tensorflow/tfjs-node/deps/libtensorflow.tar.gz

RUN apt-get update && \
    apt-get -y --no-install-recommends install libomp-dev && \
    apt-get clean && \
    cd node_modules/@tensorflow/tfjs-node/deps && \
    tar -xf libtensorflow.tar.gz && \
    rm libtensorflow.tar.gz

Like this, the libtensorflow.tar.gz is not contained in the final image :)

[tbilou]

I have a J3355

[bertmelis]

I can upload my version, no guaranties it will work though. I compiled it 'natively' so I don't know which options were enabled that might not work on your cpu.

[bertmelis]

EDIT: removed link to Google drive, use this instead: https://github.com/bertmelis/immich-machine-learning-no-avx

Based on tfjs 3.19.0, built for Intel J4105

[tbilou]

Thank you both.
I tried both tarballs and they both worked ;)

[C8opmBM]

bertmelis, thanks for sharing the image. It works on my j5005 :)

[bertmelis]

Just a link to the prebuild image. You can also find up-to-date files to do it yourself there:
https://github.com/bertmelis/immich-machine-learning-no-avx

[C8opmBM]

Amazing, udpdated to your repo, all working great. Thanks for this @bertmelis

[Snuupy]

I was impatient so I took bermelis' tfjs build and updated the docker image. You can find it here: https://hub.docker.com/r/snuupy/immich-machine-learning-noavx

[bertmelis]

Just a reminder: as an alternative for using ready-made images, you can also refer to a Containerfile/Dockerfile in your compose file. Of course, you'll still need the non-avx tfjs.

[Snuupy]

Thanks!

fyi I got this working locally by doing:

1. save Dockerfile to immich-machine-learning.Dockerfile
2. run wget https://github.com/bertmelis/immich-machine-learning-no-avx/raw/main/libtensorflow.tar.gz
3. change docker-compose.yml to:

  immich-machine-learning:
    # image: altran1502/immich-machine-learning:release
    # image: bertmelis1/immich-machine-learning-noavx:release
    image: "user/immich-machine-learning-noavx:release" 
    build:
      context: .
      dockerfile: immich-machine-learning.Dockerfile

4. then run:

docker-compose pull
docker-compose build immich-machine-learning
docker-compose up -d

[bertmelis]

Excellent.

I'm thinking about abandoning the no-avx image. Because it is indeed this easy to create your own image. All I will do is provide the non-avx tensorflow library. Of course, the code to built this yourself is public so even this you can do by yourself.

[Snuupy]

either is fine! maybe also group the instructions together (or I can do it) in the repo.

[bertmelis]

Ok, let's do both. Keeping up the image isn't much work (albeit manual).

You are very welcome to add the instructions!

[bigjam3]

Excellent.

I'm thinking about abandoning the no-avx image. Because it is indeed this easy to create your own image. All I will do is provide the non-avx tensorflow library. Of course, the code to built this yourself is public so even this you can do by yourself.

The image is much appreciated and makes life easier for many of us! But if you do decide you don't want to keep updating it, please let us know here so know to manually update ourselves. :)

[bt90]

The current release works without requiring AVX

[bertmelis]

Immich works, Typesense doesn't: #2055

Maybe the folks at Typesense made an updated image, I haven't checked.

[bt90]

Everything running smoothly here on my Intel Celeron G3930.

[alextran1502]

It seems like Typesense 0.24.1 breaks things on some environments. You guys can revert to 0.24.0

[756yang]

@alextran1502 After my inspection, the official archive compiled by typesense supports most x64 platforms. Based on the platform instruction set, it requires at least SSE2, SSSE3, SSE4.1 support, and may require SSE3 and SSE4.2 support.

The AVX instruction set is additional and not necessarily required.

[tebra-jl]

Hi all,
New user of Immich here.
I installed Immich in docker on linux arm64 SOC machine (Odroid HC4). I disable machine learning on docker-compose.yml because it is a small CPU without AVX.
Now I would like to run machine learning docker on my iMac with M1 silicon CPU.
Is it possible? If yes, could you point me to instructions?
Thanks

[Snuupy]

https://github.com/typesense/typesense#build-from-source just follow build guide but use v0.24

[tebra-jl]

Thanks for your help.
typesense-server@0.24.1 is now up and running on my iMac.
How to modify immich docker-compose.yml on my server Odroid to use it?

[Snuupy]

not sure if you still need help, but tag the docker image and then change docker-compose.yml to reference your newly compiled image.

[756yang]

After my inspection, the official archive compiled by typesense supports most x64 platforms. Based on the platform instruction set, it requires at least SSE2, SSSE3, SSE4.1 support, and may require SSE3 and SSE4.2 support.

The AVX instruction set is additional and not necessarily required.