Apple M1/M2 Silicon now works with the arm plugin! :) <3 #13308

raymondlo84 · 2022-10-01T03:54:43Z

raymondlo84
Oct 1, 2022

Followed these instructions and finally got my wheel built with ARM!!

https://github.com/openvinotoolkit/openvino/wiki/BuildingForMacOS_arm64

You're all amazing =)

[100%] Built target openvino_dev_wheel
raymondlo@Raymonds-MacBook-Pro build % cd wheels 
raymondlo@Raymonds-MacBook-Pro wheels % ls -lastr
total 39728
    0 drwxr-xr-x  56 raymondlo  staff      1792 Sep 30 20:48 ..
39696 -rw-r--r--   1 raymondlo  staff  19607612 Sep 30 20:48 openvino-2022.3.0-000-cp310-cp310-macosx_12_0_arm64.whl
    0 drwxr-xr-x   4 raymondlo  staff       128 Sep 30 20:48 .
   32 -rw-r--r--   1 raymondlo  staff     13345 Sep 30 20:48 openvino_dev-2022.3.0-000-py3-none-any.whl

(openvino_arm) raymondlo@Raymonds-MacBook-Pro wheels % ../../bin/arm64/Release/hello_query_device 
[ INFO ] OpenVINO Runtime version ......... 2022.3.0
[ INFO ] Build ........... 2022.3.0-000-5fef222bce1
[ INFO ] 
[ INFO ] Available devices: 
[ INFO ] CPU
[ INFO ] 	SUPPORTED_PROPERTIES: 
[ INFO ] 		Immutable: SUPPORTED_METRICS : SUPPORTED_METRICS SUPPORTED_CONFIG_KEYS RANGE_FOR_ASYNC_INFER_REQUESTS RANGE_FOR_STREAMS
[ INFO ] 		Immutable: SUPPORTED_CONFIG_KEYS : LP_TRANSFORMS_MODE DUMP_GRAPH PERF_COUNT CPU_THROUGHPUT_STREAMS CPU_BIND_THREAD CPU_THREADS_NUM CPU_THREADS_PER_STREAM NUM_STREAMS INFERENCE_NUM_THREADS AFFINITY
[ INFO ] 		Mutable: PERF_COUNT : YES
[ INFO ] 		Immutable: AVAILABLE_DEVICES : NEON
[ INFO ] 		Immutable: FULL_DEVICE_NAME : arm_compute::NEON
[ INFO ] 		Immutable: OPTIMIZATION_CAPABILITIES : FP16 FP32
[ INFO ] 		Immutable: RANGE_FOR_ASYNC_INFER_REQUESTS : 1 8 1
[ INFO ] 		Mutable: PERFORMANCE_HINT : ""
[ INFO ] 		Immutable: RANGE_FOR_STREAMS : 1 8
[ INFO ] 		Mutable: CPU_THROUGHPUT_STREAMS : 1
[ INFO ] 		Mutable: CPU_BIND_THREAD : NO
[ INFO ] 		Mutable: CPU_THREADS_NUM : 0
[ INFO ] 		Mutable: CPU_THREADS_PER_STREAM : 8
[ INFO ] 		Mutable: NUM_STREAMS : 1
[ INFO ] 		Mutable: INFERENCE_NUM_THREADS : 0
[ INFO ] 		Mutable: AFFINITY : NONE
[ INFO ] 
(openvino_arm) raymondlo@Raymonds-MacBook-Pro wheels %

Yesssssssssss!!!

raymondlo84 · 2022-10-01T04:18:32Z

raymondlo84
Oct 1, 2022
Author

:| how do I compile with the omz_downloader? It doesn't seem to come with the arm's wheel

0 replies

raymondlo84 · 2022-10-01T04:25:28Z

raymondlo84
Oct 1, 2022
Author

(openvino_arm) raymondlo@Raymonds-MacBook-Pro openvino_notebooks % ../../../bin/arm64/Release/benchmark_app -m notebooks/001-hello-world/model/v3-small_224_1.0_float.xml  
[Step 1/11] Parsing and validating input arguments
[ INFO ] Parsing input parameters
[Step 2/11] Loading OpenVINO Runtime
[ INFO ] OpenVINO: OpenVINO Runtime version ......... 2022.3.0
[ INFO ] Build ........... 2022.3.0-000-5fef222bce1
[ INFO ] 
[ INFO ] Device info: 
[ INFO ] CPU
[ INFO ] openvino_arm_cpu_plugin version ......... 2022.3.0
[ INFO ] Build ........... 2022.3.0-000-5fef222bce1
[ INFO ] 
[ INFO ] 
[Step 3/11] Setting device configuration
[ WARNING ] Performance hint was not explicitly specified in command line. Device(CPU) performance hint will be set to THROUGHPUT.
[Step 4/11] Reading network files
[ INFO ] Loading network files
[ INFO ] Read network took 237.16 ms
[ INFO ] Original network I/O parameters:
Network inputs:
    input:0 (node: input) : f32 / [...] / {1,224,224,3}
Network outputs:
    MobilenetV3/Predictions/Softmax:0 (node: MobilenetV3/Predictions/Softmax) : f32 / [...] / {1,1001}
[Step 5/11] Resizing network to match image sizes and given batch
[ WARNING ] input:0: layout is not set explicitly, so it is defaulted to NHWC. It is STRONGLY recommended to set layout manually to avoid further issues.
[Step 6/11] Configuring input of the model
[ INFO ] Network batch size: 1
Network inputs:
    input:0 (node: input) : u8 / [N,H,W,C] / {1,224,224,3}
Network outputs:
    MobilenetV3/Predictions/Softmax:0 (node: MobilenetV3/Predictions/Softmax) : f32 / [...] / {1,1001}
[Step 7/11] Loading the model to the device
[ INFO ] Load network took 40.40 ms
[Step 8/11] Setting optimal runtime parameters
[ INFO ] Device: CPU
[ INFO ]   { NETWORK_NAME , v3-small_224_1.0_float }
[ INFO ]   { OPTIMAL_NUMBER_OF_INFER_REQUESTS , 8 }
[ INFO ]   { NUM_STREAMS , 8 }
[ INFO ]   { INFERENCE_NUM_THREADS , 0 }
[Step 9/11] Creating infer requests and preparing input blobs with data
[ WARNING ] No input files were given: all inputs will be filled with random values!
[ INFO ] Test Config 0
[ INFO ] input:0  ([N,H,W,C], u8, {1, 224, 224, 3}, static):	random (image is expected)
[Step 10/11] Measuring performance (Start inference asynchronously, 8 inference requests, limits: 60000 ms duration)
[ INFO ] BENCHMARK IS IN INFERENCE ONLY MODE.
[ INFO ] Input blobs will be filled once before performance measurements.
[ INFO ] First inference took 31.69 ms

[Step 11/11] Dumping statistics report
[ INFO ] Count:      24216 iterations
[ INFO ] Duration:   60027.76 ms
[ INFO ] Latency: 
[ INFO ] 	Median:     19.56 ms
[ INFO ] 	Average:    19.75 ms
[ INFO ] 	Min:        13.20 ms
[ INFO ] 	Max:        57.99 ms
[ INFO ] Throughput: 403.41 FPS

<3 <3 <3 ;)

0 replies

ilya-lavrenov · 2022-10-03T12:19:07Z

ilya-lavrenov
Oct 3, 2022
Maintainer

@raymondlo84 I suppose openvino_dev-2022.3.0-000-py3-none-any.whl should contain omz_downloader, have you installed it?

7 replies

raymondlo84 Oct 19, 2022
Author

I'm backkkk ;) Let me try again this week ;)

raymondlo84 Oct 21, 2022
Author

@raymondlo84 I suppose openvino_dev-2022.3.0-000-py3-none-any.whl should contain omz_downloader, have you installed it?

raymondlo@Raymonds-MacBook-Pro wheels % ls -lastrh
total 39920
39888 -rw-r--r-- 1 raymondlo staff 19M Oct 20 21:39 openvino-2022.3.0-000-cp310-cp310-macosx_12_0_arm64.whl
32 -rw-r--r-- 1 raymondlo staff 13K Oct 20 21:39 openvino_dev-2022.3.0-000-py3-none-any.whl
0 drwxr-xr-x 58 raymondlo staff 1.8K Oct 20 21:44 ..
0 drwxr-xr-x 5 raymondlo staff 160B Oct 20 21:44 .
0 drwxr-xr-x 7 raymondlo staff 224B Oct 20 21:44 openvino-2022.3.0

... the openvino_dev is empty (13k...)

Attached the log a bit here.

log_cmake1.txt
log_cmake2.txt

ilya-lavrenov Oct 21, 2022
Maintainer

the openvino_dev is empty (13k...)

we optimized the binary size 😉

raymondlo84 Oct 28, 2022
Author

the openvino_dev is empty (13k...)

we optimized the binary size 😉

LOL, but 0 bytes LOL? :) I will try again from scratch...

raymondlo84 Oct 28, 2022
Author

Do you have the wheel file for M1? I wanna unpack and take a look...

raymondlo84 · 2022-10-19T19:25:52Z

raymondlo84
Oct 19, 2022
Author

% # select preferred version of python based on available ones, e.g. 3.10
% brew install python@3.10

Do we support 3.10?

4 replies

mlukasze Oct 20, 2022
Collaborator

Do we support 3.10?

not yet, but wip

ilya-lavrenov Oct 20, 2022
Maintainer

only if you build from sources

raymondlo84 Oct 28, 2022
Author

Is 3.9 recommended or 3.10?

ilya-lavrenov Dec 2, 2022
Maintainer

We support all versions in range 3.7-3.10

raymondlo84 · 2022-10-28T22:30:58Z

raymondlo84
Oct 28, 2022
Author

Um... where is the model optimizer? I tried it again -- maybe I should use python 3.9?

compile_tool openvino-2022.3.0-000-cp310-cp310-macosx_13_0_arm64.whl
deployment_manager openvino_dev-2022.3.0-000-py3-none-any.whl
raymondlo@Raymonds-MacBook-Pro-2 tools % cd compile_tool
raymondlo@Raymonds-MacBook-Pro-2 compile_tool % ls
README.md compile_tool
raymondlo@Raymonds-MacBook-Pro-2 compile_tool % cd ..
raymondlo@Raymonds-MacBook-Pro-2 tools % ls
compile_tool openvino-2022.3.0-000-cp310-cp310-macosx_13_0_arm64.whl
deployment_manager openvino_dev-2022.3.0-000-py3-none-any.whl
raymondlo@Raymonds-MacBook-Pro-2 tools % cd ..
raymondlo@Raymonds-MacBook-Pro-2 openvino_m1 % ls
docs python runtime samples setupvars.sh tools
raymondlo@Raymonds-MacBook-Pro-2 openvino_m1 % cd python
raymondlo@Raymonds-MacBook-Pro-2 python % ls
python3.10
raymondlo@Raymonds-MacBook-Pro-2 python %

0 replies

raymondlo84 · 2022-11-27T03:45:30Z

raymondlo84
Nov 27, 2022
Author

Not sure what changed but magically it works now!

I'm using macOS 13.0.1 with Python 3.10 from Brew.

0 replies

raymondlo84 · 2022-11-27T03:45:34Z

raymondlo84
Nov 27, 2022
Author

(openvino_build) rlo@Raymonds-MacBook-Pro-3 wheels % ls -lastrh
total 41048
0 drwxr-xr-x 57 rlo staff 1.8K Nov 27 11:43 ..
29656 -rw-r--r-- 1 rlo staff 14M Nov 27 11:43 openvino-2022.3.0-8846-cp310-cp310-macosx_13_0_arm64.whl
0 drwxr-xr-x 4 rlo staff 128B Nov 27 11:43 .
11392 -rw-r--r-- 1 rlo staff 5.6M Nov 27 11:43 openvino_dev-2022.3.0-8846-py3-none-any.whl
(openvino_build) rlo@Raymonds-MacBook-Pro-3 wheels % pip install openvino-2022.3.0-8846-cp310-cp310-macosx_13_0_arm64.whl
Processing ./openvino-2022.3.0-8846-cp310-cp310-macosx_13

0 replies

ilya-lavrenov · 2023-01-04T14:09:40Z

ilya-lavrenov
Jan 4, 2023
Maintainer

OpenVINO 2022.3.0 is finally released on pypi for macOS Apple Silicon

0 replies

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Apple M1/M2 Silicon now works with the arm plugin! :) <3 #13308

{{title}}

Replies: 8 comments 11 replies

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

Select a reply

Apple M1/M2 Silicon now works with the arm plugin! :) <3 #13308

raymondlo84 Oct 1, 2022

Replies: 8 comments · 11 replies

raymondlo84 Oct 1, 2022 Author

raymondlo84 Oct 1, 2022 Author

ilya-lavrenov Oct 3, 2022 Maintainer

raymondlo84 Oct 19, 2022 Author

raymondlo84 Oct 21, 2022 Author

ilya-lavrenov Oct 21, 2022 Maintainer

raymondlo84 Oct 28, 2022 Author

raymondlo84 Oct 28, 2022 Author

raymondlo84 Oct 19, 2022 Author

mlukasze Oct 20, 2022 Collaborator

ilya-lavrenov Oct 20, 2022 Maintainer

raymondlo84 Oct 28, 2022 Author

ilya-lavrenov Dec 2, 2022 Maintainer

raymondlo84 Oct 28, 2022 Author

raymondlo84 Nov 27, 2022 Author

raymondlo84 Nov 27, 2022 Author

ilya-lavrenov Jan 4, 2023 Maintainer

raymondlo84
Oct 1, 2022

Replies: 8 comments 11 replies

raymondlo84
Oct 1, 2022
Author

raymondlo84
Oct 1, 2022
Author

ilya-lavrenov
Oct 3, 2022
Maintainer

raymondlo84 Oct 19, 2022
Author

raymondlo84 Oct 21, 2022
Author

ilya-lavrenov Oct 21, 2022
Maintainer

raymondlo84 Oct 28, 2022
Author

raymondlo84 Oct 28, 2022
Author

raymondlo84
Oct 19, 2022
Author

mlukasze Oct 20, 2022
Collaborator

ilya-lavrenov Oct 20, 2022
Maintainer

raymondlo84 Oct 28, 2022
Author

ilya-lavrenov Dec 2, 2022
Maintainer

raymondlo84
Oct 28, 2022
Author

raymondlo84
Nov 27, 2022
Author

raymondlo84
Nov 27, 2022
Author

ilya-lavrenov
Jan 4, 2023
Maintainer