build: Cylon Summit recipe is failing

[AymenFJA]

Hello all,

We were trying to deploy Cylon on Summit following this recipe: https://github.com/cylondata/cylon/tree/main/summit (README). We tried two modes of installation on Summit:

1. Following the recipe mentioned above and using compute nodes and the Cylon did not finish as it was failing with dependency installation.
2. We switched to the login node, and we were able to install Cylon, and it was faster. We exported the necessary arrow and build paths. Yet we keep getting this error:
   (We ran a test from a compute node here using jsrun as recommended)
   

and this

Any help or idea is appreciated it.

[nirandaperera]

Hi Aymen, This usually happens when the shared files (.so) are not found by pycylon. Can you try adding the cylon lib directory to the LD_LIBRARY_PATH?

…

On Fri, May 12, 2023, 09:29 Aymen Alsaadi ***@***.***> wrote: Hello all, We were trying to deploy Cylon on Summit following this recipe: https://github.com/cylondata/cylon/tree/main/summit (README). We tried two modes of installation on Summit: 1. Following the recipe mentioned above and using compute nodes and the Cylon did not finish as it was failing with dependency installation. 2. We switched to the login node, and we were able to install Cylon and it was faster. We exported the necessary arrow and build paths. Yet we keep getting this error: [image: image] <https://user-images.githubusercontent.com/27039262/237984098-77d0a21c-a973-4f14-816b-d57614f2d113.png> and this [image: image] <https://user-images.githubusercontent.com/27039262/237984182-9e651261-2f70-428a-93cc-d57727ded01d.png> Any help or idea is appreciated it. — Reply to this email directly, view it on GitHub <#659>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABKS65MEQ3YZEZGRNUUGL33XFY3J5ANCNFSM6AAAAAAX7RE3B4> . You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

[okilic1]

Hi @nirandaperera.

Which library path are you talking I tried with CYLON/lib, but I don't think it is working

[mstaylor]

I have run across a similar issue on Amazon Linux and it is related to cmake not finding openMPI. Niranda may have another suggestion but perhaps try installing from a OS repo such as yum or apt or build openmpi from source. I'm doing the same in parallel :-)

[nirandaperera]

Hi @okilic1,
I think we are talking about two separate issues here. @AymenFJA's initial issue was related to running cylon after compilation. There, you need to add <cylon install dir>/lib path to the LD_LIBRARY_PATH.

I believe you are referring to building cylon. Do you have OpenMPI installed in your system? If not, you can either install it locally, or install it using conda. It should resolve the compilation problem.

[laszewsk]

Can documentation for this be improved. Do code changes need to be done?

[arupcsedu]

Is this issue producing again? It should be fixed after executing
export LD_LIBRARY_PATH=$BUILD_PATH/arrow/install/lib64:$BUILD_PATH/glog/install/lib64:$BUILD_PATH/lib64:$BUILD_PATH/lib:$LD_LIBRARY_PATH

This is already in the documentation

[AymenFJA]

@arupcsedu we tried this approach before opening the ticket, and it did not work. Did you test from your side recently with this approach on Summit? If you did, then happy to give it a try again. Maybe something has changed.

[arupcsedu]

@AymenFJA , if you have access to the summit, let's have a joint session to make cylon work. The steps by steps guidelines mentioned in the https://github.com/cylondata/cylon/tree/main/summit, was working before in summit.

[AymenFJA]

@arupcsedu Unfortunately I have no access.

[laszewsk]

@arupcsedu we all wait for approval.