[K/N] Expose program name in runtime #5281

vonox7 · 2024-03-21T15:39:55Z

*^[KTI-54606](https://youtrack.jetbrains.com/issue/KT-54606) Fixed*

SvyatoslavScherbina · 2024-04-26T14:53:10Z

kotlin-native/backend.native/tests/build.gradle

@@ -380,6 +380,10 @@ standaloneTest("throw_from_except_constr") {
    source = "runtime/exceptions/throw_from_except_constr.kt"
 }

+standaloneTest("program_name") {
+    source = "runtime/program_name/runtime_program_name.kt"
+}


This file represents the legacy test infrastructure that we are actively trying to get rid of.

Please create a proper test class in
https://github.com/JetBrains/kotlin/tree/master/native/native.tests/tests/org/jetbrains/kotlin/konan/test/blackbox instead.

I ported the testcase to the new infrastructure.

SvyatoslavScherbina · 2024-04-26T14:58:47Z

kotlin-native/runtime/src/launcher/cpp/launcher.cpp

@@ -30,6 +30,8 @@ using namespace kotlin;
 //--- Setup args --------------------------------------------------------------//

 OBJ_GETTER(setupArgs, int argc, const char** argv) {
+  kotlin::programName = argv[0];


Is argv[0] guaranteed to live long enough? What will happen if some code accesses kotlin::programName after the main function finishes?
Is argv guaranteed to always have at least one element?

programName will now be set to null before argv leaves scope. I could not find a definite answer if it would be guaranteed to live long enough, so I opted for the safe choice.

Theoretically it is guaranteed by the POSIX standard. However if it has 0 arguments, the code 1 line below would have crashed. I also added a testcase for argc=0, and added support for it via std::max(0, ...). So now kotlin executables don't crash on startup when they are launched in a non posix compatible way.

kotlin-native/runtime/src/main/kotlin/kotlin/native/Platform.kt

SvyatoslavScherbina · 2024-04-26T15:02:50Z

kotlin-native/backend.native/tests/runtime/program_name/runtime_program_name.kt

+    val programFileName = Platform.programName.substringAfterLast("/").substringBeforeLast(".")
+
+    assertEquals("program_name", programFileName)
+}


More tests are necessary. For example, the original issue mentions a nice use case:

Build multi-call binaries. For example, busybox is a single binary that contains many tools, and decides which tool to run based on how the symbolic link is called: https://busybox.net/downloads/BusyBox.html#usage (that's how it can be so small)

So checking a case with symbolic link would also be useful.

I added test for that by calling execv() in the C code. This allowed to test even more use-cases (no programName at all), while still covering the use-case of renaming/linking a binary.

# Conflicts: # kotlin-native/backend.native/tests/build.gradle

SvyatoslavScherbina · 2024-05-13T10:22:24Z

native/native.tests/build.gradle.kts

@@ -51,6 +51,7 @@ val stdlibK2Test = nativeTest("stdlibK2Test", "stdlib & frontend-fir")
 val kotlinTestLibraryTest = nativeTest("kotlinTestLibraryTest", "kotlin-test & !frontend-fir")
 val kotlinTestLibraryK2Test = nativeTest("kotlinTestLibraryK2Test", "kotlin-test & frontend-fir")
 val partialLinkageTest = nativeTest("partialLinkageTest", "partial-linkage")
+val programNameTest = nativeTest("programNameTest", "program-name")


This seems a bit redudant. I mean, running these tests separately shouldn't happen frequently, so adding a Gradle task is overkill.

Ok, I've removed it.

SvyatoslavScherbina · 2024-05-13T12:38:02Z

native/native.tests/testData/programName/kotlinPrintEntryPoint.kt

+
+fun main(args: Array<String>) {
+    // Remove path and extension (.kexe or .exe)
+    val programFileName = Platform.programName?.substringAfterLast("/")?.substringBeforeLast(".")


This test fails on Windows, because it uses \ instead of /.

Good catch, thank you. The sanitization is now platform agnostic.

SvyatoslavScherbina · 2024-05-13T13:07:19Z

native/native.tests/tests/org/jetbrains/kotlin/konan/test/blackbox/ProgramNameTest.kt

+            val result = runProcess(cExecutable.absolutePath, binaryName, *args) {
+                timeout = 60.seconds
+            }
+            assertEquals("calling exec...\n$expected", result.stdout)


I suspect this code should be fixed to handle \r\n on Windows.

Good catch, thank you again! The assertion is now platform agnostic due to additional sanitization.

SvyatoslavScherbina · 2024-05-13T13:07:42Z

native/native.tests/tests/org/jetbrains/kotlin/konan/test/blackbox/ProgramNameTest.kt

+
+        fun validate(expected: String, vararg args: String) {
+            val binaryName = kotlinCompilation.resultingArtifact.executableFile.path
+            val result = runProcess(cExecutable.absolutePath, binaryName, *args) {


This fails if the test target is different from the host. You can reproduce this by running this test with -Pkotlin.internal.native.test.target=<target> Gradle property.

Please use a proper executor through

kotlin/native/native.tests/tests/org/jetbrains/kotlin/konan/test/blackbox/support/settings/SettingsExecutor.kt

Line 18 in 8c69667

val Settings.executor: Executor

Thank you for this insight. I changed it accordingly.

The test fails on linuxArm64 target with:

java.lang.AssertionError: Expected <calling exec... programName: app args:>, actual <calling exec... exec failed>. at kotlin.test.DefaultAsserter.fail(DefaultAsserter.kt:16) at kotlin.test.Asserter$DefaultImpls.assertTrue(Assertions.kt:694) at kotlin.test.DefaultAsserter.assertTrue(DefaultAsserter.kt:11) at kotlin.test.Asserter$DefaultImpls.assertEquals(Assertions.kt:713) at kotlin.test.DefaultAsserter.assertEquals(DefaultAsserter.kt:11) at kotlin.test.AssertionsKt__AssertionsKt.assertEquals(Assertions.kt:63) at kotlin.test.AssertionsKt.assertEquals(Unknown Source) at kotlin.test.AssertionsKt__AssertionsKt.assertEquals$default(Assertions.kt:62) at kotlin.test.AssertionsKt.assertEquals$default(Unknown Source) at org.jetbrains.kotlin.konan.test.blackbox.ProgramNameTest.programNameTest$validate(ProgramNameTest.kt:49) at org.jetbrains.kotlin.konan.test.blackbox.ProgramNameTest.programNameTest(ProgramNameTest.kt:54) ...

To reproduce, you can run the test with -Pkotlin.internal.native.test.target=linuxArm64 Gradle property on a Linux/x86_64 machine. I don't have access to such a machine at the moment. Could you please check the test, if possible?

Our test infrastructure runs tests on linuxArm64 through qemu, maybe the problem is somehow related to that.

SvyatoslavScherbina · 2024-05-13T13:08:48Z

native/native.tests/tests/org/jetbrains/kotlin/konan/test/blackbox/ProgramNameTest.kt

+        // No program name - this would not be POSIX compliant, see https://pubs.opengroup.org/onlinepubs/9699919799/functions/exec.html:
+        // "[...] requires a Strictly Conforming POSIX Application to pass at least one argument to the exec function"
+        // However, we should not crash the Kotlin runtime because of this.
+        validate("programName: null\nargs:")


On Linux, this fails for me with

java.lang.AssertionError: Expected <calling exec... programName: null args:>, actual <calling exec... programName: args:>.

Please take a look.

This is very interesting. Linux behaves here differently as windows & macOS. I fixed the problem, and added more explanation in the code. I used the following C standard PDF: https://www.open-std.org/jtc1/sc22/wg14/www/docs/n1256.pdf.

Please tell me if you would want to let windows+macOS behave like linux. The way I wrote it now feels more natural to me, however I fully understand that all 3 options have its benefits and drawbacks:

A) The way I wrote it now: all 3 platforms behave the same; no program name leads to programName=null, and empty programName leads to programName="".
B) all 3 platforms behave the same; however both no program name and empty program Name lead to programName="". (the linux way of thinking)
C) Fully native behaviour: no programName will be programName="" on linux and programName=null on macOS/windows; and empty programName will be everywhere programName="".

…ek/program-name

SvyatoslavScherbina · 2024-05-28T15:03:24Z

kotlin-native/runtime/src/main/cpp/Runtime.h

@@ -8,6 +8,7 @@

 #include "Porting.h"
 #include "Memory.h"
+#include "KString.h"


Seems unused.

SvyatoslavScherbina · 2024-05-28T15:12:11Z

native/native.tests/testData/programName/main.c

+
+    // Kotlin executable name is in argv[1]
+    // Forward argv[2..n] to kotlin executable as arguments (the first one should be the programName according to posix)
+    execv(argv[1], &(argv[2]));


Btw, execv requires that the list of arguments must be terminated by a NULL pointer. Is it always guaranteed here?

SvyatoslavScherbina · 2024-05-28T16:35:42Z

native/native.tests/tests/org/jetbrains/kotlin/konan/test/blackbox/ProgramNameTest.kt

+
+        fun validate(expected: String, vararg args: String) {
+            val binaryName = kotlinCompilation.resultingArtifact.executableFile.path
+            val result = runProcess(cExecutable.absolutePath, binaryName, *args) {


The test fails on linuxArm64 target with:

java.lang.AssertionError: Expected <calling exec... programName: app args:>, actual <calling exec... exec failed>. at kotlin.test.DefaultAsserter.fail(DefaultAsserter.kt:16) at kotlin.test.Asserter$DefaultImpls.assertTrue(Assertions.kt:694) at kotlin.test.DefaultAsserter.assertTrue(DefaultAsserter.kt:11) at kotlin.test.Asserter$DefaultImpls.assertEquals(Assertions.kt:713) at kotlin.test.DefaultAsserter.assertEquals(DefaultAsserter.kt:11) at kotlin.test.AssertionsKt__AssertionsKt.assertEquals(Assertions.kt:63) at kotlin.test.AssertionsKt.assertEquals(Unknown Source) at kotlin.test.AssertionsKt__AssertionsKt.assertEquals$default(Assertions.kt:62) at kotlin.test.AssertionsKt.assertEquals$default(Unknown Source) at org.jetbrains.kotlin.konan.test.blackbox.ProgramNameTest.programNameTest$validate(ProgramNameTest.kt:49) at org.jetbrains.kotlin.konan.test.blackbox.ProgramNameTest.programNameTest(ProgramNameTest.kt:54) ...

To reproduce, you can run the test with -Pkotlin.internal.native.test.target=linuxArm64 Gradle property on a Linux/x86_64 machine. I don't have access to such a machine at the moment. Could you please check the test, if possible?

Our test infrastructure runs tests on linuxArm64 through qemu, maybe the problem is somehow related to that.

SvyatoslavScherbina · 2024-05-28T16:38:49Z

kotlin-native/runtime/src/main/kotlin/kotlin/native/Platform.kt

+     * [null] if the Kotlin code was compiled to a native library and the executable is not a Kotlin program.
+     */
+    public val programName: String?
+        get() = Platform_getProgramName()


@qurbonzoda could you please review the stdlib change?

[K/N] Expose program name in runtime

1ff362a

*^[KTI-54606](https://youtrack.jetbrains.com/issue/KT-54606) Fixed*

ivakub added the Native label Apr 8, 2024

SvyatoslavScherbina requested changes Apr 26, 2024

View reviewed changes

vonox7 added 5 commits April 29, 2024 14:57

Merge branch 'refs/heads/master' into slawicek/program-name

05d57ee

# Conflicts: # kotlin-native/backend.native/tests/build.gradle

Move programName testcase to new test infrastructure

f73045e

Add testcase for calling Platform.getProgramName from within a library

fb67427

Execute Kotlin executable via execv to pass custom programNames

2703f15

Support argc=0

3b66b6b

vonox7 requested a review from SvyatoslavScherbina April 30, 2024 14:05

Merge branch 'JetBrains:master' into slawicek/program-name

285dd67

SvyatoslavScherbina requested changes May 13, 2024

View reviewed changes

vonox7 added 6 commits May 15, 2024 10:45

Use proper executor

69e660f

Merge remote-tracking branch 'vali/slawicek/program-name' into slawic…

a69b027

…ek/program-name

Remove unneeded gradle task

6d07cf2

Ignore different windows path & line-endings

2fcf1f0

Let linux behave the same as windows/macos

3c933a7

Add empty program name validation

802c8e7

vonox7 requested a review from SvyatoslavScherbina May 26, 2024 19:03

Fix testcase on windows

64ffcfc

SvyatoslavScherbina reviewed May 28, 2024

View reviewed changes

SvyatoslavScherbina requested a review from projedi May 28, 2024 16:39

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[K/N] Expose program name in runtime #5281

[K/N] Expose program name in runtime #5281

vonox7 commented Mar 21, 2024

SvyatoslavScherbina Apr 26, 2024

vonox7 Apr 30, 2024

SvyatoslavScherbina Apr 26, 2024

vonox7 Apr 30, 2024 •

edited

SvyatoslavScherbina Apr 26, 2024

vonox7 Apr 30, 2024

SvyatoslavScherbina May 13, 2024

vonox7 May 26, 2024

SvyatoslavScherbina May 13, 2024

vonox7 May 26, 2024

SvyatoslavScherbina May 13, 2024

vonox7 May 26, 2024

SvyatoslavScherbina May 13, 2024

vonox7 May 26, 2024

SvyatoslavScherbina May 28, 2024

SvyatoslavScherbina May 13, 2024

vonox7 May 26, 2024

SvyatoslavScherbina May 28, 2024

SvyatoslavScherbina May 28, 2024

SvyatoslavScherbina May 28, 2024

SvyatoslavScherbina May 28, 2024

[K/N] Expose program name in runtime #5281

Are you sure you want to change the base?

[K/N] Expose program name in runtime #5281

Conversation

vonox7 commented Mar 21, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

vonox7 Apr 30, 2024 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

vonox7 Apr 30, 2024 •

edited