How to properly shadow Log4j Core in Gradle?

[nikita51bot]

Hi everyone. I have created a simple java application that uses Log4j and JLine. I have created a 'CustomAppender' appender plugin to make the console work properly with JLine input. If I use annotationProccesor in build.gradle, when I create a jar file (gradle shadowJar), it displays the following error on startup:

2024-03-26T18:25:20.155339Z main ERROR Unrecognized format specifier [d]
2024-03-26T18:25:20.158472400Z main ERROR Unrecognized conversion specifier [d] starting at position 16 in conversion pattern.
2024-03-26T18:25:20.159513200Z main ERROR Unrecognized format specifier [thread]
2024-03-26T18:25:20.159513200Z main ERROR Unrecognized conversion specifier [thread] starting at position 25 in conversion pattern.
2024-03-26T18:25:20.159513200Z main ERROR Unrecognized format specifier [level]
2024-03-26T18:25:20.160044800Z main ERROR Unrecognized conversion specifier [level] starting at position 35 in conversion pattern.
2024-03-26T18:25:20.160044800Z main ERROR Unrecognized format specifier [logger]
2024-03-26T18:25:20.160044800Z main ERROR Unrecognized conversion specifier [logger] starting at position 47 in conversion pattern.
2024-03-26T18:25:20.160559700Z main ERROR Unrecognized format specifier [msg]
2024-03-26T18:25:20.160559700Z main ERROR Unrecognized conversion specifier [msg] starting at position 54 in conversion pattern.
2024-03-26T18:25:20.160559700Z main ERROR Unrecognized format specifier [n]
2024-03-26T18:25:20.160559700Z main ERROR Unrecognized conversion specifier [n] starting at position 56 in conversion pattern.
2024-03-26T18:25:20.170962200Z main ERROR Unrecognized format specifier [d]
2024-03-26T18:25:20.171476300Z main ERROR Unrecognized conversion specifier [d] starting at position 16 in conversion pattern.
2024-03-26T18:25:20.171476300Z main ERROR Unrecognized format specifier [thread]
2024-03-26T18:25:20.171476300Z main ERROR Unrecognized conversion specifier [thread] starting at position 25 in conversion pattern.
2024-03-26T18:25:20.171476300Z main ERROR Unrecognized format specifier [level]
2024-03-26T18:25:20.171992300Z main ERROR Unrecognized conversion specifier [level] starting at position 35 in conversion pattern.
2024-03-26T18:25:20.171992300Z main ERROR Unrecognized format specifier [logger]
2024-03-26T18:25:20.171992300Z main ERROR Unrecognized conversion specifier [logger] starting at position 47 in conversion pattern.
2024-03-26T18:25:20.171992300Z main ERROR Unrecognized format specifier [msg]
2024-03-26T18:25:20.172505200Z main ERROR Unrecognized conversion specifier [msg] starting at position 54 in conversion pattern.
2024-03-26T18:25:20.172505200Z main ERROR Unrecognized format specifier [n]
2024-03-26T18:25:20.172505200Z main ERROR Unrecognized conversion specifier [n] starting at position 56 in conversion pattern.

If I remove annotationProccesor and in log4j2.xml in the configuration tag enter packages="org.example", the program starts working correctly, but displays the following text:

2024-03-26T18:38:02.165679600Z main WARN The use of package scanning to locate plugins is deprecated and will be removed in a future release
2024-03-26T18:38:02.172365200Z main WARN The use of package scanning to locate plugins is deprecated and will be removed in a future release
2024-03-26T18:38:02.173900Z main WARN The use of package scanning to locate plugins is deprecated and will be removed in a future release
2024-03-26T18:38:02.192486400Z main WARN The use of package scanning to locate plugins is deprecated and will be removed in a future release

Please help me, what am I doing wrong?

[tayllana]

[nikita51bot]

Thank you for your reply!

1. Unfortunately even with your pattern:

<?xml version="1.0" encoding="UTF-8"?>
<Configuration status="warn">
    <Appenders>
        <CustomAppender name="CustomAppender" class="org.example.CustomAppender">
            <PatternLayout>
                <Pattern>
                    %d [%t] %-5level %logger{36} - %msg%n
                </Pattern>
            </PatternLayout>
        </CustomAppender>
    </Appenders>
    <Loggers>
        <Root level="trace">
            <AppenderRef ref="CustomAppender" level="INFO"/>
        </Root>
    </Loggers>
</Configuration>

it doesn't work either, the same error is displayed
2. When I write the 'class' attribute in the 'CustomAppender' tag and specify the path to my class, the error is displayed:

2024-03-26T19:42:12.139722500Z main ERROR Error processing element CustomAppender ([Appenders: null]): CLASS_NOT_FOUND
2024-03-26T19:42:12.168130300Z main ERROR Unable to locate appender "CustomAppender" for logger config "root"

Unfortunately, this did not solve my problem

[ppkarwasz]

@tayllana,

We don't allow AI-generated answers in this repo. If you actually wrote the answer yourself can you explain why:

1. In your "Unrecognized format specifier error" section you give an example with a %d specifier. Nikita clearly wrote that the specifier is not recognized:

   2024-03-26T18:25:20.155339Z main ERROR Unrecognized format specifier [d]
   

2. In your "Deprecation warning for package scanning" you use a class attribute in the <CustomAppender> element:

   <CustomAppender name="CUSTOM_APPENDER" class="com.example.CustomAppender"/>

   Where did you find such an attribute? In Log4j 2.x we never used it.

[ppkarwasz]

@nikita51bot,

You are using the Shadow Gradle plugin to collapse all your application dependencies. By breaking the original packaging of the log4j-api and log4j-core JARs, you will encounter the following three problems:

• both JARs are multi-release JARs to work on both Java 8 and Java 11+. You need to set Multi-Release: true in your manifest (cf. Shadow plugin documentation):

  shadowJar {
    manifest {
      attributes 'Multi-Release': 'true'
    }
  }

• you overwrite Log4j2Plugins.dat files from multiple Log4j artifacts, which causes the problems with format specifiers and other Log4j plugins you have been experiencing. For this you need to use Sebastian Bögl's shadow-log4j-plugin:

  shadowJar{
    transform(com.github.jengelman.gradle.plugins.shadow.transformers.Log4j2PluginsCacheFileTransformer)
  }

• you overwrite ServiceLoader files from multiple JARs, so you need to merge them too (cf. merging service descriptor files):

  shadowJar {
    mergeServiceFiles()
  }

[nikita51bot]

Hi! Can you tell me, what should build.gradle look like with all the fixes? I tried this

plugins {
    id 'com.github.johnrengelman.shadow' version '8.1.1'
    id 'java'
}

group = 'org.example'
version = '1.0'

repositories {
    mavenCentral()
}

dependencies {
    implementation 'org.apache.logging.log4j:log4j-api:2.23.1'
    implementation 'org.apache.logging.log4j:log4j-core:2.23.1'
    annotationProcessor 'org.apache.logging.log4j:log4j-core:2.23.1'

    implementation 'org.jline:jline:3.25.1'

    compileOnly 'org.projectlombok:lombok:1.18.32'
    annotationProcessor 'org.projectlombok:lombok:1.18.32'
}

shadowJar {
    manifest {
        attributes(
                'Main-Class': 'org.example.Main',
                'Multi-Release': 'true'
        )
    }
    transform(com.github.jengelman.gradle.plugins.shadow.transformers.Log4j2PluginsCacheFileTransformer)
    mergeServiceFiles()
}

Unfortunately, it didn't give a positive result
META-INF\org\apache\logging\log4j\core\config\plugins\Log4j2Plugins.dat file is created in the jar file and contains (if you open it with sublime text with utf8 encoding):

<0x00><0x00><0x00><0x01><0x00><0x04>core<0x00><0x00><0x00><0x01><0x00><0x00>customappender<0x00><0x1a>org.example.CustomAppender<0x00><0x08>appender<0x01><0x00>

<0x00> is code character, not text

[ppkarwasz]

@nikita51bot,

The Log4j2PluginsCacheFileTransformer seems to be affected by two bugs:

• it does not transform Log4j2Plugins.dat files in your own project, but only those found in JAR files. You can work around this by creating a Gradle sub-project for the appender,
• it deletes all Log4j2Plugins.dat files if there is only one such file (this is similar to Log4j2PluginsCacheFileTransformer deletes cache if only one is available logging-log4j-transform#87). This should not be a problem in your case.

Edit: I added a comment about these problems to johnrengelman/shadow#427

[nikita51bot]

And how can I create a gradle subproject? I don't have much experience in development and I don't understand how to do this
I would be grateful if you could give me an example

[ppkarwasz]

nikita51bot/ExampleApp#1 should provide a workaround for the Gradle Shadow plugin issues.

However you should probably report the problem so that it can be fixed.

[nikita51bot]

Thank you so much! The solution to move the appender to a separate sub-project solved my problem!