java使用maven-shade-plugin打包
<p>最近正在练习Flink JAVA编程，但是发现打包后老大的问题。</p><p>环境说明：</p>
<ul>
<li>jdk17</li>
<li>springboot 3.4.6</li>
<li>flink 1.20.0</li>
</ul>
<p> </p>
<h1>一、问题发现和解决</h1>
<p>在pom.xml的依赖部分，和flink有关的内容如下：</p>
<pre class="highlighter-prismjs language-xml prismjs-lines-highlighted" tabindex="0"><code> <!-- Original Flink Dependencies -->
<dependency>
<groupId>org.apache.flink</groupId>
<artifactId>flink-core</artifactId>
<version>${flink.version}</version>
<exclusions>
<exclusion>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-api</artifactId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>org.apache.flink</groupId>
<artifactId>flink-streaming-java</artifactId>
<version>${flink.version}</version>
<exclusions>
<exclusion>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-api</artifactId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>org.apache.flink</groupId>
<artifactId>flink-table-api-java</artifactId>
<version>${flink.version}</version>
<!--<scope>provided</scope> -->
<exclusions>
<exclusion>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-api</artifactId>
</exclusion>
</exclusions>
</dependency>
<!-- tableapi 桥接器，主要和DataStream进行对接 -->
<dependency>
<groupId>org.apache.flink</groupId>
<artifactId>flink-table-api-java-bridge</artifactId>
<version>${flink.version}</version>
<!--<scope>provided</scope> -->
<exclusions>
<exclusion>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-api</artifactId>
</exclusion>
</exclusions>
</dependency>
<!-- ide调试 -->
<dependency>
<groupId>org.apache.flink</groupId>
<artifactId>flink-table-planner-loader</artifactId>
<version>${flink.version}</version>
<!--<scope>provided</scope> -->
<exclusions>
<exclusion>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-api</artifactId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>org.apache.flink</groupId>
<artifactId>flink-table-runtime</artifactId>
<version>${flink.version}</version>
<exclusions>
<exclusion>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-api</artifactId>
</exclusion>
</exclusions>
</dependency>
<!-- 连接器 -->
<dependency>
<groupId>org.apache.flink</groupId>
<artifactId>flink-connector-files</artifactId>
<version>${flink.version}</version>
</dependency>
<dependency>
<groupId>org.apache.flink</groupId>
<artifactId>flink-csv</artifactId>
<version>${flink.version}</version>
</dependency>
<dependency>
<groupId>org.apache.flink</groupId>
<artifactId>flink-connector-jdbc</artifactId>
<version>${flink-connector.jdbc.version}</version>
</dependency>
<!-- 执行工厂定义 -->
<dependency>
<groupId>org.apache.flink</groupId>
<artifactId>flink-clients</artifactId>
<version>${flink.version}</version>
<exclusions>
<exclusion>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-api</artifactId>
</exclusion>
</exclusions>
</dependency></code></pre>
<p> </p>
<p>一开始用的是spring-boot-maven-plugin进行打包，但是总会提示某些类找不到。</p>
<p>事实是有关的jar都打入了，但就是无法找到。主要的原因是SPI无法找到（在META-INF/services/下无法找到对应的spi定义）</p>
<p>折腾了一会，发现无法搞定。</p>
<p>后来发现可以使用maven-shade-plugin解决这个问题。</p>
<p>注：SPI即Service Provider Interface，中文称为<strong>服务供应商接口</strong></p>
<p>参考地址：https://docs.oracle.com/en/java/javase/11/docs/api/java.base/java/util/ServiceLoader.html</p>
<p> </p>
<p>打包配置如下：</p>
<pre class="highlighter-prismjs language-xml prismjs-lines-highlighted" tabindex="0"><code><build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<version>3.13.0</version>
<configuration>
<source>${java.version}</source>
<target>${java.version}</target>
<encoding>${project.build.sourceEncoding}</encoding>
<parameters>true</parameters>
</configuration>
</plugin>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-shade-plugin</artifactId>
<version>3.6.0</version>
<executions>
<!-- Run shade goal on package phase -->
<execution>
<phase>package</phase>
<goals>
<goal>shade</goal>
</goals>
<configuration>
<artifactSet>
<!-- 如果想排除一些包 -->
<excludes>
<!--<exclude>org.apache.flink:force-shading</exclude>-->
<exclude>com.google.code.findbugs:jsr305</exclude>
</excludes>
</artifactSet>
<filters>
<filter>
<!-- Do not copy the signatures in the META-INF folder.
Otherwise, this might cause SecurityExceptions when using the JAR. -->
<artifact>*:*</artifact>
<excludes>
<exclude>META-INF/*.SF</exclude>
<exclude>META-INF/*.DSA</exclude>
<exclude>META-INF/*.RSA</exclude>
</excludes>
</filter>
</filters>
<transformers>
<transformer implementation="org.apache.maven.plugins.shade.resource.ManifestResourceTransformer">
<mainClass>org.lzfto.flink.demo.DemoApplication</mainClass>
</transformer>
<transformer implementation="org.apache.maven.plugins.shade.resource.ServicesResourceTransformer"/>
</transformers>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
<resources>
<resource>
<directory>src/main/resources/</directory>
<includes>
<include>**/*.js</include>
<include>**/*.css</include>
<include>**/*.yml</include>
<include>**/*.xml</include>
<include>**/*.txt</include>
<include>**/*.png</include>
<include>**/*.jpeg</include>
<include>**/*.jpg</include>
<include>**/*.svg</include>
<include>**/*.properties</include>
<include>**/*.Factory</include>
<include>**/*.json</include>
<include>META-INF/services/*.Factory</include>
</includes>
</resource>
<resource>
<!-- directory 表示取该目录下的文件 -->
<directory>libs</directory>
<!--targetPath 指定打包到哪个目录下 默认是放到class目录下 -->
<targetPath>/BOOT-INF/lib/</targetPath>
<!-- 取符合格式的所有文件 *代表全部 -->
<includes>
<include>**/*.jar</include>
</includes>
</resource>
</resources>
<finalName>lzfto-${lzfto.flinktest.version}</finalName>
</build></code></pre>
<p> </p>
<p>Shade插件使用ServicesResourceTransformer解决SPI加载问题。</p>
<p> </p>
<h1>二、比较Shade和Springboot</h1>
<p> </p>
<p>为了便于行文，以Shade表示maven-shade-plugin,以Springboot表示spring-boot-maven-plugin</p>
<p>也就是说一般情况下，使用<span style="background-color: rgba(255, 255, 255, 1); padding: 0 0 0 2px"><span style="color: rgba(0, 0, 0, 1); background-color: rgba(255, 255, 255, 1); font-family: "幼圆"; font-size: 11pt; white-space: pre">maven-shade-plugin</span></span>比使用spring-boot-maven-plugin更好一些，好处有两点：</p>
<p><strong>1.很容易解决SPI问题。也许spring-boot-maven-plugin可以通过自定义LayOut解决SPI问题</strong></p>
<p><strong>2.Shade打好的包加载速度更快</strong>，看下图（用rar打开后的看到的文件结构),并没有Springboot那样把库都放在BOOT-INF/libs，而是完全展开的class</p>
<p><img src="https://img2024.cnblogs.com/blog/1177268/202506/1177268-20250603152150210-1800381093.png"></p>
<p> </p>
<p>当然，springboot也并非没有优势，使用较高版本的springboot，有一些额外的配置（看个人需要），例如（部分特新示例)<span style="background-color: rgba(255, 255, 255, 1); padding: 0 0 0 2px"><span style="color: rgba(0, 0, 0, 1); background-color: rgba(255, 255, 255, 1); font-family: "幼圆"; font-size: 11pt; white-space: pre">：</span></span></p>
<p><span style="background-color: rgba(255, 255, 255, 1); padding: 0 0 0 2px"><span style="color: rgba(0, 0, 0, 1); background-color: rgba(255, 255, 255, 1); font-family: "幼圆"; font-size: 11pt; white-space: pre">参考：https://docs.spring.io/spring-boot/maven-plugin/build-info.html</span></span></p>
<pre class="highlighter-prismjs language-xml prismjs-lines-highlighted" tabindex="0"><code><project>
<modelVersion>4.0.0</modelVersion>
<artifactId>build-info</artifactId>
<build>
<plugins>
<plugin>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-maven-plugin</artifactId>
<executions>
<execution>
<goals>
<goal>build-info</goal>
</goals>
<configuration>
<additionalProperties>
<encoding.source>UTF-8</encoding.source>
<encoding.reporting>UTF-8</encoding.reporting>
<java.version>${java.version}</java.version>
</additionalProperties>
</configuration>
</execution>
</executions>
</plugin>
</plugins>
</build>
</project></code></pre>
<p> </p>
<p>大部分情况下，二者都需要maven-compiler-plugin解决某些问题，例如：</p>
<ol>
<li>指定编译参数，如<code class=" inline">-parameters</code>保留方法参数名、<code class=" inline">-Xlint</code>启用警告检查等</li>
<li>指定编译版本和代码编码</li>
<li>处理模块系统</li>
</ol>
<p>有关编译器插件的内容参见：https://maven.apache.org/plugins/maven-compiler-plugin/</p>
<p> </p>
</div>
<div id="MySignature" role="contentinfo">
<p>本文来自博客园，作者：正在战斗中，转载请注明原文链接：https://www.cnblogs.com/lzfhope/p/18908589</p><br><br>
来源：https://www.cnblogs.com/lzfhope/p/18908589 楼主好！看到你在折腾Flink打包，这坑我当初也踩过不少哈哈。用maven-shade-plugin打Flink项目最核心的原则就是：绝对不要把Flink自身的依赖打进最终jar包！不然提交到集群百分百会报类冲突或者ClassNotFound。
给你几个实操建议：
[]把pom里所有flink相关的依赖scope都改成provided，这样本地编译能过，打包时也不会被塞进去。
[]SpringBoot的打包插件和shade插件机制冲突，建议直接关掉spring-boot-maven-plugin，纯靠shade来生成fat jar。
[*]如果遇到第三方依赖冲突（比如slf4j、guava这些），可以在shade插件配置里用relocation给它们换个包名，或者用filters把重复的类过滤掉。
Flink官方其实有专门的打包最佳实践文档，照着配基本能避开90%的坑。要是还有具体的报错日志，随时贴出来大家帮你一起排查！祝你早日跑通，开发之路越来越顺！
頁:
[1]