How to smooth the upgrade after transferring to an apache organization

[funky-eyes]

迁移到apache organization后，原io.seata包下的代码都要迁移至org.apache.seata中，这种改动很可能会造成用户侧升级时需要改造代码，故我们需要讨论一个方法尽量减少用户的迁移成本。
首先2.x分支上我们已经对tcc以及事务代理这块的spring依赖做了去除，目前遗留的可能是saga模块中对spring的强依赖是一个问题。
我们可以参考dubbo社区的经验，将api和注解保留io.seata这个包，在org.apache.seata中新增相关的api和注解。

1. 在扫描器，代理中增加对org.apache.seata包中的注解支持
2. 对于api，将实现代码转移到org.apache.seata包中，io.seata中仅需集成org.apache.seata包中的类，如TransactionalTemplate：

public class TransactionalTemplate extends org.apache.seata.TransactionalTemplate{
}

After migrating to the Apache organization, the code under the original io.seata package needs to be transferred to org.apache.seata. This kind of change is likely to require code modifications when users upgrade, so we need to discuss a method to minimize the migration cost for users.
Firstly, on the 2.x branch, we have already removed the spring dependencies for TCC and transaction proxy. The remaining issue may be the strong dependency on Spring in the saga module.
We can refer to the experience of the Dubbo community, where the API and annotations are retained in the io.seata package, and 1. relevant API and annotations are added in the org.apache.seata package.

1. Add support for annotations from the org.apache.seata package in scanners and proxies.
2. For the API, move the implementation code to the org.apache.seata package, and in the io.seata package, only integrate the classes from the org.apache.seata package, such as TransactionalTemplate:

public class TransactionalTemplate extends org.apache.seata.TransactionalTemplate{
}

[slievrly]

https://github.com/apache/incubator-seata/wiki/Design-of-io.seata-Upgrade-Compatibility-Solution

[ptyin]

API 梳理

TODO

• 对于每个 API 建一个 sample 进行测试 ➡️ sample/api
• SPI 扩展进行通用处理
• 序列化带类名
• 漏掉的 API 后续版本可以补充

API

DefaultHttpExecutor.java

全局事务生命周期

tm.api 包

RootContext.java

TMClient.java

RMClient.java

数据源代理

DataSourceProxy.java

DataSourceProxyXA.java

Spring

spring.annotation

GlobalTransactionScanner.java

EnableAutoDataSourceProxy.java

TCC

seata-tcc rm.tcc.api

ProxyUtil.java

BusinessActionContext.java

BusinessActionContextParameter.java

Saga

核心入口

StateMachineEngine.java

StateMachineConfig.java

实现

ProcessCtrlStateMachineEngine.java

DefaultStateMachineConfig.java

DbStateMachineConfig.java

组件

AsyncCallback.java

ProcessContext.java

HierarchicalProcessContext.java

实现

ProcessContextImpl.java

领域模型

domain 包下所有内容

验证

验证规则

Rule.java

SPI 扩展

拦截器

StateHandlerInterceptor.java

StateRouterInterceptor.java

TransactionPropagationInterceptor.java
https://github.com/apache/incubator-seata/blob/2.x/integration/http-jakarta/src/main/java/io/seata/integration/http/JakartaTransactionPropagationInterceptor.java

https://github.com/apache/incubator-seata/tree/2.x/integration/grpc

[funky-eyes]

Since the raft protocol class also needs to retain the old package names for compatibility with the
io.seata.server.cluster.raft.snapshot.RaftSnapshot

[funky-eyes]

Rraft synchronization messages in io.seata.server.cluster.raft.sync.msg.dto need to be handled with compatibility

[funky-eyes]

io.seata:seata-server迁移至org.apache.seata:seata-server待讨论问题:
当新老版本seata-server采用raft集群同步时，新版本使用的新class下的,由于当时设计的序列化器使用了jackson以及协议类使用了writeObject，纷纷记录了类的信息，如io.seata.server.cluster.raft.sync.msg.dto包下的类，目前在新版本上可以通过保留老的类来做到向下兼容，但是如果新版本的seata-server一旦被选为leader，由它发起同步task，老版本的seata-server将因为本地没有这个类，而导致序列化失败。
当然有几种解决方案来处理：
1.保留所有的老类，同步时直接使用老类，百分百兼容，但是会导致这部分类一直是在io.seata包下 （简单，但是太过粗糙，做法不优雅）

2.保留所有老类，增加集群间的版本协商能力，通过jraft扩展的出集群同步能力，在选主前记录每个节点的版本，使用最低版本的协议进行兼容，直到所有节点都是最新版才启用新包下的同步类。 （复杂度高，但100%兼容）

3.仅保留协议类，因为新版向下兼容，升级的过程中会重选举，新节点不太可能成为leader，只要新节点超过半数，老节点序列化异常并不会影响集群稳定性，举个例子假设节点 1,2,3 3为leader，1和2滚动升级，3继续为leader，1,2节点兼容节点3的同步消息，集群没有任何影响，最后再升级节点3，集群完成升级。如果节点2为leader的情况下升级，那么升级1,2后，触发重选举，可能1,2之间产生一个新leader，新leader进行同步时，节点3由于序列化失败，状态机不会继续往下执行，但是同步log继续会落到磁盘中，当节点3升级后，他便能解析之前的raft log，能迅速恢复的数据一致的状态。（高度兼容，官网侧升级方案即可）
希望大家能参与讨论下raft集群模式下的io.seata:seata-server如何平滑迁移至org.apache.seata:seata-server中来。

The issue being discussed is the migration from io.seata:seata-server to org.apache.seata:seata-server in a Raft cluster synchronization scenario. The problem arises due to the use of different packages and changes in class paths. In the new version, the classes are located in a different package, causing deserialization failures in the old version when it acts as the leader and initiates synchronization tasks.

There are several proposed solutions:

1.Preserve all the old classes and use them directly for synchronization. This ensures complete compatibility but results in the classes remaining in the io.seata package. This approach is simple but not elegant.

2.Preserve all the old classes and introduce version negotiation capabilities between cluster nodes. Using extensions provided by jraft, record the version of each node before leader election and use the lowest version protocol for compatibility until all nodes are upgraded to the latest version. This solution is more complex but guarantees 100% compatibility.

3.Keep only the protocol classes. Since the new version is backward compatible, during the upgrade process, there will be a re-election, and it is unlikely that a new node will become the leader. As long as more than half of the nodes are upgraded, the serialization failures in the old nodes won't affect the stability of the cluster. This solution provides a high level of compatibility and can be considered as an official upgrade plan.

The discussion aims to determine the best approach for a smooth migration from io.seata:seata-server to org.apache.seata:seata-server in a Raft cluster mode.

[funky-eyes]

Involves modules that need to be compatible with the io.seata package.
seata-http-jakarta
seata-http
tcc
seata-saga-statelang
seata-saga-engine
seata-saga-engine-store
integration-tx-api
spring
rm-datasource
tm
seata-grpc
server
The modules with relatively low impact are.
seata-grpc
integration-tx-api (because 2.0 only launched, basically internal call, external direct use of api should be less users)
seata-http-jakarta (springboot3 and tomcat10 should have fewer users for now)
server (The raft cluster mode was only introduced in 2.0, and it is marked as a sexual function, so it is theoretically possible that compatibility is not 100% compatible)

[xingfudeshi]

I'll handle the following module:
integration-tx-api
spring
rm-datasource
tm

[xingfudeshi]

Progress Update:
Except for the server module, I have made compatibility with lower versions of all interfaces related to SPI,the PR is #6315. These modules are as follows:

In addition, during the processing, I found that the integration of Apache Dubbo used the following dependencies[1]:

<dependency> 
            <groupId>org.apache.dubbo.extensions</groupId> 
            <artifactId>dubbo-filter-seata</artifactId> 
            <exclusions> 
                <exclusion> 
                    <groupId>org.apache.seata</groupId> 
                    <artifactId>seata-core</artifactId> 
                </exclusion> 
            </exclusions> 
        </dependency> 

It references the package name of io. seata, and I have already made it compatible. However, in the future, we may need to upgrade it to. org. apache. seata.

[1]https://github.com/apache/dubbo-spi-extensions/tree/master/dubbo-filter-extensions/dubbo-filter-seata/src/main/java/org/apache/dubbo/seata

[funky-eyes]

我想我遗漏了一点，我们的spi是否要向下兼容，这是一个问题。让我举个例子以便于理解：
假设用户使用io.seata.rm.datasource.exec.InsertExecutor，当我们改动了包名就为org.apache.seata.rm.datasource.exec.InsertExecutor
此时用户升级了seata版本为apache seata时就会失效。
我不确定我们是否要像dubbo一样，对老的spi实现保持兼容。

I think I'm missing the point that it's a matter of whether or not our spi is going to be backward compatible. Let me give you an example to make it easier to understand: Suppose the user is using io.seata.rm.datasource.exec.InsertExecutor and when we change the package name to org.apache.seata.rm.datasource.exec. InsertExecutor fails when the user upgrades the seata version to apache seata. I'm not sure we want to keep compatibility with older spi implementations like dubbo.

[xingfudeshi]

My suggestion is to make it fully compatible to avoid problems after users upgrade.

[xingfudeshi]

The compatibility processing of the following modules has been completed:
integration-tx-api
spring
rm-datasource
tm
Next, I will continue to work on the TCC module

[slievrly]

At present, the modules I have completed include:
seata-http-jakarta
seata-http
seata-grpc