Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Get the redo record's START_SCN and COMMIT_SCN #126

Open
vgrails opened this issue Apr 10, 2024 · 4 comments
Open

Get the redo record's START_SCN and COMMIT_SCN #126

vgrails opened this issue Apr 10, 2024 · 4 comments
Labels
enhancement New feature or request

Comments

@vgrails
Copy link

vgrails commented Apr 10, 2024

Is your feature request related to a problem? Please describe.
The OpenLogReplicator's json output has scn, c_scn field, like:

{"scn":358135,"tm":1712602205000000000,"c_scn":358141,"xid":"0x0005.00d.00000191","payload":[{"op":"begin"}]}
{"c_scn":358141,"xid":"0x0005.00d.00000191","payload":[{"op":"c","schema":{"owner":"TESTUSR1","table":"T2","obj":20060,"columns":[{"name":"ID","type":"number","precision":-1,"scale":0,"nullable":false,"key_type":2,"key_name":"SYS_C007005","order_by":"ASC"},{"name":"DATA","type":"varchar2","length":128,"nullable":true,"key_type":0,"key_name":"","order_by":"ASC"}]},"after":{"ID":3,"DATA":"##########STR##########STR##########STR##########STR##########STR##########STR##########STR#########3"}}]}
{"c_scn":358141,"xid":"0x0005.00d.00000191","payload":[{"op":"commit"}]}

while dumping redo from Logminer by SQL:

SELECT (XIDUSN || '.' || XIDSLT || '.' || XIDSQN) AS XID, SCN,START_SCN,COMMIT_SCN, TABLE_NAME, SQL_REDO FROM V\$LOGMNR_CONTENTS WHERE REGEXP_LIKE(TABLE_NAME, '(T1|T2|T3|T4|T5)') AND SCN >=${scn}

result is like:

SCN:358141 START_SCN:358135 COMMIT_SCN:358142 XID:5.13.401 SQL:insert into "TESTUSR1"."T2"("ID","DATA") values ('3','##########STR##########STR##########STR##########STR##########STR##########STR##########STR#########3');

after a little big digging, we can find that:

  1. the transaction id (xid) is exactly the same
  2. the data is exactly the same
  3. openlogreplicator's start scn and c_scn is the not same as logminer's start_scn;

Describe the solution you'd like
Know the matching start_scn and commit_scn in openlogreplicator's output or how to calculate the start_scn and commit_scn by simple math op from current JSON output.

BTW, if the meaning of scn in begin output (in above example, is 358135) the meaning of c_scn in commit output? (in above example, is 358141)? Can find any matching record in Logminer's output

Describe alternatives you've considered
not found yet, but maybe can use XID to identify the transaction but it is not monotonically increasing

Additional context
primary usage of start_scn and commit_scn is for smooth connection of full sync and increment sync and validation

@vgrails
Copy link
Author

vgrails commented Apr 10, 2024

Thanks for amazing high performance and free Oracle redo parser, Bersler !

@bersler
Copy link
Owner

bersler commented May 8, 2024

@vgrails , Can you provide some redo log file which contains the invalid SCN value?

@bersler
Copy link
Owner

bersler commented May 8, 2024

For me it is a problem to set up everything, logminer, config, etc. I need either a script to run to get the results or some redo log data to reproduce the issue.

@bersler bersler added bug Something isn't working reproduction missing labels May 8, 2024
@bersler bersler added enhancement New feature or request and removed bug Something isn't working reproduction missing labels May 17, 2024
@bersler
Copy link
Owner

bersler commented May 17, 2024

After thinking about it I would say this it not a but rather an enhancement. By default OLR puts the commit timestamp to all DML operations from the transaction. This is more logical. Actually if you look at the database and would like to restore the content up to some SCN - the particular DML did not change at that moment. Thus it is misleading that it changed at that SCN. It actually changed at the moment when the transaction has been successfully committed.
Thus, the actual SCN of the DML operation should be the COMMIT SCN.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants