Skip to content
/ xdmq Public

next-generation stock online transaction processing system POC. distributed message queue that support total ordering and fault-tolerance.

Notifications You must be signed in to change notification settings

xnum/xdmq

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

16 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

# Xnum Distributed Message Queue

I don't force SYNC any data to disk but mmaped-file.

## Reqs

- libuv
- raft
- msgpack

## For producer

send TCP Packet which follows `msg.h` spec

pick up a random 64bit id to identify your message, duplication is acceptable but just confuse yourself.

The id may be used to filter duplicated message in the future.

Once the message has been committed, server replys you id with OK or Error.

The process flow is async so you don't have to wait.

Connect to a random node, if it's not the leader, it'll tell you and close connection.

## For consumer

send line-end message

`GET N M` results the message index from N to M (less than)

`INFO` results current last index 

## Performance

1024 * 256 messages from seperate 4 producers => 54s

## Known Bugs

### missing latest line

1. producers sending data
2. follower disconnected
3. follower reconnected
4. leader keeps check latest message commited
5. followers missed latest data compares with leader

If clients reconnect and sends data continusly, all logs are keep same

For this bug, because the client waits for the log commit and don't do anything.
And leader wait for append entry response, there aren't any data produced.
therefore no log check procedure were triggered.

So if any request timedout happened, the producer should just reconnect and send again.

About

next-generation stock online transaction processing system POC. distributed message queue that support total ordering and fault-tolerance.

Topics

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published