Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Create a cluster table orm class with engines.Distributed which has a logs attribute, how to use variable to indicate it? #298

Open
flyly0755 opened this issue Mar 12, 2024 · 0 comments

Comments

@flyly0755
Copy link

flyly0755 commented Mar 12, 2024

For example, one cluster table orm class as below:

from clickhouse_sqlalchemy.ext.declarative import declarative_base
from clickhouse_sqlalchemy import engines
ChBase = declarative_base()
class TableXXXCluster(ChBase):
      # some filed attributes
    __tablename__ = 'TableXXXCluster'
    __table_args__ = (
        engines.Distributed(logs='cluster', default='currentDatabase()',
                            hits='TableXXX', sharding_key='fieldint'),
        {'comment': 'Lay LandMark cluster table'}
    )

def batch_create_tables(bind):
    ChBase.metadata.create_all(bind=bind) 

With logs attribute, there is some trouble when creating cluster table.
Use batch function batch_create_tables to batch create all tables with ChBase attribute showed as below.

ckuser = "ckuser"
ckpwd = "ckpwd"
ckip = "ckip"
ckport = "8123"
ckdbname = 'ckdb1'


uri = f"clickhouse://{ckuser}:{ckpwd}@" \
      f"{ckip}:{ckport}/{ckdbname}"
engine = create_engine(uri, echo=False)
session = sessionmaker(bind=engine)()
#session.execute('SELECT 1')
batch_create_tables(engine)
# or create single table 
TableXXXCluster.__table__.create(engine)

Can create cluster table TableXXXCluster on db ckdb1 and with Cluster cluster,
But if I want to create with Cluster 'test_shard_localhost' or some else,
I need to change the code from logs='cluster' to logs='test_shard_localhost', this is a very bad coding behavior.
Is there something like default='currentDatabase()' config, so can create cluster with extra clustername parameter like

engines.Distributed(logs='getCluster()', default='currentDatabase()',
                            hits='TableXXX', sharding_key='fieldint'),
kwargs = {'clustername': 'clusterA'}
batch_create_tables(engine, **kwargs) 
#or 
TableXXXCluster.__table__.create(engine, **kwargs)
@flyly0755 flyly0755 changed the title Create a cluster table orm class with engines.Distributed which has a logs attribute, how use variable to indicate it? Create a cluster table orm class with engines.Distributed which has a logs attribute, how to use variable to indicate it? Mar 13, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant