问题描述:

I am using Maxwell to write changes from a MySql database to Kafka 0.9.0, is there a way to configure Maxwell, so it can partition the data in different topics by a hash function over a column?

网友答案:

You can specify database | table | primary_key | column for producer_partition_by configuration parameter.

As seen in the documentation here,

HASH_FUNCTION(HASH_STRING) % TOPIC.NUMBER_OF_PARTITIONS

The HASH_FUNCTION is either java's hashCode or murmurhash3. The default HASH_FUNCTION is hashCode. Murmurhash3 may be set with the kafka_partition_hash option. The seed value for the murmurhash function is hardcoded to 25342 in the MaxwellKafkaPartitioner class.

The HASH_STRING may be (database, table, primary_key, column). The default HASH_STRING is the database. The partitioning field can be configured using the producer_partition_by option.

相关阅读:
Top