Kafka

Parameters

Name	Type	Description
Bootstrap Servers	String	A list of host/port pairs to use for establishing the initial connection to the Kafka cluster. The client will make use of all servers irrespective of which servers are specified here for bootstrapping—this list only impacts the initial hosts used to discover the full set of servers. This list should be in the form `host1:port1,host2:port2,...`. Since these servers are just used for the initial connection to discover the full cluster membership (which may change dynamically), this list need not contain the full set of servers (you may want more than one, though, in case a server is down).
Topic Pattern	String	Topic pattern in which the records will be sent. You can use patterns like '{namespace}' and/or '{stream}' to send the message to a specific topic based on these values. Notice that the topic name will be transformed to a standard naming convention.
Test Topic	String	Topic to test if Airbyte can produce messages.
Sync Producer	Boolean	Wait synchronously until the record has been sent to Kafka.
Protocol	Object	Protocol used to communicate with brokers.
Client ID	String	An ID string to pass to the server when making requests. The purpose of this is to be able to track the source of requests beyond just ip/port by allowing a logical application name to be included in server-side request logging.
ACKs	String	The number of acknowledgments the producer requires the leader to have received before considering a request complete. This controls the durability of records that are sent.
Enable Idempotence	Boolean	When set to 'true', the producer will ensure that exactly one copy of each message is written in the stream. If 'false', producer retries due to broker failures, etc., may write duplicates of the retried message in the stream.
Compression Type	String	The compression type for all data generated by the producer.
Batch Size	Integer	The producer will attempt to batch records together into fewer requests whenever multiple records are being sent to the same partition.
Linger ms	String	The producer groups together any records that arrive in between request transmissions into a single batched request.
Max in Flight Requests per Connection	Integer	The maximum number of unacknowledged requests the client will send on a single connection before blocking. Can be greater than 1, and the maximum value supported with idempotency is 5.
Client DNS Lookup	String	Controls how the client uses DNS lookups. If set to use_all_dns_ips, connect to each returned IP address in sequence until a successful connection is established. After a disconnection, the next IP is used. Once all IPs have been used once, the client resolves the IP(s) from the hostname again. If set to resolve_canonical_bootstrap_servers_only, resolve each bootstrap address into a list of canonical names. After the bootstrap phase, this behaves the same as use_all_dns_ips. If set to default (deprecated), attempt to connect to the first IP address returned by the lookup, even if the lookup returns multiple IP addresses.
Buffer Memory	String	The total bytes of memory the producer can use to buffer records waiting to be sent to the server.
Max Request Size	Integer	The maximum size of a request in bytes.
Retries	Integer	Setting a value greater than zero will cause the client to resend any record whose send fails with a potentially transient error.
Socket Connection Setup Timeout	String	The amount of time the client will wait for the socket connection to be established.
Socket Connection Setup Max Timeout	String	The maximum amount of time the client will wait for the socket connection to be established. The connection setup timeout will increase exponentially for each consecutive connection failure up to this maximum.
Max Block ms	String	The configuration controls how long the KafkaProducer's send(), partitionsFor(), initTransactions(), sendOffsetsToTransaction(), commitTransaction() and abortTransaction() methods will block.
Request Timeout	Integer	The configuration controls the maximum amount of time the client will wait for the response of a request. If the response is not received before the timeout elapses the client will resend the request if necessary or fail the request if retries are exhausted.
Delivery Timeout	Integer	An upper bound on the time to report success or failure after a call to 'send()' returns.
Send Buffer bytes	Integer	The size of the TCP send buffer (SO_SNDBUF) to use when sending data. If the value is -1, the OS default will be used.
Receive Buffer bytes	Integer	The size of the TCP receive buffer (SO_RCVBUF) to use when reading data. If the value is -1, the OS default will be used.

No items found.