Configuring Avro schema for Kafka data set
When you configure a Kafka data set, you can choose Apache Avro as your data format for the Kafka message values and message keys. Avro is a lightweight binary message encoding, which relies on schemas to structure the encoded data.
Avro schemas are held in external storage. Your keys and values may have different schemas, which are stored in different registries. Configure an Avro schema by following the steps below.
For more information, see the Apache Avro documentation.
- On your Kafka data set creation form, in the Message values or Message keys section, select Avro from the Data format list.
- Click the Open icon next to the Avro schema field.
- In the Short description field, enter a short description or title.
- In the Avro schema field, enter an Avro schema
name.Your Avro schema name must start with a letter.
- Click Create and Oopen.
- In the Schema registry section, in the URL field, enter the URL of the registry in which you want to store the schema.
- In the Authentication source section, select the
authentication option for the registry.
- To disable authentication, select No authentication.
- To enable authentication, select Authentication, and then enter a user name and password.
- Optional: To associate keys and values with your Avro schema, click Add key-value pair.
- Optional: To upload your Avro schema, click the Upload schema
button.
- Optional: To use versioning of the schemas, select the Perform schema
evolution when reading messages check box. Click
Upload different schema, and then upload your second
schema.Schema evolution ensures backward compatibility of the updated schema with the previous schemas.
- Click Save.
Previous topic Configuring application settings for Kafka data set topics Next topic Creating a Kinesis data set