Avro Schema Nesting Types

I am new to Avro and am looking at documentation for nested types. I have an example below that works well, but many different types inside the model will have addresses. Is it possible to define the address.avsc file and the link to it as a nested type? If possible, can you take one more step and have a list of addresses for the client? Thanks in advance.

{"namespace": "com.company.model", "type": "record", "name": "Customer", "fields": [ {"name": "firstname", "type": "string"}, {"name": "lastname", "type": "string"}, {"name": "email", "type": "string"}, {"name": "phone", "type": "string"}, {"name": "address", "type": {"type": "record", "name": "AddressRecord", "fields": [ {"name": "streetaddress", "type": "string"}, {"name": "city", "type": "string"}, {"name": "state", "type": "string"}, {"name": "zip", "type": "string"} ]} } ] } 
+6
source share
2 answers

There are 4 possible ways:

  • Including it in the pom file as indicated in this.
  • Declare all your types in one avsc file.
  • Using a single static analyzer, which first analyzes all imported data, and then analyzes the actual data types.
  • (This is a hack) Use the avdl file and use the import, for example https://avro.apache.org/docs/1.7.7/idl.html#imports . Although the IDL is for RPC calls.

Example for 2. Declare all your types in one avsc file. Also responsible for declaring the array at.

 [ { "type": "record", "namespace": "com.company.model", "name": "AddressRecord", "fields": [ { "name": "streetaddress", "type": "string" }, { "name": "city", "type": "string" }, { "name": "state", "type": "string" }, { "name": "zip", "type": "string" } ] }, { "namespace": "com.company.model", "type": "record", "name": "Customer", "fields": [ { "name": "firstname", "type": "string" }, { "name": "lastname", "type": "string" }, { "name": "email", "type": "string" }, { "name": "phone", "type": "string" }, { "name": "address", "type": { "type": "array", "items": "com.company.model.AddressRecord" } } ] }, { "namespace": "com.company.model", "type": "record", "name": "Customer2", "fields": [ { "name": "x", "type": "string" }, { "name": "y", "type": "string" }, { "name": "address", "type": { "type": "array", "items": "com.company.model.AddressRecord" } } ] } ] 

Example for 3. Using a single static analyzer

 Parser parser = new Parser(); // Make this static and reuse parser.parse(<location of address.avsc file>); parser.parse(<location of customer.avsc file>); parser.parse(<location of customer2.avsc file>); 

If we want to carry out the scheme, that is, if we want to create new records, we can either do https://avro.apache.org/docs/1.5.4/api/java/org/apache/avro/Schema.Parser. html # getTypes () method to get the schema or

 Parser parser = new Parser(); // Make this static and reuse Schema addressSchema =parser.parse(<location of address.avsc file>); Schema customerSchema=parser.parse(<location of customer.avsc file>); Schema customer2Schema =parser.parse(<location of customer2.avsc file>); 
+17
source

To add @Princey James to the answer, the nested type must be defined before it is used.

0
source

Source: https://habr.com/ru/post/984270/


All Articles