What is the best way to specify Protobuf for use with Netty (preferably using the built-in protobuf support)

Question

I specify the protocol in the protocol buffers . The transport layer uses Netty Protocol Buffer Support - the significance is that Netty ProtobufDecoder accepts one and only one MessageLite type.

Now I want to send different types of messages on this channel, with each subtype having structured information associated with it. Protocol buffers do not have an inheritance mechanism, so I use some kind of composition. I'm not sure if I will do it right.

My approach was to classify my different events with an enumeration and encapsulate their differences using optional members. See My .proto below, I simplified it for clarity.

My problem is that the receiving code needs to establish a connection between EventType.ERROR and ErrorEventDetail. It's just a little awkward.

Simplified Events.proto :

 package events; option java_package = "com.example"; option java_outer_classname = "EventProtocol"; message Event { enum EventType { START = 0; DELEGATE = 1; ERROR = 2; STOP = 3; } required events.Event.EventType event_type = 1 [default = START]; required int32 id = 2; required int64 when = 3; optional StartEventDetail start_event_detail = 4; optional DelegateEventDetail delegate_event_detail = 5; optional ErrorEventDetail error_event_detail = 6; optional StopEventDetail stop_event_detail = 7; } message StartEventDetail { required string object_name = 1; } message DelegateEventDetail { required int32 object_id = 2; required string task = 3; } message ErrorEventDetail { required string text = 1; required int32 error_code = 2; optional Event cause = 3; } message StopEventDetail { required int32 object_id = 2; } 

Is this optimal? Would I be better off using the extension in some way, or maybe another use of enum ?

Or even, should I create a brand new OneToOneDecoder that can identify the message type by some kind of header? I could do it, but I would rather not ...

thanks

Answer 1

It looks like you're pretty close / already using one of the Google protobufs methods called Connection Types

The bottom line is that you have a dedicated type field that you would "switch" to find out which message to receive:

 message OneMessage { enum Type { FOO = 1; BAR = 2; BAZ = 3; } // Identifies which field is filled in. required Type type = 1; // One of the following will be filled in. optional Foo foo = 2; optional Bar bar = 3; optional Baz baz = 4; } 

where Foo, Bar and Baz can be defined as separate messages in other files. And you can enable the type to get the actual payload (this is Scala, but you can do the same with the Java switch ):

 OneMessage.getType match { case OneMessage.Type.FOO => val foo = OneMessage.getFoo // do the processing true case OneMessage.Type.BAR => val bar = OneMessage.getBar // do the processing true case OneMessage.Type.BAZ => val baz = OneMessage.getBaz // do the processing true } 

Answer 2

I initially solved the same problem using the extension mechanism that I am documenting here

But I found that the Java code needed to work with extensions was terribly ugly and verbose, so I switched to the Union method as described. The code is much cleaner because the generated Java code provides a way to receive and assemble each message at a time.

I use two mechanisms to determine which additional message to retrieve. I use the switch method, also described in another answer when performance is required, and I use the reflection method when performance is not a problem, and I do not want to support the switch statement, I just create a Message handle for each message. The following is an example of the reflection method, in my case the java shell is a class called Commands, and Netty is decoded for me. First, he tries to find a handler that has a specific message as a parameter, and then, if that fails, he calls a method that uses the name of the camel case. For this to work, Enum must have an underlined name for the camel case report.

 // Helper that stops me having to create a switch statement for every command // Relies on the Cmd enum naming being uppercase version of the sub message field names // Will call the appropriate handle(Message) method by reflection // If it is a command with no arguments, therefore no sub message it // constructs the method name from the camelcase of the command enum private MessageLite invokeHandler(Commands.Command cmd) throws Exception { Commands.Command.Cmd com= cmd.getCmd(); //String name= CaseFormat.UPPER_UNDERSCORE.to(CaseFormat.LOWER_UNDERSCORE, com.name()); String name= com.name().toLowerCase(); jlog.debug("invokeHandler() - Looking up {} from {}", name, com.name()); FieldDescriptor field= Commands.Command.getDescriptor().findFieldByName(name); if(field != null) { // if we have a matching field then extract it and call the handle method with that as a parameter Object c = cmd.getField(field); jlog.debug("invokeHandler() - {}\n{}", c.getClass().getCanonicalName(), c); Method m = getClass().getDeclaredMethod("handle", String.class, c.getClass()); return (MessageLite) m.invoke(this, cmd.getUser(), c); } // else we call a method with the camelcase name of the Cmd, this is for commands that take no arguments other than the user String methodName= "handle"+CaseFormat.UPPER_UNDERSCORE.to(CaseFormat.UPPER_CAMEL, com.name()); jlog.debug("invokeHandler() - using method: {}", methodName); Method m = getClass().getDeclaredMethod(methodName, String.class); return (MessageLite) m.invoke(this, cmd.getUser()); } 

Answer 3

another approach is to use the extension mechanism supported by protobuf. I use this approach in situations where the type of union is too large.