AWS Lambda Streaming Streaming Client in Java

Question

I am trying to write a byte stream client in Java for the AWS Lambda function. I created the Lambda function as an implementation of RequestStreamHandler. The basis for this project is described in the documents here .

public class LambdaFunctionHandler implements RequestStreamHandler { static final String bucket = "anS3bucket"; static final String key = "anS3KeyToAJpegFile"; @Override public void handleRequest(InputStream input, OutputStream output, Context context) throws IOException { AmazonS3 s3Client = new AmazonS3Client( new EnvironmentVariableCredentialsProvider()); try { context.getLogger().log("Downloading an object\n"); S3Object s3object = s3Client.getObject(new GetObjectRequest( bucket, key)); context.getLogger().log("Content-Type: " + s3object.getObjectMetadata().getContentType() + "\n"); InputStream in = s3object.getObjectContent(); int b = 0; context.getLogger().log("Writing jpeg on output\n"); while ((b = in.read()) > -1) { output.write(b); } } catch (AmazonServiceException e) { System.out.println("Error Message: " + e.getMessage()); } } } 

This hard drive works great on the Lambda test console. I can load the JAR and run the lambda function (by clicking "Test"). What this function does is that it loads the contents of the jpeg file and writes the stream of bytes to the OutputStream. I see binary output in the test console as the result of a function. So far so good. In the end, I ran ImageMagick in jpeg and resized it - this is the goal of this project.

My client code looks like this:

 public interface ImageService { @LambdaFunction(functionName="ImageProcessing") OutputStream getImageStream(InputStream data); } public class LambdaImageTest { public static void main(String[] args) throws IOException { AWSLambdaClient lambda = new AWSLambdaClient(new ProfileCredentialsProvider()); lambda.configureRegion(Regions.EU_WEST_1); ImageService service = LambdaInvokerFactory.build(ImageService.class, lambda); // Call lambda function, receive byte stream OutputStream out = service.getImageStream(null); System.out.println(out); // This code is not complete } 

When I try to get a stream of bytes in a Java client, I fail. There seems to be no way to get a stream of bytes. It looks like the client is trying to read the reposition as json data, which I don't want here. I want to read a stream of bytes directly (binary content of jpeg). The error I am getting is:

 Exception in thread "main" com.amazonaws.services.lambda.invoke.LambdaSerializationException: Failed to parse Lambda function result at com.amazonaws.services.lambda.invoke.LambdaInvokerFactory$LambdaInvocationHandler.getObjectFromPayload(LambdaInvokerFactory.java:210) at com.amazonaws.services.lambda.invoke.LambdaInvokerFactory$LambdaInvocationHandler.processInvokeResult(LambdaInvokerFactory.java:189) at com.amazonaws.services.lambda.invoke.LambdaInvokerFactory$LambdaInvocationHandler.invoke(LambdaInvokerFactory.java:106) at com.sun.proxy.$Proxy3.getImageStream(Unknown Source) at se.devo.lambda.image.LambdaImageTest.main(LambdaImageTest.java:33) Caused by: com.fasterxml.jackson.core.JsonParseException: Invalid UTF-8 middle byte 0xff at [Source: [ B@42257bdd ; line: 1, column: 4] at com.fasterxml.jackson.core.JsonParser._constructError(JsonParser.java:1487) 

How to get byte stream data correctly in AWS Lambda java client?

Answer 1

I have found a solution. The LambdaInvokerFactory class will ALWAYS treat request and response data as JSON and therefore serialize and deserialize, which is the problem. However, the source code contains the key to the answer, and I snatched out the part that makes the lambda function call, but I bypassed JSON deserialization and directly addressed the payload. Simple, but it really should have been in the LambdaInvokerFactory class already ...

Here is my fully working solution. Lambda Function Code:

 public class LambdaFunctionHandler implements RequestStreamHandler { public void handleRequest(InputStream input, OutputStream output, Context context) throws IOException { ObjectMapper mapper = new ObjectMapper(); AmazonS3 s3Client = new AmazonS3Client( new EnvironmentVariableCredentialsProvider()); try { // Need to deserialize JSON data ourselves in Lambda streaming mode String data = getJSONInputStream(input); context.getLogger().log("JSON data:\n'" + data + "'\n"); context.getLogger().log("Deserialize JSON data to object\n"); ImageRequest request = mapper.readValue(data, ImageRequest.class); context.getLogger().log(String.format("Downloading S3 object: %s %s\n", request.getBucket(), request.getKey())); S3Object s3object = s3Client.getObject(new GetObjectRequest( request.getBucket(), request.getKey())); context.getLogger().log("Content-Type: " + s3object.getObjectMetadata().getContentType() + "\n"); InputStream in = s3object.getObjectContent(); int b = 0; byte[] buf = new byte[2048]; context.getLogger().log("Writing image on output\n"); while ((b = in.read(buf)) > -1) { output.write(buf, 0, b); } } catch (AmazonServiceException e) { System.out.println("Error Message: " + e.getMessage()); } } private String getJSONInputStream(InputStream input) throws IOException { BufferedReader reader = new BufferedReader(new InputStreamReader(input)); String data = ""; String line; while ((line = reader.readLine()) != null) { data += line; } return data; } } 

Client code:

 public class LambdaImageTest { private static final ObjectMapper MAPPER = new ObjectMapper(); public static void main(String[] args) throws IOException { String bucketName = args[0]; String key = args[1]; // Lambda client proxy AWSLambdaClient lambda = new AWSLambdaClient(new ProfileCredentialsProvider()); lambda.configureRegion(Regions.EU_WEST_1); // Build InvokeRequest InvokeRequest invokeRequest = buildInvokeRequest("ImageProcessing", new ImageRequest(bucketName, key)); // Invoke and get result payload as ByteBuffer. Note error handling should be done here InvokeResult invokeResult = lambda.invoke(invokeRequest); ByteBuffer byteBuffer = invokeResult.getPayload(); // Write payload to file. Output hardcoded... FileChannel out = new FileOutputStream("D:/test.jpg").getChannel(); out.write(byteBuffer); out.close(); } private static InvokeRequest buildInvokeRequest(String functionName, Object input) { InvokeRequest invokeRequest = new InvokeRequest(); invokeRequest.setFunctionName(functionName); // Lambda function name identifier invokeRequest.setInvocationType(InvocationType.RequestResponse); invokeRequest.setLogType(LogType.None); if (input != null) { try { String payload = MAPPER.writer().writeValueAsString(input); invokeRequest.setPayload(payload); } catch (JsonProcessingException ex) { throw new LambdaSerializationException("Failed to serialize request object to JSON", ex); } } return invokeRequest; } } 

It should be noted that there is a need to improve error handling. The source in LambdaInvokerFactory contains missing pieces.

Answer 2

Your ImageService interface should use object types instead of streams. For example, try:

 public interface ImageService { @LambdaFunction(functionName="ImageProcessing") byte[] getImageStream(byte[] data); }