Can gzip load this string into Amazon S3 without writing to disk?

Question

I know this is possible using Streams, but I was not sure of the correct syntax.

I would like to pass the string to the Save method and pass it to gzip and upload it to Amazon S3 without writing to disk. The current method inefficiently reads / writes to disk between them.

S3 PutObjectRequest has a constructor with InputStream as an option.

import java.io.*; import java.util.zip.GZIPOutputStream; import com.amazonaws.auth.PropertiesCredentials; import com.amazonaws.services.s3.AmazonS3; import com.amazonaws.services.s3.AmazonS3Client; import com.amazonaws.services.s3.model.PutObjectRequest; public class FileStore { public static void Save(String data) throws IOException { File file = File.createTempFile("filemaster-", ".htm"); file.deleteOnExit(); Writer writer = new OutputStreamWriter(new FileOutputStream(file)); writer.write(data); writer.flush(); writer.close(); String zippedFilename = gzipFile(file.getAbsolutePath()); File zippedFile = new File(zippedFilename); zippedFile.deleteOnExit(); AmazonS3 s3 = new AmazonS3Client(new PropertiesCredentials( new FileInputStream("AwsCredentials.properties"))); String bucketName = "mybucket"; String key = "test/" + zippedFile.getName(); s3.putObject(new PutObjectRequest(bucketName, key, zippedFile)); } public static String gzipFile(String filename) throws IOException { try { // Create the GZIP output stream String outFilename = filename + ".gz"; GZIPOutputStream out = new GZIPOutputStream(new FileOutputStream(outFilename)); // Open the input file FileInputStream in = new FileInputStream(filename); // Transfer bytes from the input file to the GZIP output stream byte[] buf = new byte[1024]; int len; while ((len = in.read(buf)) > 0) { out.write(buf, 0, len); } in.close(); // Complete the GZIP file out.finish(); out.close(); return outFilename; } catch (IOException e) { throw e; } } } 

Answer 1

I would use something like the following:

 ByteArrayOutputStream byteOut = new ByteArrayOutputStream(); GZipOuputStream gzipOut = new GZipOutputStream(byteOut); // write your stuff byte[] bites = byteOut.toByteArray(); //write the bites to the amazon stream 

You write the zip values ​​to the byte stream, and then take the byte values, you can write them to another stream. You can also transfer the stream to the amazon site (i.e., the Output stream from the http connection or something similar) and avoid all ByteArrayOutputStream.

---

Edit: I noticed your last suggestion is bleah. You can take the bytes you created, create a ByteArrayInputStream with them, and then pass this as input:

 ByteArrayInputStream byteInStream = new ByteArrayInputStream(bites); 

It should read from the input stream to the output stream, if I understand what you are describing correctly. Otherwise, you can simply write the output stream.

Answer 2

This is what aperkines have suggested. I don't know the AS3 interface, so his suggestion to create a ByteArrayInputStream over a byte array is probably the way out.

 import java.io.*; import java.util.zip.GZIPOutputStream; import com.amazonaws.auth.PropertiesCredentials; import com.amazonaws.services.s3.AmazonS3; import com.amazonaws.services.s3.AmazonS3Client; import com.amazonaws.services.s3.model.PutObjectRequest; public class FileStore { public static void Save(String data) throws IOException { ByteArrayOutputStream baos = new ByteArrayOutputStream(); Writer writer = new OutputStreamWriter(baos); writer.write(data); writer.flush(); writer.close(); byte[] zippedBytes = gzipFile(baos.toByteArray()); AmazonS3 s3 = new AmazonS3Client(new PropertiesCredentials( new FileInputStream("AwsCredentials.properties"))); String bucketName = "mybucket"; String key = "test/" + zippedFile.getName(); s3.putObject(new PutObjectRequest(bucketName, key, new ByteArrayInputStream(zippedBytes)); } public static byte[] gzipFile(byte[] bytes) throws IOException { try { ByteArrayOutputStream baos = new ByteArrayOutputStream(); GZIPOutputStream out = new GZIPOutputStream(baos); out.write(bytes, 0, bytes.length); // Complete the GZIP file out.finish(); out.close(); return baos.toByteArray(); } catch (IOException e) { throw e; } } }