Set string value on map only if it matches threshold bytes

Question

I have a tasks list object that I repeat and add each task object to StringBuilder , and then a new line, as shown below. Now I will continue to add the task object to the same string builder until it reaches the limit of 60,000 bytes. As soon as it reaches the limit, I will fill this line as a value on the map, and the key will have a file name with an incremental index. And then I will build a reset line constructor and another thing and repeat this process again.

So, if I have a large tasks object, then I will split it into several string objects, the size of which should be less than 60,000 bytes.

I got the code below, but I always see that the value on the card is larger than 60,000 bytes. Something is wrong, what am I doing? I also populate the HashMap in two different places - one if the limit is reached, and the other if the limit is not reached.

  public void populate(final List<Task> tasks) { Map<String, String> holder = new HashMap<>(); int size = 0; int index = 0; StringBuilder sb = new StringBuilder(); for (Task task : tasks) { sb.append(task).append(System.getProperty("line.separator")); size = sb.toString().getBytes(StandardCharsets.UTF_8).length; if (size > 60000) { String fileName = "tasks_info_" + index + ".txt"; holder.put(fileName, sb.toString()); index++; sb = new StringBuilder(); size = 0; } } // for cases where we don't reach the limit if(sb.toString().length > 0) { String fileName = "tasks_info_" + index + ".txt"; holder.put(fileName, sb.toString()); } System.out.println(holder); } 

Note. If each Task object is larger than 60000 bytes , I will immediately delete this object and move on to the next entry. But in reality this will not happen.

Update:

 public void populate(final List<Task> tasks, final long timestamp) { Map<String, String> holder = new HashMap<>(); int size = 0; int index = 0; int nl = System.getProperty("line.separator").getBytes(StandardCharsets.UTF_8).length; StringBuilder sb = new StringBuilder(); // new change sb.append(timestamp).append(System.getProperty("line.separator")); for (Task task : tasks) { int ts = String.valueOf(task).getBytes(StandardCharsets.UTF_8).length; if (size + ts + nl > 60000) { String fileName = "tasks_info_" + index + ".txt"; holder.put(fileName, sb.toString()); index++; sb = new StringBuilder(); // new change sb.append(timestamp).append(System.getProperty("line.separator")); size = 0; } sb.append(task).append(System.getProperty("line.separator")); size += ts + nl; } // for cases where we don't reach the limit if (size > 0) { // size can only be 0 if you have 0 tasks String fileName = "tasks_info_" + index + ".txt"; holder.put(fileName, sb.toString()); } System.out.println(holder); } 

Answer 1

The reason why it does not work has already been mentioned in other answers (you add after it has already exceeded the limit). But I think that not one of the implementations is still true, not only because the size of the new line has been omitted.

 public Map<String, String> populate(final List<Task> tasks) { Map<String, String> holder = new HashMap<>(); if (tasks.size() == 0) return holder; int index = 0; int nl = System.getProperty("line.separator").getBytes(StandardCharsets.UTF_8).length; StringBuilder sb = new StringBuilder(); sb.append(System.currentTimeMillis()).append(System.getProperty("line.separator")); int size = sb.toString().getBytes(StandardCharsets.UTF_8).length; for (Task task : tasks) { int ts = String.valueOf(task).getBytes(StandardCharsets.UTF_8).length; if (size + ts + nl > 60000) { String fileName = "tasks_info_" + index + ".txt"; holder.put(fileName, sb.toString()); index++; sb = new StringBuilder(); sb.append(System.currentTimeMillis()).append(System.getProperty("line.separator")); size = sb.toString().getBytes(StandardCharsets.UTF_8).length; } sb.append(task).append(System.getProperty("line.separator")); size += ts + nl; } String fileName = "tasks_info_" + index + ".txt"; holder.put(fileName, sb.toString()); return holder; } 

Answer 2

I tried your code to be sure of the solution. You need to try to check the next size before adding a new task value to SB. I changed the code to reflect this decision. Could you try this and tell us if this worked for you or not?

 public void populate(final List<Task> tasks) { Map<String, String> holder = new HashMap<>(); int size = 0; int index = 0; int lengthTask = 0; int lengthSb = 0; StringBuilder sb = new StringBuilder(); for (Task task : tasks) { lengthTask = task.toString().getBytes(StandardCharsets.UTF_8).length; lengthSb = sb.toString().getBytes(StandardCharsets.UTF_8).length; if (size + lengthTask + lengthSb > 60000) { System.out.println("last added size "+size); String fileName = "tasks_info_" + index + ".txt"; holder.put(fileName, sb.toString()); index++; sb = new StringBuilder(); size = 0; } sb.append(task).append(System.getProperty("line.separator")); size += sb.toString().getBytes(StandardCharsets.UTF_8).length; System.out.println("size "+size); } // for cases where we don't reach the limit String fileName = "tasks_info_" + index + ".txt"; holder.put(fileName, sb.toString()); System.out.println(holder); } 

Answer 3

If I understand the code correctly, I think it should be

  size = sb.toString().getBytes(StandardCharsets.UTF_8).length; 

And currently, it inserts into the card after the row has a size of more than 60,000 bytes, so the values ​​on the card have more than 60,000 bytes.

 if (size > 60000) { ... holder.put(fileName, sb.toString()); ... }