Clojure - Try not to hold on to your head?

Question

I am new to Clojure and don't have much Java experience.

I am working on a system that is built around functions that turn one or more input lazy sequences into a new lazy sequence. For example, a function that takes two sequences and creates a lazy diff sequence between inputs.

In test cases, do I verify that the sequences are implemented? at the right time, but I can’t figure out how to verify that I didn’t “keep my head” somewhere except to work on a huge sequence and wait for OutOfMemoryError. I don’t care about this approach, since the “huge sequence” is arbitrary and the tests take a lot of time.

I was looking for information about the Java Garbage collector, but could not find what I was looking for (or did not understand).

Suggestions?

Answer 1

One possible way is to save the WeakReference in seq, and then at some point later, after all the seq links had to be dropped, forced garbage collection, dereferenced WeakReference with .get and saw that it cleared. For instance:

 (let [r (atom nil) s (atom nil)] (let [x (repeat 1000 1)] (reset! r (java.lang.ref.WeakReference. x)) (reset! s (map (partial * 2) x))) ;; the lazy seq created by map has not yet been realized. (is (false? (realized? @s))) ;; Request garbage collection. (System/gc) ;; since it not realized, the result from map still has a reference ;; to the head of its input seq. (is (false? (nil? (.get @r)))) ;; Realize the seq returned from map. (dorun @s) (is (realized? @s)) ;; Request garbage collection again. (System/gc) ;; Once the result from map has been realized, it should discard its reference ;; to the head of the input seq. (is (nil? (.get @r)))) 

Be careful, however, testing anything related to garbage collection and memory allocation in Java is an inaccurate science because this stuff is managed asynchronously by the JVM. In particular, System.gc() only assumes that the JVM does garbage collection; There is no guarantee that a weak link will be cleared when this method returns, so this test is prone to false crashes. For this reason, many style checkers will mark System.gc() calls as "voodoo".

In general, you are best off writing solid code and debugging a profiler, such as VisualVM , only when it becomes apparent that you have a memory leak.

Answer 2

The “proper” way to control access to GC objects is to use some reference type java.lang.ref , preferably PhantomReference , together with a ReferenceQueue .

PhantomReference differs from other java.lang.ref classes in that they cannot be used to get the link target; they are useful with ReferenceQueue s. In addition, objects are considered phantom accessible only after they have been completed.

The template looks like this:

 (def rq (java.lang.ref.ReferenceQueue.)) (def pref (java.lang.ref.PhantomReference. [1 2] rq)) (future (.remove rq 0) (println :done)) (System/gc) ;;=> :done 

The last line displays a printout from the future.

I have a small library that is supposed to handle the template, but, alas, it seems that it needs some maintenance work ... I will try to figure it out soon and publish the link when it is done.

Answer 3

(.totalMemory (Runtime/getRuntime)) will track the total memory size used by jvm.

Checking the allocated memory, storing the value, running your code in a sequence of several thousand elements, and then checking the allocated memory again should get what you want.

Perhaps you can reduce the default size and granularity to check for increased memory usage in unit tests.