Linux Kernel Invalidating TLB Entries

Question

In the linux kernel, I wrote code similar to copy_page_range (mm / memory.c) to copy memory from one process to another with COW optimization. Destination and source addresses can be offset by PAGE_SIZE , and COW is still working. However, I noticed that in the user program, when I copy from the same source address to different destination addresses, the TLB does not seem to be cleared correctly. At a high level, my user level code does the following (I copy exactly one page, 0x1000 bytes on my machine at a time):

src = 0x20000000

• Write to SRC (call the linked page page1 ).
• Syscall to copy the SRC to 0x30000000 in the assignment process. Now, the src process address 0x20000000 and the destination process address 0x30000000 point to the same page ( page1 ).
• Write something else for the SRC (this should cause a page error for COW processing). Suppose the source address now points to page2 .
• Syscall to copy SRC to 0x30001000 during assignment.

At this point, two separate pages must exist: SRC 0x20000000 page2 DST 0x30000000 page1 DST 0x30001000 page2

I find that in step 3, when I write something else in src 0x20000000, a page error does not occur. When checking actual page mappings: SRC 0x20000000 page1 DST 0x30000000 page1 DST 0x30001000 page1

In my code, if I call flush_tlb_page and pass in the source address, the user code works as expected with the corresponding page displays. Therefore, I am convinced that I do not support TLB correctly. In copy_page_range kernel calls mmu_notifier_invalidate_range_start/end before and after changing the page tables. I do the same and double check that I am really passing the correct struct_mm and addresses to mmu_notifier_invalidate_range_start/end . Does this function not work when flushing tlb?

So, literally, when I finished typing this, I checked dup_mmap and realized that the main caller copy_page_range , dup_mmap (kernel / fork.c) calls flush_tlb_mm . I assume that I should call flush_cache_range and flush_tlb_range before and after my kernel code. It's right? What exactly does mmu_notifier_invalidate_range_start/end do?

Answer 1

Yes, if you are doing something that changes the page tables, you need to make sure that the TLB is invalid as needed.

mmu_notifier_invalidate_range_start/end just call the MMU notification interceptors; these hooks exist only so that other kernel code can be told when TLB invalidation occurs. The only places that MMU notifiers install are

• KVM (hardware virtualization) uses them to process page swaps; he must be aware of the invalid TLB of the host in order to keep the virtual guest MMU in sync with the host.
• GRU (a specialized hardware driver in huge SGI systems) uses MMU notifiers to store mapping tables in GRU hardware in synchronization with the MMU processor.

But in any case, when you call the MMU notification hooks, you should also call the TLB drop functions, if the kernel does not already do this for you.