Let's say we have this file test.xlsx:

Using openxlsx, we can read the book as a dataframe object to retrieve only data:
library(openxlsx)
df1 <- read.xlsx("test.xlsx")
df1
If we need to extract comments, we need to read as an object of the book:
wb <- loadWorkbook("test.xlsx")
Check the structure of the object to see where the cell reference is stored:
str(wb$comments)
# List of 3
# $ :List of 2
# ..$ :List of 5
# .. ..$ ref : chr "B2"
# ...
# ... etc
, :
lapply(wb$comments, function(sheet)
sapply(sheet, "[[", "ref"))
, 3 , 1- 2 B2 B5, 2 .