Exact conversion from character & # 8594; POSIXct-> with submillisecond dates

Question

I have a datetime column of character in a file. I upload the file (in data.table ) and do what requires the conversion of the column to POSIXct . Then I need to write the POSIXct value to the file, but the datetime will not be the same (because it is printed incorrectly).

This print / formatting issue is well known and has been discussed several times. I read a few posts describing this problem. The most authoritative answers that I found are given in response to this question . The answers to this question provide two functions ( myformat.POSIXct and form ) that should solve this problem, but they don't seem to work in this example:

 x <- "04-Jan-2013 17:22:08.139" options("digits.secs"=6) form(as.POSIXct(x,format="%d-%b-%Y %H:%M:%OS"),format="%d-%b-%Y %H:%M:%OS3") [1] "04-Jan-2013 17:22:08.138" form(as.POSIXct(x,format="%d-%b-%Y %H:%M:%OS"),format="%d-%b-%Y %H:%M:%OS4") [1] "04-Jan-2013 17:22:08.1390" myformat.POSIXct(as.POSIXct(x,format="%d-%b-%Y %H:%M:%OS"),digits=3) [1] "2013-01-04 17:22:08.138" myformat.POSIXct(as.POSIXct(x,format="%d-%b-%Y %H:%M:%OS"),digits=4) [1] "2013-01-04 17:22:08.1390" 

My sessionInfo :

 R version 2.15.2 (2012-10-26) Platform: x86_64-w64-mingw32/x64 (64-bit) locale: [1] LC_COLLATE=English_United Kingdom.1252 LC_CTYPE=English_United Kingdom.1252 [3] LC_MONETARY=English_United Kingdom.1252 LC_NUMERIC=C [5] LC_TIME=C attached base packages: [1] stats graphics grDevices datasets utils methods base other attached packages: [1] fasttime_1.0-0 data.table_1.8.9 bit64_0.9-2 bit_1.1-9 [5] sas7bdat_0.3 chron_2.3-43 vimcom_0.9-6 loaded via a namespace (and not attached): [1] tools_2.15.2 

Answer 1

As already mentioned, the answers to the questions that you have already indicated how to print / format the value do not match the actual value . This is just a print issue.

 R> as.POSIXct('2011-10-11 07:49:36.3')-as.POSIXlt('2011-10-11 07:49:36.3') Time difference of 0 secs R> as.POSIXct('2011-10-11 07:49:36.2')-as.POSIXlt('2011-10-11 07:49:36.3') Time difference of -0.0999999 secs 

Your understanding that POSIXct less accurate than POSIXlt is incorrect. You are also wrong in that you cannot include a POSIXlt object as a column in data.frame.

 R> x <- data.frame(date=Sys.time()) R> x$date <- as.POSIXlt(x$date) R> str(x) 'data.frame': 1 obs. of 1 variable: $ date: POSIXlt, format: "2013-03-13 07:38:48" 

Answer 2

So, I think you need to add a bit of fraud to my offer here: https://stackoverflow.com/a/212818/ This seems to work, but may possibly include other errors; study and think carefully about what he is doing before using anything important.

 myformat.POSIXct <- function(x, digits=0) { x2 <- round(unclass(x), digits) attributes(x2) <- attributes(x) x <- as.POSIXlt(x2) x$sec <- round(x$sec, digits) + 10^(-digits-1) format.POSIXlt(x, paste("%Y-%m-%d %H:%M:%OS",digits,sep="")) } 

Answer 3

When you write

I understand that the POSIXct view is less accurate than the POSIXlt view

you are simply mistaken.

This is the same view for both - up to a millisecond on Windows and up to (almost) microseconds on other OSs. Have you read help(DateTimeClasses) ?

As for your last question, yes, the development version of my RcppBDT package uses Boost Date.Time and can fully work in nanoseconds if your OS supports it and you enable the correct view. But it replaces POSIXct and does not yet support time object vectors.

Edit: Regarding your next question:

 R> one <- Sys.time(); two <- Sys.time(); two - one Time difference of 7.43866e-05 secs R> R> as.POSIXlt(two) - as.POSIXlt(one) Time difference of 7.43866e-05 secs R> R> one # options("digits.sec"=6) on my box [1] "2013-03-13 07:30:57.757937 CDT" R> 

Edit 2: I think you are just experiencing that the floating point representation on computers is inaccurate:

 R> print(as.numeric(as.POSIXct("04-Jan-2013 17:22:08.138", + format="%d-%b-%Y %H:%M:%OS")), digits=18) [1] 1357341728.13800001 R> print(as.numeric(as.POSIXct("04-Jan-2013 17:22:08.139", + format="%d-%b-%Y %H:%M:%OS")), digits=18) [1] 1357341728.13899994 R> 

The difference is not exactly 1/1000 as you expected.