bug#21912: TAI<->UTC conversion botches the unknown

[Zefram]

Probing the existence of leap seconds on particular days, via srfi-19's
TAI-to-UTC conversion. The methodology here is to take noon UT on the
day of interest, convert to TAI, add 86400 seconds, then convert to UTC
and display. The resulting time of day is 11:59:59 if there is a leap
second that day, and 12:00:00 if there is not.

scheme@(guile-user)> (use-modules (srfi srfi-19))
scheme@(guile-user)> (date->string (time-tai->date (add-duration (julian-day->time-tai 2455743) (make-time time-duration 0 86400)) 0) "~4")
$1 = "2011-07-01T12:00:00Z"
scheme@(guile-user)> (date->string (time-tai->date (add-duration (julian-day->time-tai 2456109) (make-time time-duration 0 86400)) 0) "~4")
$2 = "2012-07-01T11:59:59Z"
scheme@(guile-user)> (date->string (time-tai->date (add-duration (julian-day->time-tai 2457204) (make-time time-duration 0 86400)) 0) "~4")
$3 = "2015-07-01T12:00:00Z"
scheme@(guile-user)> (date->string (time-tai->date (add-duration (julian-day->time-tai 2457935) (make-time time-duration 0 86400)) 0) "~4")
$4 = "2017-07-01T12:00:00Z"

For 2011-06-30 it is correct that there was not a leap second, and for
2012-06-30 it is correct that there was. But for 2015-06-30 it says
there was not a leap second, when in fact there was. For 2017-06-30
it says there will not be a leap second, when in fact it is not yet
determined whether there will be.

Really both of these errors come from the same cause. At the time
this Guile 2.0.11 was released, the leap status of 2015-06-30 had not
yet been determined. Both 2015 and 2017 fall within the future period
beyond the scope of this Guile's static leap second knowledge.

The bug is not that Guile doesn't know that there was a leap second
in 2015. As the 2017 case illustrates, it's impossible for it to
know all the leap second scheduling about which it can be asked.
The bug is that Guile *thinks* it knows about all future leap seconds.
It specifically thinks that there will be no leaps at all beyond the
historically-scheduled ones that it knows about.

Guile ought to be aware of how far its leap table extends, and signal
an error when asked to perform a TAI<->UTC conversion that falls outside
its scope.

-zefram

[Mark H Weaver]

Post by Zefram
Guile ought to be aware of how far its leap table extends, and signal
an error when asked to perform a TAI<->UTC conversion that falls outside
its scope.

I sympathize with your preference to raise an error rather than return
invalid results, but unfortunately I don't think this proposal is really
practical, given the SRFI-19 API. The main problem is that, although
it's not specified in the document, the SRFI-19 reference implementation
uses UTC for all julian days, modified julian days, and dates. For
example, 'time-tai->date' and 'time-tai->julian-day' include an implicit
conversion to UTC.

As a result, if we were to adopt your proposal, it would never be
possible to print a date more than 6 months in the future, and for users
of stable distributions such as Debian, it is not be possible to print
the current date.

We need a way to print future dates. In theory, we could print future
dates in TAI without the leap second table, using the convention of
printing TAI dates as in <https://cr.yp.to/proto/utctai.html>, but
that's not what SRFI-19 does, and it has no API to support that way of
printing TAI dates.

Also, it's debatable whether it's desirable to print future dates in TAI
this way, since in the far future the times will drift far away from
mean solar time, and the calendar will eventually drift from the usual
alignment of the solar year to the months on the calendar.

So, I think this is a valid issue, but I don't see how it can be
addressed within SRFI-19. I would need to be addressed in a new SRFI,
and even then it's not clear to me what's the right way forward.

What do you think?

Regards,
Mark

[John Cowan]

What is more, there are no TAI<->UTC conversion tables from before 1961
(when UTC began) and probably never will be. There was heated debate in
the R7RS-small working group, and we finally settled on a compromise:

(current-second) [r]eturns an inexact number representing the current time
on the International Atomic Time (TAI) scale. The value 0.0 represents
midnight on January 1, 1970 TAI (equivalent to ten seconds before midnight
Universal Time) and the value 1.0 represents one TAI second later. Neither
high accuracy nor high precision are required; in particular, returning
Coordinated Universal Time plus a suitable constant might be the best an
implementation can do.

I now see that the "ten seconds" is incorrect, and I am filing an erratum:
the correct figure is 4.2131700 seconds per <
http://maia.usno.navy.mil/ser7/tai-utc.dat>.

Post by Mark H Weaver

Post by Zefram
Guile ought to be aware of how far its leap table extends, and signal
an error when asked to perform a TAI<->UTC conversion that falls outside
its scope.

I sympathize with your preference to raise an error rather than return
invalid results, but unfortunately I don't think this proposal is really
practical, given the SRFI-19 API. The main problem is that, although
it's not specified in the document, the SRFI-19 reference implementation
uses UTC for all julian days, modified julian days, and dates. For
example, 'time-tai->date' and 'time-tai->julian-day' include an implicit
conversion to UTC.
As a result, if we were to adopt your proposal, it would never be
possible to print a date more than 6 months in the future, and for users
of stable distributions such as Debian, it is not be possible to print
the current date.
We need a way to print future dates. In theory, we could print future
dates in TAI without the leap second table, using the convention of
printing TAI dates as in <https://cr.yp.to/proto/utctai.html>, but
that's not what SRFI-19 does, and it has no API to support that way of
printing TAI dates.
Also, it's debatable whether it's desirable to print future dates in TAI
this way, since in the far future the times will drift far away from
mean solar time, and the calendar will eventually drift from the usual
alignment of the solar year to the months on the calendar.
So, I think this is a valid issue, but I don't see how it can be
addressed within SRFI-19. I would need to be addressed in a new SRFI,
and even then it's not clear to me what's the right way forward.
What do you think?
Regards,
Mark

[Mark H Weaver]

Hi John,

Post by John Cowan
What is more, there are no TAI<->UTC conversion tables from before
1961 (when UTC began) and probably never will be. There was heated
debate in the R7RS-small working group, and we finally settled on a
(current-second) [r]eturns an inexact number representing the current
time on the International Atomic Time (TAI) scale. The value 0.0
represents midnight on January 1, 1970 TAI (equivalent to ten seconds
before midnight Universal Time) and the value 1.0 represents one TAI
second later. Neither high accuracy nor high precision are required;
in particular, returning Coordinated Universal Time plus a suitable
constant might be the best an implementation can do.
I now see that the "ten seconds" is incorrect, and I am filing an
erratum: the correct figure is 4.2131700 seconds per
<http://maia.usno.navy.mil/ser7/tai-utc.dat>.

Actually, the correct TAI-UTC delta on January 1, 1970 TAI is
approximately 8 seconds. You are misinterpreting the relevant line from
that file:

1968 FEB 1 =JD 2439887.5 TAI-UTC= 4.2131700 S + (MJD - 39126.) X 0.002592 S

This means that TAI-UTC = 4.2131700 + (MJD - 39126) * 0.002592, where
MJD is the modified julian day of the desired TAI-UTC delta. In this
case, the MJD (modified julian day) of midnight UTC January 1, 1970 is
40587, and plugging that into the equation above yields TAI-UTC =
8.000082 seconds.

If you are doubtful, see the "Atomic Time and Leap Seconds" graph, which
shows a graph of TAI-UTC over the years 1958 to 2017, on the following page:

http://jjy.nict.go.jp/mission/page1-e.html

Note that between January 1961 and January 1972, the TAI-UTC delta was
not an integer, and instead of the discontinuous leap second jumps that
we've had since 1972, the two clocks ran at different rates in those
years.

Mark

[John Cowan]

Post by Mark H Weaver
This means that TAI-UTC = 4.2131700 + (MJD - 39126) * 0.002592, where
MJD is the modified julian day of the desired TAI-UTC delta. In this
case, the MJD (modified julian day) of midnight UTC January 1, 1970 is
40587, and plugging that into the equation above yields TAI-UTC =
8.000082 seconds.

Thanks. I have changed the errata list, the R7RS source (in the "errata"
branch), and the cached copy of r7rs.pdf.