Configuring time synchronization
Configuring time synchronization to maintain accurate timekeeping across network devices
Abstract
Providing feedback on Red Hat documentation Copy linkLink copied to clipboard!
We appreciate your feedback on our documentation. Let us know how we can improve it.
Submitting feedback through Jira (account required)
- Log in to the Jira website.
- Click Create in the top navigation bar.
- Enter a descriptive title in the Summary field.
- Enter your suggestion for improvement in the Description field. Include links to the relevant parts of the documentation.
- Click Create at the bottom of the dialogue.
Chapter 1. Introduction to chrony suite Copy linkLink copied to clipboard!
The implementation of the Network Time Protocol (NTP) is chrony. You can use chrony:
-
To synchronize the system clock with
NTPservers - To synchronize the system clock with a reference clock, for example a GPS receiver
- To synchronize the system clock with a manual time input
-
As an
NTPv4(RFC 5905)server or peer to provide a time service to other computers in the network
chrony performs well in a wide range of conditions:
- Including network connections
- Heavily congested networks
- Changing temperatures (ordinary computer clocks are sensitive to temperature)
- Systems that do not run continuously, or run on a virtual machine.
chrony consists of chronyd, a daemon that runs in user space, and chronyc, a command line program which can be used to monitor the performance of chronyd and to change various operating parameters when it is running.
The command line utility chronyc can monitor and control the chronyd daemon. This utility provides a command prompt which lets you enter a number of commands to query the current state of chronyd and make changes to its configuration. By default, chronyd accepts only commands from a local instance of chronyc, but it can be configured to accept monitoring commands also from remote hosts. The remote access should be restricted.
Chapter 2. Using chrony Copy linkLink copied to clipboard!
Learn how to start and stop chronyd, check it is synchronized, and manually adjust the system clock.
2.1. Managing chrony Copy linkLink copied to clipboard!
The chrony suite is installed by default on Red Hat Enterprise Linux. You can start, stop, and check the status of chronyd.
Procedure
Optional: Check if the chrony suite is installed by running the following command as
root:dnf install chrony
# dnf install chronyCopy to Clipboard Copied! Toggle word wrap Toggle overflow The default location for the chrony daemon is
/usr/sbin/chronyd. The command line utility will be installed to/usr/bin/chronyc.To check the status of
chronyd, issue the following command:systemctl status chronyd chronyd.service - NTP client/server Loaded: loaded (/usr/lib/systemd/system/chronyd.service; enabled) Active: active (running) since Wed 2013-06-12 22:23:16 CEST; 11h ago
$ systemctl status chronyd chronyd.service - NTP client/server Loaded: loaded (/usr/lib/systemd/system/chronyd.service; enabled) Active: active (running) since Wed 2013-06-12 22:23:16 CEST; 11h agoCopy to Clipboard Copied! Toggle word wrap Toggle overflow To start
chronyd, issue the following command asroot:systemctl start chronyd
# systemctl start chronydCopy to Clipboard Copied! Toggle word wrap Toggle overflow To ensure
chronydstarts automatically at system start, issue the following command asroot:systemctl enable chronyd
# systemctl enable chronydCopy to Clipboard Copied! Toggle word wrap Toggle overflow To stop
chronyd, issue the following command asroot:systemctl stop chronyd
# systemctl stop chronydCopy to Clipboard Copied! Toggle word wrap Toggle overflow To prevent
chronydfrom starting automatically at system start, issue the following command asroot:systemctl disable chronyd
# systemctl disable chronydCopy to Clipboard Copied! Toggle word wrap Toggle overflow
2.2. Manually adjusting the system clock Copy linkLink copied to clipboard!
You can manually adjust the system clock.
Procedure
To step the system clock immediately, by passing any adjustments in progress by slewing, enter:
chronyc makestep
# chronyc makestepCopy to Clipboard Copied! Toggle word wrap Toggle overflow ImportantIf the
rtcfiledirective is used, the real-time clock should not be manually adjusted. Random adjustments would interfere with chrony's need to measure the rate at which the real-time clock drifts.
2.3. Disabling a NetworkManager dispatcher script Copy linkLink copied to clipboard!
The chrony dispatcher script manages the online and offline state of the NTP servers. As a system administrator, you can disable the dispatcher script to keep chronyd polling the servers constantly.
The NetworkManager executes the chrony dispatcher script during interface reconfiguration, stop or start operations. However, if you configure certain interfaces or routes outside of NetworkManager, you can encounter the following situation:
- The dispatcher script might run when no route to the NTP servers exists, causing the NTP servers to switch to the offline state.
- If you establish the route later, the script does not run again by default, and the NTP servers remain in the offline state.
To ensure that chronyd can synchronize with your NTP servers, which have separately managed interfaces, disable the dispatcher script.
Procedure
To disable the
chronydispatcher script, create a symlink to/dev/null:ln -f -s /dev/null /etc/NetworkManager/dispatcher.d/20-chrony-onoffline
# ln -f -s /dev/null /etc/NetworkManager/dispatcher.d/20-chrony-onofflineCopy to Clipboard Copied! Toggle word wrap Toggle overflow NoteAfter this change, the NTP servers remain in the online state at all times.
2.4. Setting up chrony in an isolated network Copy linkLink copied to clipboard!
For a network that is never connected to the internet, one computer is selected to be the primary timeserver. The other computers are either direct clients of the server, or clients of clients. On the server, the drift file must be manually set with the average rate of drift of the system clock. If the server is rebooted, it will obtain the time from surrounding systems and calculate an average to set its system clock. Thereafter it resumes applying adjustments based on the drift file. The drift file will be updated automatically when the settime command is used.
To set up chrony for a system in an isolated network, follow the steps mentioned below:
Procedure
On the system selected to be the server, edit
/etc/chrony.confas follows:Copy to Clipboard Copied! Toggle word wrap Toggle overflow Where
<subnet>is the network from which the clients are allowed to connect. Use Classless Inter-Domain Routing (CIDR) notation to specify the subnet.On the systems selected to be direct clients of the server, edit the
/etc/chrony.confas follows:Copy to Clipboard Copied! Toggle word wrap Toggle overflow Where
<server_fqdn>is the host name of the server, and<server_ip_address>is the address of the server . Clients with this configuration will resynchronize with the server if it restarts.On the client systems which are not to be direct clients of the server, the
/etc/chrony.conffile should be the same except that thelocalandallowdirectives should be omitted.In an isolated network, you can also use the
localdirective that enables a local reference mode, which allowschronydoperating as anNTPserver to appear synchronized to real time, even when it was never synchronized or the last update of the clock happened a long time ago.To allow multiple servers in the network to use the same local configuration and to be synchronized to one another without confusing clients that poll more than one server, use the
orphanoption of thelocaldirective which enables the orphan mode. Each server needs to be configured to poll all other servers withlocal. This ensures that only the server with the smallest reference ID has the local reference active and other servers are synchronized to it. When the server fails, another one takes over.
2.5. Configuring remote monitoring access Copy linkLink copied to clipboard!
The chronyc utility can access chronyd by using the following methods:
- IPv4 or IPv6.
-
A domain socket, which is accessible locally by the
rootandchronyuser.
By default, chronyc connects to the Unix domain socket. The default path is /var/run/chrony/chronyd.sock. If this connection fails, chronyc tries to connect to 127.0.0.1 and then ::1.
Only the following monitoring commands, which do not affect the behavior of chronyd, are allowed from the network:
- activity
- manual list
- rtcdata
- smoothing
- sources
- sourcestats
- tracking
- waitsync
By default, the commands are accepted only from localhost (127.0.0.1 or ::1).
All other commands are allowed only through the Unix domain socket. When sent over the network, chronyd responds with a Not authorised error, even if it is from localhost.
The following procedure describes how to access chronyd remotely with chronyc.
Procedure
Configure
chronyto listen on local interface by adding the following to the/etc/chrony.conffile:bindcmdaddress 0.0.0.0
bindcmdaddress 0.0.0.0Copy to Clipboard Copied! Toggle word wrap Toggle overflow and
bindcmdaddress ::
bindcmdaddress ::Copy to Clipboard Copied! Toggle word wrap Toggle overflow Add the following content to the
/etc/chrony.conffile to allow commands from remote IP addresses, networks, and subnet:cmdallow 192.168.1.0/24
cmdallow 192.168.1.0/24Copy to Clipboard Copied! Toggle word wrap Toggle overflow cmdallow 2001:db8::/64
cmdallow 2001:db8::/64Copy to Clipboard Copied! Toggle word wrap Toggle overflow Open port 323 in the firewall to allow connections from remote systems:
firewall-cmd --permanent --add-port=323/udp
# firewall-cmd --permanent --add-port=323/udpCopy to Clipboard Copied! Toggle word wrap Toggle overflow Reload the firewall configuration:
firewall-cmd --reload
# firewall-cmd --reloadCopy to Clipboard Copied! Toggle word wrap Toggle overflow For more information, see the
chrony.conf(5)man page on your system.
2.6. Checking if chrony is synchronized Copy linkLink copied to clipboard!
You can check if chrony is synchronized with the use of the tracking, sources, and sourcestats commands.
Procedure
To check chrony tracking, enter:
Copy to Clipboard Copied! Toggle word wrap Toggle overflow The
chronycsources command displays information about the current time sources thatchronydis accessing.Copy to Clipboard Copied! Toggle word wrap Toggle overflow You can specify the optional
-vargument to print more verbose information. In this case, extra caption lines are shown as a reminder of the meanings of the columns.The
sourcestatscommand displays information about the drift rate and offset estimation process for each of the sources currently being examined bychronyd. To check chrony source statistics, enter the following command:chronyc sourcestats 210 Number of sources = 1 Name/IP Address NP NR Span Frequency Freq Skew Offset Std Dev =============================================================================== abc.def.ghi 11 5 46m -0.001 0.045 1us 25us
$ chronyc sourcestats 210 Number of sources = 1 Name/IP Address NP NR Span Frequency Freq Skew Offset Std Dev =============================================================================== abc.def.ghi 11 5 46m -0.001 0.045 1us 25usCopy to Clipboard Copied! Toggle word wrap Toggle overflow The optional argument
-vcan be specified, meaning verbose. In this case, extra caption lines are shown as a reminder of the meanings of the columns. For more information, see thechronyc(1)man page on your system.
2.7. Monitoring NTP clients Copy linkLink copied to clipboard!
Display the list of clients that accessed the chronyd Network Time Protocol (NTP) server. This information is useful for verifying which clients are actively synchronizing and checking their connection history.
Prerequisites
-
The
chronydservice is running. -
chronydis configured to allow NTP clients. For example, add theallowdirective to/etc/chrony.conf.
Procedure
Display a list of clients by using the
chronyc clientscommand:chronyc clients
# chronyc clientsCopy to Clipboard Copied! Toggle word wrap Toggle overflow Review the command output:
Hostname NTP Drop Int IntL Last Cmd Drop Int Last ==================================================================================== client1.example.net 10 0 6 - 10s 0 0 6 12s client2.example.net 8 0 6 - 25s 0 0 6 28s desktop-192-0-2-100.example.net 3 6 - 9s 0 0 6 11s
Hostname NTP Drop Int IntL Last Cmd Drop Int Last ==================================================================================== client1.example.net 10 0 6 - 10s 0 0 6 12s client2.example.net 8 0 6 - 25s 0 0 6 28s desktop-192-0-2-100.example.net 3 6 - 9s 0 0 6 11sCopy to Clipboard Copied! Toggle word wrap Toggle overflow The output fields include:
- Hostname: The hostname or IP address of the client.
- NTP: The number of NTP packets received from the client.
-
Drop: The number of packets dropped by
chronyd, typically due to rate-limiting. - Int: The client’s polling interval in seconds (as a log base 2 value, e.g., 6 = $2^6$ = 64s).
-
IntL: The average inter-poll interval (in seconds) for the client when rate limiting is active. This field shows
'-'if rate limiting is not currently active for the client. - Last: The elapsed time since the last NTP packet was received from this client. Use this value to determine which clients are synchronized or recently synchronized.
Cmd: The number of
chronyc(monitoring) command packets received from the client.NoteIf you have a number of clients and the command is slow to return, it might be due to performing DNS lookups. Use the
-noption to disable hostname resolution and display IP addresses instead:# chronyc -n clientsImportantBy default,
chronydmonitors up to 4,096 clients. If you have more clients than this, you can increase the memory limit withclientloglimitin/etc/chrony.conffile. For example:clientloglimit 10000000.For more information, see the
chronyc(1)andchronyd(8)man pages on your system.
Chapter 3. Chrony with hardware timestamping Copy linkLink copied to clipboard!
Hardware (HW) timestamping in some Network Interface Controller (NICs) provides accurate timestamping of incoming and outgoing packets. NTP timestamps are usually created by the kernel and chronyd with the use of the system clock. However, when HW timestamping is enabled, the NIC uses its own clock to generate the timestamps when packets are entering or leaving the link layer or the physical layer. When used with NTP, hardware timestamping can significantly improve the accuracy of synchronization. For best accuracy, both NTP servers and NTP clients need to use hardware timestamping. Under ideal conditions, a sub-microsecond accuracy might be possible.
Another protocol for time synchronization that uses hardware timestamping is PTP.
Unlike NTP, PTP relies on assistance in network switches and routers. If you want to achieve the best accuracy of synchronization, use PTP on networks that have switches and routers with PTP support, and prefer NTP on networks that do not have such switches and routers.
3.1. Verifying support for hardware timestamping Copy linkLink copied to clipboard!
To verify that hardware timestamping with NTP is supported by an interface, use the ethtool -T command. An interface can be used for hardware timestamping with NTP if ethtool lists the SOF_TIMESTAMPING_TX_HARDWARE and SOF_TIMESTAMPING_TX_SOFTWARE capabilities and also the HWTSTAMP_FILTER_ALL filter mode.
Procedure
Display a device’s time stamping capabilities and associated PTP hardware clock:
ethtool -T enp1s0
# ethtool -T enp1s0Copy to Clipboard Copied! Toggle word wrap Toggle overflow
3.2. Enabling hardware timestamping Copy linkLink copied to clipboard!
You can enable the hardware timestamping on one or multiple interfaces by using the hwtimestamp directive in the /etc/chrony.conf file. The directive can either specify a single interface, or a wildcard character can be used to enable hardware timestamping on all interfaces that support it.
Procedure
Edit the
/etc/chrony.conffile and make the following changes:Add the
hwtimestampsetting for interfaces which support hardware timestamping. For example:hwtimestamp enp1s0 hwtimestamp eno*
hwtimestamp enp1s0 hwtimestamp eno*Copy to Clipboard Copied! Toggle word wrap Toggle overflow You can use the * wildcard if no other application, such as
ptp4luses hardware timestamping.Configure a short client polling interval by appending the
minpollandmaxpolloptions to the server setting, for example:server ntp.example.comlocal minpoll 0 maxpoll 0
server ntp.example.comlocal minpoll 0 maxpoll 0Copy to Clipboard Copied! Toggle word wrap Toggle overflow For hardware timestamping, you must configure a shorter polling interval than the default range (64-1024 seconds) to minimize the offset of the system clock.
Enable the NTP interleaved mode by appending the
xleaveoption to the server setting:server ntp.example.comlocal minpoll 0 maxpoll 0 xleave
server ntp.example.comlocal minpoll 0 maxpoll 0 xleaveCopy to Clipboard Copied! Toggle word wrap Toggle overflow With this setting, chrony gets the hardware transmit timestamp only after sending a packet. This behavior prevents the serever from saving the timestamp in packets to which it responds. With the
xleaveoption, chrony can receive transmit timestamps that were generated after the transmission.Optional: Increase the maximum size of memory allocated for logging of client’s access on the server, for example:
clientloglimit 100000000
clientloglimit 100000000Copy to Clipboard Copied! Toggle word wrap Toggle overflow The default server configuration allows a few thousands of clients to use the interleaved mode concurrently. By increasing the value of the
clientloglimitsetting, you can configure the server for a large number of clients.
Restart the chronyd service:
systemctl restart chronyd
# systemctl restart chronydCopy to Clipboard Copied! Toggle word wrap Toggle overflow
Verification
Optional: Verify in the
/var/log/messageslog file that hardware timesamping is enabled:chronyd[4081]: Enabled HW timestamping on enp1s0 chronyd[4081]: Enabled HW timestamping on eno1
chronyd[4081]: Enabled HW timestamping on enp1s0 chronyd[4081]: Enabled HW timestamping on eno1Copy to Clipboard Copied! Toggle word wrap Toggle overflow If chronyd is configured as an NTP client or peer, display the transmit and receive timestamping modes and the interleaved mode:
Copy to Clipboard Copied! Toggle word wrap Toggle overflow Report the stability of NTP measurements:
chronyc sourcestats 210 Number of sources = 1 Name/IP Address NP NR Span Frequency Freq Skew Offset Std Dev ntp.local 12 7 11 +0.000 0.019 +0ns 49ns
# chronyc sourcestats 210 Number of sources = 1 Name/IP Address NP NR Span Frequency Freq Skew Offset Std Dev ntp.local 12 7 11 +0.000 0.019 +0ns 49nsCopy to Clipboard Copied! Toggle word wrap Toggle overflow This stability is reported in the
Std Devcolumn. With hardware timestamping enabled, stability of NTP measurements should be in tens or hundreds of nanoseconds, under normal load.
3.3. Configuring PTP-NTP bridge Copy linkLink copied to clipboard!
If a highly accurate Precision Time Protocol (PTP) primary timeserver is available in a network that does not have switches or routers with PTP support, a computer may be dedicated to operate as a PTP client and a stratum-1 NTP server. Such a computer needs to have two or more network interfaces, and be close to the primary timeserver or have a direct connection to it. This will ensure highly accurate synchronization in the network.
Procedure
-
Configure the ptp4l and phc2sys programs from the
linuxptppackages to use one interface to synchronize the system clock by usingPTP. Configure
chronydto provide the system time by using the other interface:bindaddress 203.0.113.74 hwtimestamp enp1s0 local stratum 1
bindaddress 203.0.113.74 hwtimestamp enp1s0 local stratum 1Copy to Clipboard Copied! Toggle word wrap Toggle overflow Restart the
chronydservice:systemctl restart chronyd
# systemctl restart chronydCopy to Clipboard Copied! Toggle word wrap Toggle overflow
Chapter 4. Overview of Network Time Security (NTS) in chrony Copy linkLink copied to clipboard!
Network Time Security (NTS) is an authentication mechanism for Network Time Protocol (NTP), designed to scale substantial clients. It verifies that the packets received from the server machines are unaltered while moving to the client machine. Network Time Security (NTS) includes a Key Establishment (NTS-KE) protocol that automatically creates the encryption keys used between the server and its clients.
NTS is not compatible with the FIPS and OSPP profile. When you enable the FIPS and OSPP profile, chronyd that is configured with NTS can abort with a fatal message. You can disable the OSPP profile and FIPS mode for chronyd service by adding the GNUTLS_FORCE_FIPS_MODE=0 setting to the /etc/sysconfig/chronyd file.
4.1. Enabling Network Time Security (NTS) on a client Copy linkLink copied to clipboard!
By default, Network Time Security (NTS) is not enabled. You can enable NTS in the /etc/chrony.conf file.
Prerequisites
- The time server supports NTS.
Procedure
-
Edit the
/etc/crony.conffile to make the following changes: Specify the server with the
ntsoption in addition to theiburstoption.For example: server time.example.com iburst nts server nts.netnod.se iburst nts server ptbtime1.ptb.de iburst nts
For example: server time.example.com iburst nts server nts.netnod.se iburst nts server ptbtime1.ptb.de iburst ntsCopy to Clipboard Copied! Toggle word wrap Toggle overflow Add the following setting to avoid repeating the Network Time Security-Key Establishment (NTS-KE) session during system boot:
ntsdumpdir /var/lib/chrony
ntsdumpdir /var/lib/chronyCopy to Clipboard Copied! Toggle word wrap Toggle overflow If present, comment out or remove the following setting to disable synchronization with Network Time Protocol (NTP) servers provided by
DHCP:sourcedir /run/chrony-dhcp
sourcedir /run/chrony-dhcpCopy to Clipboard Copied! Toggle word wrap Toggle overflow Restart the
chronydservice:systemctl restart chronyd
systemctl restart chronydCopy to Clipboard Copied! Toggle word wrap Toggle overflow
Verification
Verify if the
NTSkeys were successfully established:Copy to Clipboard Copied! Toggle word wrap Toggle overflow The
KeyID,Type, andKLenshould have non-zero values. If the value is zero, check the system log for error messages fromchronyd.Verify the client is making NTP measurements:
Copy to Clipboard Copied! Toggle word wrap Toggle overflow The
Reachcolumn should have a non-zero value; ideally 377. If the value rarely gets 377 or never gets to 377, it indicates that NTP requests or responses are getting lost in the network.For more details, see the
chrony.conf(5)man page on your system.
4.2. Enabling Network Time Security (NTS) on a time server Copy linkLink copied to clipboard!
If you run your own Network Time Protocol (NTP) server, you can enable the server Network Time Security (NTS) support to facilitate its clients to synchronize securely.
If the NTP server is a client of other servers, that is, it is not a Stratum 1 server, it should use NTS or symmetric key for its synchronization.
Prerequisites
-
Server private key in
PEMformat -
Server certificate with required intermediate certificates in
PEMformat
Procedure
Edit the
/etc/chrony.conffile, and make the following changes:ntsserverkey /etc/pki/tls/private/<ntp-server.example.net>.key ntsservercert /etc/pki/tls/certs/<ntp-server.example.net>.crt
ntsserverkey /etc/pki/tls/private/<ntp-server.example.net>.key ntsservercert /etc/pki/tls/certs/<ntp-server.example.net>.crtCopy to Clipboard Copied! Toggle word wrap Toggle overflow Set permissions on both the private key and the certificate file that allow the chrony user to read the files, for example
chown root:chrony /etc/pki/tls/private/<ntp-server.example.net>.key /etc/pki/tls/certs/<ntp-server.example.net>.crt chmod 644 /etc/pki/tls/private/<ntp-server.example.net>.key /etc/pki/tls/certs/<ntp-server.example.net>.crt
# chown root:chrony /etc/pki/tls/private/<ntp-server.example.net>.key /etc/pki/tls/certs/<ntp-server.example.net>.crt # chmod 644 /etc/pki/tls/private/<ntp-server.example.net>.key /etc/pki/tls/certs/<ntp-server.example.net>.crtCopy to Clipboard Copied! Toggle word wrap Toggle overflow -
Ensure that the
ntsdumpdir /var/lib/chronysetting is present. Open the required ports in firewalld:
firewall-cmd --permanent --add-port={323/udp,4460/tcp} firewall-cmd --reload# firewall-cmd --permanent --add-port={323/udp,4460/tcp} # firewall-cmd --reloadCopy to Clipboard Copied! Toggle word wrap Toggle overflow Restart the
chronydservice:systemctl restart chronyd
# systemctl restart chronydCopy to Clipboard Copied! Toggle word wrap Toggle overflow
Verification
Perform a test from a client machine:
Copy to Clipboard Copied! Toggle word wrap Toggle overflow The
System clock wrongmessage indicates the NTP server is accepting NTS-KE connections and responding with NTS-protected NTP messages.Verify the NTS-KE connections and authenticated NTP packets observed on the server:
Copy to Clipboard Copied! Toggle word wrap Toggle overflow If the value of the
NTS-KE connections acceptedandAuthenticated NTP packetsfield is a non-zero value, it means that at least one client was able to connect to the NTS-KE port and send an authenticated NTP request.