109.1 Fundamentals of internet protocols¶
Weight: 4
Candidates should demonstrate a proper understanding of TCP/IP network fundamentals.
Objectives
- Demonstrate an understanding of network masks and CIDR notation.
- Knowledge of the differences between private and public "dotted quad" IP addresses.
- Knowledge about common TCP and UDP ports and services (20, 21, 22, 23, 25, 53, 80, 110, 123, 139, 143, 161, 162, 389, 443, 465, 514, 636, 993, 995).
- Knowledge about the differences and major features of UDP, TCP and ICMP.
- Knowledge of the major differences between IPv4 and IPv6.
- Knowledge of the basic features of IPv6.
Terms
/etc/services, IPv4, IPv6, Subnetting, TCP, UDP, ICMP
TCP/IP¶
TCP/IP (Transmission Control Protocol/Internet Protocol) is the stack of protocols that lets computers talk to each other, on the internet and on most other networks. Despite the name, it is not just TCP and IP: it includes UDP, ICMP, DNS, SMTP, ARP and many more, each doing one job.
services SSH, HTTP, DNS, SMTP ... "what do we say?"
|
transport TCP or UDP + ports "which program on that machine?"
|
network IP (+ ICMP) addresses "which machine?"
IP addresses¶
The Internet Protocol (IP) gives every device a logical address so that packets can be sent to it. Each address must be unique on its network, so the network knows where each packet goes. One device can have more than one address. Two versions are in use today: IPv4 and IPv6.
An IPv4 address is 32 bits, split into 4 groups of 8 bits called octets. It is written as four decimal numbers separated by dots, the dotted quad A.B.C.D, like 1.1.1.1:
Each octet goes from 0 to 255 (11111111 in binary). So 1.2.3.4, 100.0.0.100 and 192.168.1.4 are all valid. In total that makes 256 × 256 × 256 × 256 = about 4.3 billion addresses, which is not enough for every device on Earth. That shortage is why private addresses, NAT and IPv6 exist.
Address classes¶
In theory, IPv4 addresses are divided into classes by their first octet. Each class has a default netmask:
| Class | First octet | Range | Default mask | Example |
|---|---|---|---|---|
| A | 1 to 126 | 1.0.0.0 to 126.255.255.255 | 255.0.0.0 (/8) |
10.25.13.10 |
| B | 128 to 191 | 128.0.0.0 to 191.255.255.255 | 255.255.0.0 (/16) |
141.150.200.1 |
| C | 192 to 223 | 192.0.0.0 to 223.255.255.255 | 255.255.255.0 (/24) |
200.178.12.242 |
127 is missing on purpose: addresses starting with 127 are the loopback, the machine talking to itself (127.0.0.1). The whole 127.0.0.0/8 range is loopback, and 224.0.0.0 and up is multicast, not assigned to individual hosts. The default masks are only defaults; as you will see, any mask can be used with any address.
Public and private addresses¶
Public addresses are handed out by IANA (the Internet Assigned Numbers Authority), through regional registries and your provider. They are unique on the whole internet and routed across it.
Private addresses are reserved for internal networks: homes, offices, schools. They are never routed on the internet, so anyone can use them inside their own network, and the same private address is reused in millions of networks. Inside one network they must still be unique.
| Class | Private range | Size |
|---|---|---|
| A | 10.0.0.0 to 10.255.255.255 | about 16 million addresses |
| B | 172.16.0.0 to 172.31.255.255 | about 1 million |
| C | 192.168.0.0 to 192.168.255.255 | about 65 thousand |
In short: 10/8, 172.16-31, 192.168/16.
How do devices with private addresses reach the internet? Through a router doing NAT (Network Address Translation). The router has one public address. When a device inside asks for something on the internet, the router sends the request from its own public address, and passes the answer back to the device:
private network the internet (public addresses only)
192.168.2.100 ---+
| +----------------+
192.168.2.200 ---+------| NAT router |------- 87.3.91.4 ------> servers
| | private<->public| (one public IP)
172.16.1.1 ---+ +----------------+
All the devices can reach the internet, but no one on the internet can reach them directly, unless the router is configured to allow it (a port is forwarded).
The same idea, drawn another way:
private network Internet
192.168.2.100 ─┐
192.168.2.200 ─┼─► [ NAT: 87.3.91.4 ] ─────► public servers
172.16.1.2 ─┘ one public IP
Binary and decimal¶
To work with netmasks you need to convert octets between decimal and binary. Each of the 8 bits has a value. Add up the values of the bits that are 1:
More examples:
| 128 | 64 | 32 | 16 | 8 | 4 | 2 | 1 | Decimal |
|---|---|---|---|---|---|---|---|---|
| 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| 1 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 128 |
| 1 | 0 | 0 | 0 | 0 | 0 | 0 | 1 | 129 |
| 0 | 0 | 0 | 0 | 0 | 1 | 1 | 0 | 6 |
| 1 | 0 | 1 | 1 | 0 | 0 | 0 | 0 | 176 |
| 1 | 0 | 1 | 1 | 0 | 0 | 1 | 1 | 179 |
| 1 | 1 | 1 | 1 | 1 | 1 | 1 | 1 | 255 |
Binary to decimal, for 10110000: the 1 bits are worth 128, 32 and 16, and 128 + 32 + 16 = 176.
Decimal to binary, for 105: divide by 2 again and again, writing down each remainder, until the quotient is 1:
105 / 2 = 52 remainder 1
52 / 2 = 26 remainder 0
26 / 2 = 13 remainder 0
13 / 2 = 6 remainder 1
6 / 2 = 3 remainder 0
3 / 2 = 1 remainder 1
Write the last quotient (1), then the remainders from the bottom up: 1101001. Fill with zeros on the left to make 8 bits: 01101001. Check it: 64 + 32 + 8 + 1 = 105.
So 11000000.10101000.00000001.00001111 is 192.168.1.15.
Netmasks and CIDR¶
An address has two parts: the network part, shared by all devices on the same network, and the host part, different for each device. The netmask says where the split is. It has the same 32-bit format as an address: the 1 bits mark the network part and the 0 bits the host part.
CIDR (Classless Inter-Domain Routing) notation just counts the 1 bits: /24 means the first 24 bits are the network. It is shorter, and is used in place of the dotted form:
| Decimal | CIDR | Binary |
|---|---|---|
| 255.0.0.0 | /8 |
11111111.00000000.00000000.00000000 |
| 255.255.0.0 | /16 |
11111111.11111111.00000000.00000000 |
| 255.255.255.0 | /24 |
11111111.11111111.11111111.00000000 |
| 255.255.255.128 | /25 |
11111111.11111111.11111111.10000000 |
| 255.255.255.192 | /26 |
11111111.11111111.11111111.11000000 |
With 192.168.1.1/24, the first 24 bits (192.168.1) are the network and the last 8 bits are the host. Every device from 192.168.1.1 to 192.168.1.254 is on the same network and can talk directly.
Network and broadcast addresses¶
Every network has two reserved addresses that no device can use:
- the network address, the first of the range, which names the network itself,
- the broadcast address, the last of the range, which sends a packet to every host on the network.
The usable host addresses are everything in between. Two masks on the same address:
192.168.8.12/24 (255.255.255.0) |
192.168.8.12/16 (255.255.0.0) |
|
|---|---|---|
| range | 192.168.8.0 to 192.168.8.255 | 192.168.0.0 to 192.168.255.255 |
| network address | 192.168.8.0 | 192.168.0.0 |
| broadcast address | 192.168.8.255 | 192.168.255.255 |
| hosts | 192.168.8.1 to 192.168.8.254 | 192.168.0.1 to 192.168.255.254 |
To calculate them for any mask, work in binary:
- Network address = address AND mask (a bit is 1 only where both are 1).
- Broadcast address = the network address with all host bits set to 1.
For 192.168.8.12 with mask 255.255.255.192 (/26):
address 11000000.10101000.00001000.00001100 192.168.8.12
mask 11111111.11111111.11111111.11000000 255.255.255.192
----------------------------------- AND
network 11000000.10101000.00001000.00000000 192.168.8.0
broadcast 11000000.10101000.00001000.00111111 192.168.8.63 (host bits all 1)
So 192.168.8.12/26 is in the network 192.168.8.0 to 192.168.8.63, and hosts can use 192.168.8.1 to 192.168.8.62. The tool ipcalc does this math for you, for example with a /24:
IP: 11000000.10101000.00000100.00001100 (192.168.4.12)
Netmask: 11111111.11111111.11111111.00000000 (255.255.255.0, /24)
Network: 11000000.10101000.00000100.00000000 (192.168.4.0)
Broadcast: 11000000.10101000.00000100.11111111 (192.168.4.255)
With 192.168.1.0/24, the first three octets are the network and the last octet identifies the host, so 192.168.1.2 and 192.168.1.200 are on the same network and talk directly, while anything outside goes through the router.
Subnetting¶
Subnetting splits a network into smaller ones by adding bits to the mask. Each extra bit doubles the number of subnets and halves their size:
/24 255.255.255.0 1 network 192.168.8.0 - 192.168.8.255
/25 255.255.255.128 2 subnets 192.168.8.0 - 192.168.8.127
192.168.8.128 - 192.168.8.255
/26 255.255.255.192 4 subnets 192.168.8.0 - 192.168.8.63
192.168.8.64 - 192.168.8.127
192.168.8.128 - 192.168.8.191
192.168.8.192 - 192.168.8.255
Every subnet loses two addresses to its network and broadcast addresses, so the more you split, the fewer addresses are left for hosts.
The default route¶
Machines on the same network talk to each other directly. A machine on 192.168.10.0/24 cannot send directly to 192.168.200.100, which is on another network. It needs a router, also called a gateway, connected to both networks with an address in each (say 192.168.10.1 and 192.168.200.1).
Each host is configured with a default route: the address to send every packet whose destination is not on its own network. Hosts on 192.168.10.0/24 use 192.168.10.1, and hosts on 192.168.200.0/24 use 192.168.200.1. The default route is also how machines on a private network (LAN) reach the internet (WAN).
TCP, UDP and ICMP¶
IP gets a packet to the right machine. The transport protocols, TCP and UDP, get it to the right program on that machine.
TCP¶
TCP (Transmission Control Protocol) is connection-oriented and reliable. It sets up a connection first, then checks that every packet arrives, in the right order, and resends any that get lost. The application does not need to worry about any of this. Use it when every byte matters, like downloading a file, web pages, email or SSH. It is like a careful conversation:
A: Are you ready? B: Yes.
A: Send me file XYZ. B: Here is part 1.
A: Got part 1, it is correct. B: Here is part 2.
A: Part 2 was damaged. B: Sending part 2 again.
All this checking costs time.
UDP¶
UDP (User Datagram Protocol) just sends. It does not check that packets arrive or stay in order; if the application needs that, it must do it itself. Less control makes it faster. For a video call or a live stream, a lost second is better skipped than resent late, so UDP is the right choice. DNS queries and NTP also use UDP.
ICMP¶
ICMP (Internet Control Message Protocol) works at the network layer, next to IP. It carries no user data. It is used to control and troubleshoot the network:
- detecting unreachable destinations,
- redirecting routes,
- controlling traffic volume,
- checking whether a host is alive, which is what
pingdoes.
$ ping google.com
PING google.com (173.194.32.135) 56(84) bytes of data.
64 bytes from 173.194.32.135: icmp_seq=1 ttl=48 time=239 ms
64 bytes from 173.194.32.135: icmp_seq=2 ttl=48 time=236 ms
^C
--- google.com ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1001ms
| TCP | UDP | ICMP | |
|---|---|---|---|
| layer | transport | transport | network |
| delivery checked, order kept, resends | yes | no | no |
| speed | slower | faster | |
| typical use | web, mail, SSH, file transfer | video, voice, DNS, NTP | ping, error messages |
Ports¶
An IP address finds the machine, but many programs run on it. A port number says which program should get the packet. A server program listens on a known port, and the client connects to that address and port, written with a colon: 200.216.10.15:443 reaches the HTTPS service on that host. In the same way 5.1.23.1:80 reaches the web server, while :21 reaches the FTP server on the same host.
- A port is a 16-bit number, so it goes up to 65535.
- Ports 1 to 1023 are called privileged ("well-known") ports. Services listen on them, and only root may bind them.
- Ports 1024 to 65535 are used by clients for the other end of the connection.
- The standard numbers are controlled by IANA, so SSH is port 22 on every system.
The ports you must know for the exam:
| Port | Service |
|---|---|
| 20 | FTP, data |
| 21 | FTP, control |
| 22 | SSH (secure shell) |
| 23 | Telnet (remote login without encryption) |
| 25 | SMTP (sending mail) |
| 53 | DNS (name resolution) |
| 80 | HTTP (web) |
| 110 | POP3 (receiving mail) |
| 123 | NTP (time) |
| 139 | NetBIOS |
| 143 | IMAP (accessing mail) |
| 161 | SNMP (network management) |
| 162 | SNMP traps (notifications) |
| 389 | LDAP (directory) |
| 443 | HTTPS (secure HTTP) |
| 465 | SMTPS (secure SMTP) |
| 514 | RSH (remote shell). The same number is also used by syslog for remote logging (108.2) |
| 636 | LDAPS (secure LDAP) |
| 993 | IMAPS (secure IMAP) |
| 995 | POP3S (secure POP3) |
A memory trick: the secure versions above 400 end in S: HTTPS 443, SMTPS 465, LDAPS 636, IMAPS 993, POP3S 995. On Linux, the standard ports and their names are listed in /etc/services.
IPv6¶
IPv4's 4.3 billion addresses are not enough for today's world, where phones, cars, TVs and sensors all want to be online (the Internet of Things). NAT helps, but the lasting answer is IPv6.
An IPv6 address has 128 bits, written as 8 groups of 16 bits, in hexadecimal, separated by colons:
That gives about 3.4 × 10^38 addresses, enough for anything we can imagine.
Shortening addresses¶
Two rules make IPv6 addresses shorter:
- Leading zeros in a group can be dropped:
0db8becomesdb8, and0000becomes0. - One run of all-zero groups can be replaced by
::, but only once in an address. Twice would be ambiguous: you could not tell how many zeros each::hides.
2001:0db8:85a3:0000:0000:0000:0000:7344 full
2001:0db8:85a3:0:0:0:0:7344 zeros shortened
2001:0db8:85a3::7344 zero groups replaced by ::
2001:0db8:85a3:0000:0000:1319:0000:7344 two runs of zeros
2001:0db8:85a3::1319:0:7344 only one of them can become ::
So 2001:0db8:0a0b:12f0:0000:0000:0000:0001 can be written 2001:db8:a0b:12f0::1.
Address types¶
| Type | A packet sent to it reaches |
|---|---|
| unicast | one interface. By default the left 64 bits are the network and the right 64 bits the interface |
| multicast | every interface in a group |
| anycast | only one interface out of a group |
IPv6 has no broadcast. To reach every host on the local network, send to the multicast address ff02::1, much like using the multicast address 224.0.0.1 in IPv4.
Main differences from IPv4¶
| Feature | IPv4 | IPv6 |
|---|---|---|
| address size | 32 bits | 128 bits |
| written as | decimal, dots: 192.168.1.4 |
hexadecimal, colons: 2001:db8::1 |
| address with a port | 200.216.10.15:443 |
in brackets: [2001:db8:85a3::7344]:443 |
| broadcast | yes | no, multicast to ff02::1 instead |
| packet lifetime field | TTL (Time to Live) | Hop Limit |
| finding neighbours | ARP | NDP (Neighbor Discovery Protocol), with more features |
| automatic configuration | needs DHCP | SLAAC (Stateless Address Autoconfiguration): hosts configure themselves |
| local address | every interface has a link-local address starting with fe80:: (fe80::/10) |
Ports, TCP and UDP work the same in both versions.
Summary¶
TCP/IP is a stack of protocols: IP gives each host a unique address so the network finds the machine, TCP or UDP find the program through a port, and ICMP handles control messages. An IPv4 address is 32 bits written as four decimal octets from 0 to 255 (about 4.3 billion in total), and the old classes A (1 to 126), B (128 to 191) and C (192 to 223) have default masks of /8, /16 and /24, with 127 kept for loopback. Public addresses are unique on the internet; the private ranges (10/8, 172.16-31, 192.168/16, that is 10.0.0.0/8, 172.16.0.0 to 172.31.255.255 and 192.168.0.0/16) can be reused in any private network, such as home devices, and reach the internet through a NAT router sharing one public address.
The netmask splits an address into network and host bits, and CIDR writes it as the number of network bits, so 255.255.255.0 is /24. I find the network address with a binary AND of address and mask (host bits 0), and the broadcast address by setting all host bits to 1; those two are reserved, and the hosts get everything in between. Adding bits to the mask splits a network into smaller subnets, each with fewer usable hosts. Packets for other networks go to the default route, the gateway. I convert octets by adding the bit values 128, 64, 32, 16, 8, 4, 2, 1, or by dividing by 2 and reading the remainders from the bottom.
TCP is connection-oriented and reliable, confirming delivery and order and resending lost packets; UDP is faster, best-effort and checks nothing, which suits live audio, video, DNS and NTP; ICMP carries no data and is used by ping and for error reports. A port selects the program. Ports from 1 to 1023 are privileged service ports, /etc/services maps every name to its number, and I know the common ones: 20 and 21 FTP, 22 SSH, 23 Telnet, 25 SMTP, 53 DNS, 80 HTTP, 110 POP3, 123 NTP, 139 NetBIOS, 143 IMAP, 161 and 162 SNMP, 389 LDAP, 443 HTTPS, 465 SMTPS, 514 RSH, 636 LDAPS, 993 IMAPS and 995 POP3S.
IPv6 addresses are 128 bits in eight hexadecimal groups, giving a practically endless supply, shortened by dropping leading zeros and replacing one run of zero groups with ::. They come as unicast, multicast and anycast, and there is no broadcast. IPv6 also brings brackets around addresses with ports, the Hop Limit in place of TTL, NDP in place of ARP, SLAAC self-configuration, and a link-local fe80:: address on every interface.