Code Talk - Networking

Published Friday, November 03, 2017

Low level talk about how information moves across the internet.

Why? Because we’re moving to AWS and Scot wants to try teaching this.

Types of addresses:

IP address Mac address
Dynamic Static
Related to geolocation (network location associated with physical location) Hardwired to computer’s network card
Higher level (IP layer, layer 3) Lower level (ethernet, layer 2)
32bit, example: 48bit, usually represented in hex, example: F1:4F:AF:FF:FF:FF

ifconfig - shows IP address, netmask, and more

$ ifconfig
lo0: flags=8049<UP,LOOPBACK,RUNNING,MULTICAST> mtu 16384
  inet netmask 0xff000000
  inet6 ::1 prefixlen 128
  inet6 fe80::1%lo0 prefixlen 64 scopeid 0x1
  nd6 options=201<PERFORMNUD,DAD>
gif0: flags=8010<POINTOPOINT,MULTICAST> mtu 1280
stf0: flags=0<> mtu 1280
  ether a4:5e:60:d0:47:85
  inet6 fe80::4b4:2f9:9d8c:bc76%en0 prefixlen 64 secured scopeid 0x4
  inet netmask 0xfffff800 broadcast
  nd6 options=201<PERFORMNUD,DAD>
  media: autoselect
  status: active

ipcalc <ip address> <netmask> - tells you stuff about ip address and subnet

$ ipcalc 0xfffff800
Address:          00001010.00001001.01100 001.01110000
Netmask: = 21   11111111.11111111.11111 000.00000000
Wildcard:            00000000.00000000.00000 111.11111111
Network:         00001010.00001001.01100 000.00000000
HostMin:            00001010.00001001.01100 000.00000001
HostMax:         00001010.00001001.01100 111.11111110
Broadcast:         00001010.00001001.01100 111.11111111
Hosts/Net: 2046                  Class A, Private Internet

Note: brew install ipcalc if not already installed

Subnet is a neighborhood. Individual connections have IP addresses, grouped by subnet.

Local network communication

Local communication uses the Mac address.

Packet: data we’re sending across the internet, has many layers

Packet layers (inner to outer):

  1. JSON from application (for example)
  2. HTTP request
  3. TCP
  4. IP address (both source and destination)
  5. Mac address (both source and destination)

Getting closer to hardware as we go outwards.

Scenario: on our local network, we want to send a packet to everyone in subnet

When sending packets in local network, we go from IP address to Mac address using ARP request. ARP request asks “who has this IP address?”, then whichever host responds “yes”. Then source host remembers destination host Mac address.

Get all address in local network: arp -a

$ arp -a
? ( at 0:0:5e:0:1:14 on en0 ifscope [ethernet]
? ( at e0:5f:45:8d:ea:b0 on en0 ifscope [ethernet]
? ( at 64:b0:a6:e1:8e:f8 on en0 ifscope [ethernet]
? ( at 60:f4:45:8c:56:af on en0 ifscope [ethernet]
? ( at d4:61:9d:0:7c:c6 on en0 ifscope [ethernet]

Default gateway and routes

How to communicate outside our local network?

Our router is our default gateway. Check it out in System Preferences > Network > TCP/IP.

In our packet, Mac address is local router, and IP address is for example, Google’s IP address.

Check out routing tables using netstat -nr

$ netstat -nr
Routing tables

Destination        Gateway            Flags        Refs      Use   Netif Expire
default            UGSc           39        0     en0
10.9.96/21         link#4             UCS            37        0     en0       link#4             UCS             1        0     en0          0:0:5e:0:1:14      UHLWIir        39       90     en0   1085          b0:c6:9a:77:c6:10  UHLWI           0        1     en0   1143     link#4             UCS             0        0     en0        64:b0:a6:e1:8e:f8  UHLWI           0        0     en0    545        60:f4:45:8c:56:af  UHLWI           0        0     en0   1025        d4:61:9d:0:7c:c6   UHLWI           0        0     en0    763        e0:ac:cb:8c:d6:60  UHLWI           0        0     en0    752        0:24:36:b8:14:1a   UHLWI           0        0     en0    830

Why do we need IP address AND Mac address to send stuff?

  • IP addresses change
  • When you boot, you don’t have an IP address
  • Need Mac address to hit the right device (packet will go from device to device, moving towards one IP address)

Traffic crossing the internet

Methods we used to get our packet out of our local network are repeated for many other networks until we arrive at destination network and given to host.

Packet is reassembled each router it passes through, updating src Mac address and destination Mac address.

Example packet route:

IP dest
IP src 104.x.x.x
Mac address src=”your Mac address”
Mac address dest=gateway1

IP dest
IP src 104.x.x.x
Mac address src=gateway1
Mac address dest=gateway2

Use traceroute to see routers packet passes through.

$ traceroute
traceroute to (, 64 hops max, 52 byte packets
 1 (  10.974 ms  1.879 ms  0.870 ms
 2 (  2.047 ms  11.580 ms  2.100 ms
 3 (  1.985 ms  2.070 ms  1.755 ms
 4  * * *
 5 (  2.976 ms (  2.919 ms (  7.841 ms
 6 (  4.204 ms  2.743 ms  2.624 ms


Problem: we’re running out of IP address.

IPv4 is a 32bit address space, so limited to ~4 billion addresses. We’re pretty close to hitting that limit.

Solution: NAT (network address translations) gateways

NAT gateways turn local address spaces to public address spaces (aka publically routeable spaces).

In AWS, they use 1to1 NAT for almost everything with public IP address.