HOWTO Keep a network of Debians up-to-date

This article was first written in August 2005 for the BeezNest technical
website (http://glasnost.beeznest.org/articles/289).
Due to the unrivaled apt tools, Debian is probably the easiest GNU/Linux distribution to keep up-to-date, at least when you have a fast Internet connection. In a network of many machines, the goal is slightly different though: you probably want to take as little bandwidth as possible away from users.

cron-apt

The first part of the solution, and the easiest to setup, is to install package cron-apt which will download the packages early in the morning from cron, when users are not at work. No need to configure anything, it works out of the box to do the equivalent of an (apt-get) update and download the packages, without installing anything for you. You keep control. You can configure it to make it install some or all updates if you want.

apt-proxy

The second part is to setup a proxy specialized in getting Debian packages for you, only once for all machines. This proxy is apt-proxy. The second machine requesting the same package will take only a fraction of the time needed to download it from the Internet. It will also spare your Internet connection quota, so even for a few Debian machines, it may prove useful. You only need some spare disk space on a Debian machine on your network to use it (it is also performance-hungry when working). It is slightly trickier to setup, because you need to carefully define your sources. Here is a proposed configuration for it:
[DEFAULT]
;; All times are in seconds, but you can add a suffix
;; for minutes(m), hours(h) or days(d)

;; Server IP to listen on
;address = 192.168.0.254

;; Server port to listen on
port = 9999

;; Control files (Packages/Sources/Contents) refresh rate
;;
;; Minimum time between attempts to refresh a file
min_refresh_delay = 1h

;; Minimum age of a file before attempting an update (NOT YET IMPLEMENTED)
;min_age = 23h

;; Uncomment to make apt-proxy continue downloading even if all
;; clients disconnect.  This is probably not a good idea on a
;; dial up line.
;; complete_clientless_downloads = 1
complete_clientless_downloads = 1

;; Debugging settings.
;; for all debug information use this:
;; debug = all:9
debug = all:4 db:0

;; Debugging remote python console
;; Do not enable in an untrusted environment
;telnet_port = 9998
;telnet_user = apt-proxy
;telnet_password = secret

;; Network timeout when retrieving from backend servers
timeout = 15

;; Cache directory for apt-proxy
cache_dir = /var/cache/apt-proxy

;; Use passive FTP? (default=on)
;passive_ftp = on

;; Use HTTP proxy?
;http_proxy = host:port

;; Enable HTTP pipelining within apt-proxy (for test purposes)
;disable_pipelining=0

;;--------------------------------------------------------------
;; Cache housekeeping

;; Time to perform periodic housekeeping:
;;  - delete files that have not been accessed in max_age
;;  - scan cache directories and update internal tables
cleanup_freq = 1d

;; Maximum age of files before deletion from the cache (seconds)
max_age = 120d

;; Maximum number of versions of a .deb to keep per distribution
max_versions = 3

;; Add HTTP backends dynamicaly if not already defined? (default=on)
;dynamic_backends = on

;;---------------------------------------------------------------
;;---------------------------------------------------------------
;; Backend servers
;;
;; Place each server in its own [section]

[debian]
;; The main Debian archive
;; You can override the default timeout like this:
;timeout = 30

;; Rsync server used to rsync the Packages file (NOT YET IMPLEMENTED)
;;rsyncpackages = rsync://ftp.de.debian.org/debian

;; Backend servers, in order of preference
backends =
        ftp://ftp.skynet.be/debian
        ftp://ftp.belnet.be/debian
        http://ftp.de.debian.org/debian
        http://ftp2.de.debian.org/debian
        ftp://ftp.uk.debian.org/debian

[debian-non-US]
;; Debian debian-non-US archive
;timeout will be the global value
backends =
        ftp://ftp.skynet.be/debian-non-US
        ftp://ftp.belnet.be/debian-non-US
        http://ftp.uk.debian.org/debian-non-US
        http://ftp.de.debian.org/debian-non-US
        ftp://ftp.uk.debian.org/debian

[security]
;; Debian security archive
backends =
        ftp://ftp.skynet.be/debian-security
        ftp://ftp.belnet.be/debian-security
        http://security.debian.org/debian-security
        http://ftp2.de.debian.org/debian-security

;[apt-proxy]
;; Apt-proxy new versions
;backends = http://apt-proxy.sourceforge.net/apt-proxy

;[backports.org]
;; backports.org
;backends = http://backports.org/debian

[blackdown]
;; Blackdown Java
backends =
;       ftp://ftp.easynet.be/blackdown/debian
;       http://ftp.gwdg.de/pub/languages/java/linux/debian
        ftp://ftp.tux.org/java/debian/

;[debian-people]
;; people.debian.org
;backends = http://people.debian.org

;[emdebian]
;; The Emdebian project
;backends = http://emdebian.sourceforge.net/emdebian

;[rsync]
;; An example using an rsync server.  This is not recommended
;; unless http is not available, because rsync is only more
;; efficient for transferring uncompressed files and puts much
;; more overhead on the server.  See the rsyncpackages parameter
;; for a way of rsyncing just the Packages files.
;backends = rsync://ftp.uk.debian.org/debian

[java]
backends =
        ftp://ftp2.skynet.be/pub/ftp.blackdown.org/debian

[multimedia]
backends =
        ftp://ftp.nerim.net/debian-marillat

[bxlug]
backends =
        http://apt.bxlug.be/
An example matching sources.list file for Sarge would be:
# See sources.list(5) for more information, especialy
# Remember that you can only use http, ftp or file URIs
# CDROMs are managed through the apt-cdrom tool.

# Standard Debian sources (with contrib and non-free)
deb http://apt-proxy:9999/debian/ sarge main contrib non-free
deb-src http://apt-proxy:9999/debian/ sarge main contrib non-free

# Debian Security updates
deb http://apt-proxy:9999/security/ sarge/updates main contrib non-free

# BxLUG sources
deb http://apt.bxlug.be/ experimental-sarge/
deb http://apt.bxlug.be/sarge/ main/
deb http://apt.bxlug.be/sarge/ contrib/
deb http://apt.bxlug.be/sarge/ non-free/
deb http://apt.bxlug.be/sarge/ not-debian/

# Multimedia stuff from Marillat
deb http://apt-proxy:9999/multimedia/ sarge main

# Java gcc-3.2
deb http://apt-proxy:9999/java/ sarge non-free

apticron

To be kept informed (by e-mail) of available updates, just install the package of that name.

Special note about web proxies

Using a general-purpose web proxy is generally a bad idea, because they are not meant to store large files. If you want to do it anyway, you just need to define the matching environment variable amongst these: HTTP_PROXY, HTTPS_PROXY, FTP_PROXY.