Chapter 10. System-specific Guidelines

At a Glance

Most of this handbook applies no matter what kind of hardware and operating systems you’re running. This chapter discusses some technical recommendations that are specific to Unix/Linux, Microsoft Windows, and MacOS 7-9 operating systems (MacOS X is covered by the Unix material).

Unix and Unix-like Operating Systems246

Traditionally, Unix-based systems were deployed in large-scale multiuser time-sharing environments, or as clusters of workstations with networked filesystems. Today, Unix-based systems are increasingly used as single-user workstations or servers.

Because there have been so many different versions of Unix-like operating systems, many security mechanisms are vendor-specific. It’s particularly important to read all of the manuals for your vendor’s version of Unix. Several good books, web sites, and mailing lists devoted to Unix security are listed in Annexes 2-5.

Users, Groups, and the Superuser

Much of Unix’s security relies on the separation of users and user groups. All files and processes have an associated effective user and group id that determines what privileges they have. Two users should never share the same account or user ids; instead, assign individual accounts and use groups to share file access rights among users.

On Unix-like systems, the user root (uid 0) is the superuser and usually has the ability to modify every aspect of the system. Accordingly, protecting the root account and processes that run with root privileges is a critical aspect of Unix security. Avoid using the root account for routine activities, and disable logins by root. When you must use root, use the su command (or a variation like sudo) to change from your normal user account to root. This both creates accountability through logging and requires that an attacker subvert two accounts to gain superuser access. Obviously, it behooves you to limit access to commands like su; on some systems, only users in the wheel group can use su.

Some Unix kernels support the ability to limit what even root can do while the system is running in its normal mode, through the use of kernel security levels or capabilities. Check your vendor’s documentation to determine if these limits are available in your version, and take advantage of them if they are.

If you have particularly sensitive data files, you may wish to keep them encrypted and on removable media to guard against their exposure in the event that the root account is compromised. Note, however, that encryption is not sufficient if the decryption program resides on the same system, as an attacker with superuser access can modify the decryption program to save a copy of the decrypted file.

Filesystems and Security

Files in Unix are assigned an owner and a group, and the chmod command is used to specify what the owner, members of the file’s group, and everyone else can do to the file. These file permissions include the ability to read the file’s contents, write to the file, and execute the file as a program. Permissions on directories are used to

246 Throughout this section, we will be referring to Unix and Unix-based systems interchangeably. Although the term “Unix” will appear in this text most often, the information is applicable to Linux and other variants on Unix.

control who can examine files in the directory and who can add or remove files from a directory. Learn how to use chown, chgrp, and chmod to control access to files, and ls to list important file access information.

Each Unix user has a umask value that determines the default permissions for files and directories they create. Use the umask command in login scripts to insure that users have an appropriate umask value. The value 027 allows only others in the user’s group to read and execute, but not modify, files. The value 077 denies any access to anyone but the user themselves.

Some Unix systems support more fine-grained access control lists (ACLs) for files. ACLs can grant or deny permission for individual users to read, write, or execute files. If your system supports ACLs, learn how to use them.

Some Unix systems support immutable and append-only attributes on files. An immutable file can’t be altered, even by root, unless the system is booted in a special low-security mode from the console. An append-only file can only be added to, and not otherwise modified or deleted; log files are good candidates for append-only attributes. If your system provides these, take advantage of them.

Unix files with the setuid (SUID) permission run with the effective user id of their owner, rather than of the user who executes them. SUID files allow one user to execute commands that require the privileges of another (often root); as such, they represent a potential vulnerability. Setgid (SGID) files run with the effective group id of their group, rather than that of the user who executes them. You should periodically scan your system for SUID/SGID files and make sure that you know why they have those permissions. Keep a list of these files printed out. Avoid writing SUID/SGID programs yourself; never write SUID/SGID shell scripts. Some filesystems can be mounted with a nosuid option that prevents SUID/SGID permissions from functioning; it’s a good idea to mount user home directories and other non-system partitions with this option.

Unix represents devices as files. For example, printers, serial ports, hard drives, and even the system’s memory are accessible through device files. Although device files are generally found in the /dev directory, they can be created anywhere by a user with appropriate (usually superuser) privileges. If an unauthorized user can read your system memory, they can access sensitive information of other users; if they can write to your system memory, they can compromise the system. The same caution applies to raw disk devices and several other kinds of devices. Scan for device files on your system and insure that they have appropriate ownership and permissions. If your filesystems can be mounted with a nodev option, consider using that. On some systems, a file called logindevperm or fbtab controls how device permissions are modified when a user logs in at the console (to prevent, for example, remote users from turning on the microphone and monitoring the room). If you have this file, check to be sure it’s properly set up.

Encryption

Several Unix commands seem to obfuscate data but are ineffective as encryption. Don’t use rot13 or the crypt command, as they are trivial to break. Many systems provide strong encryption through a des command, or through the openssl application and libraries. Similarly, don’t rely on the sum command for cryptographically strong checksums. Instead use md5, md5sum, or openssl to generate cryptographic message digests.

TCP/IP Networking

Unix systems are often used for network applications and services. Many network services are started by the inetd (or xinetd) daemon. Examine the configuration file(s) used by this daemon and disable any services that you do not need; protect other services with the TCP wrapper daemon tcpd, unless your inetd has built-in support for TCP wrappers.

Other network services are started at system boot by files in the /etc/init.d or /etc/rc*.d directories on in the files /etc/rc and /etc/rc.local. Disable any services that you do not use. Pay particular attention to services that may provide outsiders with information about your system or its users, such as fingerd.

Every Unix system should run its own host-based packet-filtering firewall. Consult vendor documentation to determine if your system has a firewall and how to use it. Typical firewall configuration tools include ipfw, ipchains, and iptables. These firewalls should be configured to block all packets by default, and to allow only packets destined for services you intend to provide. An external firewall should be used to prevent outsiders from accesses several protocols and services that might be offered by hosts in your organization (e.g., SNMP, NFS, NTP, LPD, Samba, RIP). Use static routing whenever possible.

Traditionally, several standard services depended on authentication by the hostname or IP address of the client, or by passwords sent in plaintext over the network connection. Neither of these approaches is secure. Instead, applications should use cryptographic approaches to authentication, with either shared or public key systems. Many applications (telnet, rlogin, rcp, rsh, ftp) can be disabled and replaced with Secure Shell (ssh), which provides strong authentication. Do so, and remove any .rhosts or /etc/hosts.equiv entries that list ip addresses of trusted machines. Other services (pop, imap, http, ldap) should be compiled with the OpenSSL libraries to provide SSL/TLS connections to clients so that passwords are not transmitted unencrypted.

When possible, run network services as non-root users. Many network daemons can be configured to start up as root (in order to bind to a network port lower than 1024, which requires root privileges on most Unix systems), and then give up their privileges and change to a non-root user. Give each daemon its own non-root account to use, rather than sharing a single (“nobody”) account. When possible, run network services in a chroot jail environment to limit the damage possible if they are compromised.

If you run anonymous FTP services, use an up-to-date version of the FTP daemon. Don’t provide your real

/etc/passwd file in the FTP area. Make sure that /etc/ftpusers, the list of users who cannot connect by FTP, includes at least root, uucp, bin, and any other account that does not belong to an actual human being. Be paranoid about the directory permissions and ownership in the FTP area; configure “incoming” directories to prevent downloads and “outgoing” directories to prevent uploads. Scan your FTP logs regularly.

If possible, use postfix, exim, or qmail instead of sendmail on mail servers. Never use anything but the latest version of your MTA. Use mail aliases to insure that mail to any valid non-user account is delivered to a real user; in particular, be sure that mail can be delivered to root, postmaster, and abuse addresses. Protect the mail aliases file from changes by unauthorized individuals. If you have mail aliases that deliver mail to programs or files, examine them carefully and delete them if possible.

Do run the authd/identd daemon if you have multiuser machines. This can help you if you receive a report of someone at your system attacking another system. Use a version that returns encrypted identifiers to avoid exposing user information to outside systems.

If you don’t use RPC, disable the portmapper daemon; if you do, restrict access to it and use the securenets feature if it’s available. Disable any RPC services provide by inetd that you don’t use (and rexd in particular). Use Secure RPC if it’s available on your system. Only Secure RPC provides a reasonable basis for using NIS+ or NFS. Avoid using NIS or NIS+ in compatibility mode. If you use NFS, use version 3 in TCP mode if possible, and limit the number of filesystems you export and the set of hosts who may mount them. Try to export filesystems read-only. Unless you specifically tell it not to, NFS protects exported root-owned files from modification by root on the client host so it can be beneficial to insure that all exported files and directories are owned by root, rather than by other users (such as bin) that might exist on the client host.

If you use X11, enable the best authentication possible. Kerberos or Secure RPC are strong authentication systems. “Magic cookies” are weaker. The xhost program provides the least security. Tunneling X11 connections through SSH also provides strong protection.

If you provide SMB service with Samba, prefer “user” or “domain” security to “share” security. Enable encrypted passwords, and require that clients use a recent version of the SMB protocol with Samba’s “min protocol” option. Don’t use the “admin user” option or map the DOS archive bit to the Unix executable permission. Learn how to use the “veto files” option.

Keep an eye on your network. Scan the output of netstat and lsof regularly to see what network connections are being made to and from your system. Use who and last to see user connections. Use nmap, Nessus, ISS, and other network security scanners to probe your system from the outside to see if you have vulnerabilities that should be corrected. For some machines, it may be best to disconnect them from the network altogether.

Defending Accounts

The first line of defense for Unix accounts is their password. Unix systems don’t store user passwords in plaintext; instead, they store a cryptographic hash of the password that can not be decrypted into the password. When a user logs in, the system computes the hash of the password they type and compares it to the stored hash.

Older Unix systems stored user account information and encrypted passwords in the /etc/passwd file. This file must be world readable so that processes can associate user ids with login names. Unfortunately, that meant that local users (or others) could copy the passwd file and attempt to find passwords by encrypting common dictionary words, account names, etc. and comparing their encrypted attempts with the passwords listed in the files.

Newer Unix systems continue to store public account information in /etc/passwd, but store the encrypted password information in a file called /etc/shadow (or sometimes /etc/passwd.adjunct) that is readable only by root.

Many Unix systems come with several default accounts that are used to separate process or file ownership privileges, such as daemon, bin, uucp, etc. Make sure that the encrypted password entry for all of these accounts begins with a “*” character so that no possible password can be used to access the account. Here’s an excerpt of a /etc/shadow file:

root:$1$24g7KF8j$Rjky384Fd1PvtSCOJ/WW.1:12264:0:99999:7:::134551156 bin: *:10890:0:99999:7::: daemon:*:10890:0:99999:7::: adm:*:10890:0:99999:7::: lp:*:10890:0:99999:7::: sync:*:10890:0:99999:7::: shutdown:*:10890:0:99999:7::: halt:*:10890:0:99999:7:::

In this example, only the root account has a valid password. No one can log into the other accounts (although root can still assume their privileges with the su command if necessary). On many systems, account passwords can be set to expire after a certainly amount of time, which provides valuable protection against an attacker taking over a dormant account which the actual owner wouldn’t notice. Use a lifetime between one and six months. On many systems, you can require that passwords meet certain strength requirements (length of password, variety of characters, etc.) This functionality is often available through PAM on systems that support it.

It’s a good idea not to create default or guest accounts, but if you must, consider using the rsh or rbash restricted shell so that the account can only run a limited number of commands (don’t confuse this with the remote shell client, also called rsh). Make sure that none of the commands provide access to an unrestricted shell (as many editors do).

Protecting Against Programmed Threats

Never unpack or compile new software as root. It’s often possible to compile software in a chroot environment to protect yourself against some kinds of Trojan horses.

Keep an eye on the PATH environment variable of users (especially root). The PATH specifies the list of directories that will be checked when a command is typed without giving an absolute path. Root’s PATH should only contain standard directories that are writable only by trusted accounts, and that are regularly audited for changes (using software like Tripwire or AIDE). Don’t put “.” (the current directory) in the PATH, as this makes it easy for attackers to trick root into running Trojaned software. When working as root, get into the habit of typing full paths to important commands (e.g. /bin/su). You should also use full paths in all shell scripts, startup files, or cron jobs that you write.

Preventing Denial of Service Attacks

Unix systems offer several protections against denial of service attacks. Many systems support per-user limits on CPU and memory usage through PAM or other login files, and per-user limits on disk usage through the quota system, if you enable it (and you should).

Processes and Memory

The ps command is used to view processes running on the system. On BSD-based Unix flavors, ps –auxw lists all processes; on SVR5-based flavors, use ps –elf. Each process has a process id number that is used to reference it in commands that interact with running processes.

Keep an eye on user processes. Use programs like top and lsof regularly to see what processes your system is running, and by whom. Enable process accounting so you can keep track of processes that have been run in the past and users who might be using excessive processing time.

The nice or renice commands can be used to reduce the amount of CPU priority a process has, and are useful for long-running background tasks. Root can also use nice to increase CPU priority for processes, which can be helpful when user processes are bogging down the system and root needs to get enough CPU time to stop them.

The kill command is used to send a signal to a process. Some signals are used to tell daemons to reread configuration files or to notify them about changes in the system. Other signals can be used to suspend or terminate a process. The TERM signal (sent by default with kill <process-id>, or explicitly using kill –TERM < processid>) often terminates a process; the KILL signal unconditionally terminates a process. The TSTP signal suspends a process, and is useful when you want to make an image of the process’s memory with gcore for forensic purposes, or when self-replicating processes have taken up all the process slots. In the latter case, you can suspend each process and then kill them all at once so that they can not spawn.

Unix systems support virtual memory, traditionally called swap space. When the system processes require more memory than RAM installed, disk space devoted to swapping is used instead. Insure that you have sufficient swap

space on disk partitions (some Unix systems can also swap to files on standard filesystem partitions, although this is less efficient).

Disks

In addition to the quota system, protect disks by isolating critical partitions from those that might be filled accidentally or intentionally, such as mail spool or file upload areas. Insure that each partition has sufficient space and inodes for file storage.

Monitor disk usage regularly and encourage users to archive and delete old files.

Microsoft Operating Systems

Microsoft’s operating systems began with a focus on standalone personal computing. They were soon actively deployed in networks, initially using Microsoft’s own protocols, and later converging primarily on TCP/IP. Systems based on Windows 3.x and Windows 95/98/ME are largely suitable only as client workstations; in contrast, systems based on Windows NT (including Windows 2000 and XP) are often configured as servers and have much more sophisticated security controls.247 Differences in Windows versions can be dramatic. If you are in an environment that mixes several versions of Windows, each may require different attention. This section focuses primarily on hardening security on NT-based systems.

As with other operating systems, there’s no substitute for reading the manuals, as well as other books, web sites, and mailing lists devoted to Windows security. Microsoft’s web site includes a large security section with many documents and useful tools, including the Baseline Security Analyzer, a program that analyzes the configuration of NT-based systems and makes recommendations for hardening them. Run it frequently.

Users, Groups, and the Administrator

Windows also uses Users and Groups to control permissions; Groups in particular usually define the abilities of their Users, though finer-grained per-user access controls are also available. As distributed, the user “Administrator” is granted membership in the “Administrators” group, which provides superuser privileges over the system, and represents a key target for attackers.

The administrator account can be protected in several ways. Changing the name of the account to something else makes automated attacks more difficult (though it is often still possible to determine the new name); creating a disabled dummy account named Administrator can help you detect attacks. Administrator logins can be restricted to the local console, and can be audited.

It’s crucial to keep track of which users belong to which groups. The “Computer Management” application in the Administrative Tools folder provides a view of all Users and Groups that are defined.

Filesystems and Security

Windows systems can use two types of filesystems: FAT-based filesystems (FAT, VFAT, FAT32) are compatible with all versions of Microsoft’s operating systems, while NTFS filesystems are only support by NT-based versions. Only NTFS provides any filesystem security. The FAT filesystem has no concept of file ownership or access control, and should not be used on any secure system.

247 Oddly, DOS systems are also useful as servers in some situations. Because they are single-user systems that offer few points of attack, they can be highly suitable for use as single-purpose log servers, terminal servers, firewalls, and even DNS servers.

Access to file and directories on NTFS systems are managed through Access Control Lists (ACLs). ACLs typically specify which permissions – read, write, execute, list contents, modify, or full control, among others – have been granted to which groups of users. Each object in the filesystem (and in the Windows registry) has an associated ACL or inherits one from a folder above it.

The ACL system is a powerful and complicated security tool that requires considerable study. Microsoft provides some security templates that assign reasonable ACLs to system folders and registry keys, but you may wish to be more restrictive.

Encryption

Microsoft Windows provides a unified CryptoAPI library for cryptographic support. On NTFS filesystems, files and directories can be encrypted using the cipher.exe tool, which sets up an transparent encrypted file system (EFS). EFS is based on public key cryptography so remote users can access their encrypted data as long as they present the appropriate private key; in addition, EFS can be configured so administrators can recover encrypted data if the key is lost (which may or may not enhance security).

TCP/IP Networking

Before Windows 2000

Microsoft Windows supported a peer-to-peer Ethernet networking model (NetBIOS over the NetBEUI transport protocol) before the widespread emergence of TCP/IP, and the legacy of NetBIOS remains in Microsoft’s Printer and File sharing services, which are implemented as NetBIOS over TCP/IP (sometimes called “NBT”). The file sharing protocol itself is referred to as Server Message Block (SMB) or CIFS. Internet RFCS 1001 and 1002 describe NetBIOS over TCP/IP in great detail.

NetBIOS includes its own host name resolution and authentication protocols. In the simplest model, NetBIOS nodes (hosts) discover each other and register their names on the network by using broadcast packets. In addition to being difficult to scale up to larger networks, this mode makes it relatively simple for nodes to “steal” one another’s registered name and effectively impersonate one another.

A more secure mode of operation requires the NetBIOS nodes to communicate (point-to-point) with hosts designated as NetBIOS name service nodes (sometimes called WINS servers) to register and look up names, and with NetBIOS datagram distribution nodes to broadcast packets at the NetBIOS level. The NetBIOS name servers can provide safeguards against machines spoofing each other’s names. In addition, the registry key \HKEY_LOCAL_ MACHINE\CurrentControlSet\Services\Netbt\Parameters\NoNameReleaseOnDemand can be set to 1 to prevent servers from responding to name release requests that might be forged by an attacker who wishes to claim a server’s name and impersonate the server.

In most cases, users who wish to use a resource must first log into the SMB server providing that resource. The login process in modern SMB dialects uses challenge- response authentication.248 When a user requests to log in, the SMB server sends a unique challenge string to the client. The client encrypts this string using a session key computed from a cryptographic hash of the user’s password and returns the response to the SMB server. The SMB server performs the same computation and compares its results to the client’s. If they match, the user is authenticated. The exact form of the computation depends on the SMB dialect in use; two major approaches (“LM” and “NT”) are currently defined.

248 Older SMB dialects (e.g., that used in Windows for Workgroups) allowed plaintext passwords to be sent over the network.

Note that this approach implies that the SMB server (or some other authentication server with which it communicates) has the user’s hashed password available to it (but not necessarily the cleartext password). If this server is compromised, all the user hashed passwords are compromised (so the attacker may be able to masquerade as the user and connect to other SMB servers). On the other hand, this approach prevents the cleartext or hashed password from ever traveling over the network. SMB authentication servers must thus be protect like Kerberos domain controllers.

If Windows file sharing will not be used, NetBIOS over TCP/IP can be completely disabled in the Advanced TCP/IP Settings. If all machines in the network support the newer versions of the NetBIOS/SMB protocols, they should be configured to only respond to requests using the latest possible protocol version (NTLMv2 in most cases), to prevent an attacker from negotiating an earlier, flawed protocol. If remote administration of filesystems is not necessary, the registry key HKEY_LOCAL_MACHINE\System\CurrentControlSet\Services\LanmanServer\Parameters\ AutoShareWKS can be set to 0 to disable it.

Windows can be configured to allow remote users access not only to files, but also to registry keys. The security permissions on the HKEY_LOCAL_MACHINE\System\CurrentControlSet\Control\SecurePipeServers\winreg key determine which users can remotely modify the registry. It is important to insure that this group include only Administrators if remote access is necessary, and nobody at all otherwise. The Remote Registry Access service can also be disabled.

The Advanced TCP/IP Settings for network adaptors on Windows NT-based systems include a simple packet filter that can be configured to allow or deny incoming TCP or UDP packets by destination port, as well as filtering non-IP protocols.

Since Windows 2000

Windows 2000 domains provide significantly more control over participating clients than was available in earlier versions. Notably, domain security policies can override clients’ local security policies when the client joins the domain, which can be useful to centrally insure that your client workstations have strong security.

Windows 2000 and later systems use Kerberos as their primary network security layer, although they continue to support NetBIOS, and the recommendations above apply. Kerberos, as discussed earlier in this handbook, provides for secure authentication and authorization for network services. The Windows 2000 Primary Domain Controller is the Kerberos master.

Windows 2000 also supports IPsec for creating virtual private networks. IPsec negotiation can be requested or required on client and server connections. IPsec is configured through the IP Security Policy Management application. Earlier Windows versions used a proprietary Microsoft protocol (PPTP) for VPN tunnels; in most cases, you should prefer IPsec unless you need to support older systems.

Windows XP also added a built-in stateful packet filter called Internet Connection Filter (ICF) that is ideal for systems that will be used as Internet clients. By default it only allows incoming packets associated with connections initially established by the client.

Defending Accounts

Recent Windows-based systems support long passwords for accounts. Encourage or force users to use longer passwords or passphrases rather than shorter ones to reduce the risk of password guessing. These systems also support “complexity requirements” for passwords and password expiration. On systems participating in Windows

2000 domains, passwords are stored on the domain controller and managed in the usual Kerberos fashion. Account lockout settings can also be turned on to make password-guessing attempts more costly.

By default, most Windows systems come without security auditing turned on. Auditing is configured in the Local Security Policy (or through the domain security policy). It can be useful to turn on auditing for account logons and account management (success and failure) in order to keep an eye on attempted logins. Audited events are displayed in the Event Viewer. Logging several kinds of failure events (such as failed privilege use) can also be helpful. Be sure to set maximum sizes for all logs (through the Event Viewer), and to disable guest access to logs.

Protecting Against Programmed Threats

Windows NT-based systems often come configured with several services enabled. Windows services, like Unix daemons, are background processes that provide functions to applications. In some cases, these services provide access to outsiders via the network, as they offer remote access services (like telnet) or remote procedure calls. For example, the Messenger service permits remote machines to pop up alert windows on local machines, and has been abused by spammers.

Using the Services application in the computer management console, ensure that all unnecessary services are stopped and disabled. On clients that don’t share files, the telnet, server, remote registry access, and several other remote access services can be disabled to decrease points of vulnerability (sometimes at the expense of centralized management).

Using the local or group security policy, ensure that anonymous users have no access without explicit permissions (this setting appears in the Security Options folder in Local Policies on Windows 2000).

Preventing Denial of Service Attacks

Processes and Memory

Windows processes can be monitored and halted with the Task Manager. Task Manager can also adjust process priority to one of six levels, from “Low” to “Realtime”, and display memory usage. Because few Windows systems are used in multiuser timesharing environments, process and memory overflow attacks are usually the result of an errant program that can be detected and halted through Task Manager.

Disks

NTFS supports a user quota system that can be used to protect disks from overflow conditions. Again, this is most useful on shared client workstations, as servers should generally have few users other than their administrators, and server applications often require administrative privileges.

Network

Windows NT-based systems provide several registry settings that can help protect them from some kinds of network denial of service attacks, such as SYN flooding. In most cases, however, these settings are not enabled. Settings to examine include (in \HKEY_LOCAL_MACHINE\CurrentControlSet\Services\Tcpip\Parameters) SynAttackProtect, TcpMaxHalfOpen, TcpMaxHalfOpenRetried,

Other kinds of denial of service attacks can be made more difficult by disabling automatic detection and discovery functions. The keys EnablePMTUDiscovery, EnableDeadGWDetect, and EnableICMPRedirects should be set to 0 to prevent the system from responding to unusual network conditions in possibly surprising ways. Interfaces should be directed not to perform automatic router discovery, and should be configured for static routes.