File Storage

The CSL's primary file storage system is AFS. AFS is a distributed file system that enables sharing files across both local area and wide area networks, and is available on all supported workstations.

AFS is based on a distributed file system originally developed at the Information Technology Center at Carnegie-Mellon University that was called the Andrew File System.

Some useful features of AFS:

  • In conjuction with Kerberos, AFS provides a global authentication system (all passwords are verified with a site-wide database).
  • Access Control Lists (ACLs) provide more flexibility in setting file access permissions than traditional Unix file systems.
  • Users can access AFS files at remote sites, if given appropriate permissions.

Supported Windows 7 workstations have additional network file services available. See the Windows Documentation.

AFS Commands Quick Reference

These tables list the most common AFS commands used by users. All commands have two help functions: <command> help and <command option> -help.

File Service Commands Description
fs listquota <path>, fs lq <path> list the AFS quota for the volume containing file or directory path. If no path is specified, the current directory is used.
fs listacl <path>, fs la <path> List the Acess Control List for the directory path (or the directory containing path). If no path is specified, the current directory is used.
fs setacl <path> <user> <acl>, fs sa <path> <user> <acl> Set the Acess Control List for the user user on the directory path.
fs copyacl <srcpath> <destpath> Copy the Acess Control Lists from the directory srcpath to the directory destpath. Note that this does not delete the existing Access Control Lists from destpath.
afs_rseta <path> <user> <acl> Recursively set the Acess Control List for the user user on the directory path and all subdirectories of path.
fs whereis <path> Print the name of the AFS Fileserver for the volume containing file or directory path If no path is specified, the current directory is used.
AFS Group Commands Description
pts creategroup <username:group> Create a group named username:group. username must be your username.
pts adduser <user> <username:group> Add user user to group username:group. Usually username is your username.
pts removeuser <user> <username:group> Remove user user from group username:group. Usually username is your username.
pts membership <user> List the groups user is a member of
pts membership <username:group> List the members of group username:group
pts listowned <username> List the groups owned by username
Authentication Commands Description
tokens Print information about the AFS Authentication Token of the current process authentication group (PAG).
kalog Renew/Create Kerberos Ticket and AFS Token
Miscellaneous Commands Description
up <srcdir> <destdir> Recursively copy srcdir to destdir, preserving AFS Access Control Lists.

File-related Commands

Quotas

The AFS file system has per-volume disk space quotas. To examine your quota and the current usage, use:

On Linux systems:

fs listquota <path>, or
fs lq <path>
Where <path> is the pathname or a file or directory in the volume.

On Windows systems:

  • Right-click on a directory, and use the AFS | Volume/Partition | Properties menu, or
  • In a command window use the fs listquota command as listed for Linux

Disk quota and amount used are reported in 1K Byte Blocks. The percentage used of the volume quota and the file server disk partition holding that volume are also reported.

lff - Large File Finder

lff is a program which can be used to find large files and directories in a directory tree. It searches all files, including hidden files (dotfiles) and subdirectories of a directory tree. It can also omit directories within the tree from the search.
It has the capability to output the format in Kilobytes or Megabytes, and is configurable with command line options.

lff option Description
-f, --fileminsize <size> Sets the minimum file size that you are searching for. It defaults to 1 megabyte. You are able to specify the size in either bytes, kilobytes, or megabytes. To specify bytes, just use a number. To specify kilobytes, append a k or a K after the number. To specify megabytes, append either a m or a M after the number.
-d, --dirminsize <size> Sets the minimum directory size that you are searching for. It defaults to 5 megabytes. You can specify kilobytes or megabytes the same as described for --fileminsize.
-h, -?, --help Shows command usage and immediately exits.
-k[K], --kilobytes (Case insensitive in the single character implementation) Formats the output to kilobytes instead of bytes.
-m[M], --megabytes (Case insensitive in the single character implementation) Formats the output to megabytes instead of bytes.
-F, --Fileonly (Case Sensitive) Limits the output to only display large files instead of large directories and files.
-D, --Dironly (Case Sensitive) Limits the output to only display large directories instead of large files and directories.
-o, --omitdirs Sets directories to omit. They should be in the directory tree that you are checking, but the program does not make sure of this and it will not cause any problems if the omitted directories are elsewhere. It will only cause minor slowdown.
-v, --version Displays the current version of the program and immediately exits.

lff Examples

The most common usage of lff will probably be to check a home directory to find large files and directories (such as the mozilla cache) when an AFS quota has been filled. The easiest way to do this is:
/s/std/bin/lff ~

In order to format output to kilobytes, the following will work:
/s/std/bin/lff ~ -k

A more complex usage would be the following:
/s/std/bin/lff ~ -k -f 2m -d 6m -o ~/public
That example would scan your home directory, output the file sizes in kilobytes, search for files 2 megabytes or larger, directories 6 megabytes or larger, and omit the public directory.

It is possible to scan and/or omit multiple directories. To scan /tmp and /usr while omitting /usr/X11R6 and /usr/bin, the following command will work:
/s/std/bin/lff /tmp /usr -o /usr/X11R6 -o /usr/bin

Recursively Copying Files And AFS ACLs on Linux systems

Use the up command:
up <srcdir> <destdir>

srcdir is the source directory, destdir is the destination directory.

If destdir does not exist, it will be created. If destdir does exist, it will be the new directory. For example:

  • up olddir newdir: makes a copy of olddir, named newdir
  • mkdir newdir; up olddir newdir: makes a copy of olddir, named newdir, not newdir/olddir

Access Control and AFS Groups

The AFS Filesystem provides Access Control Lists (ACLs) and user-managed AFS user groups for controlling file access.

Access Control Lists (ACLs)

Each directory in AFS has an ACL (Access Control List). Although unix files have owner and group IDs associated with them and read/write/execute bits for the owner, group members and non-members, these bits are often ignored for files on AFS filesystems. (You will need to enable execute bits to execute programs, though, and some programs will explicitly test for certain unix file permission bits.)

The ACL of a directory applies to all files in that directory. For example, if the ACL allows read access to a user, then all readable files (according to the Unix owner permission bits) in the directory will be readable by that user. The same applies to write and execute permissions (note: in order to execute a file (run the program), you must also be able to read the file). It is impossible to have two files in a given directory with different ACLs.

Access can be specified for individual users or AFS groups.

Access Control Commands

On Linux systems

Use the fs listacl command:

fs listacl <path>
fs la <path>
path is the pathname of the directory.

On Windows systems

Right click on a directory, and use the AFS | Access Control Lists menu; or
In a command window, use the fs listacl command as shown above for Linux systems.

Setting/Modifying Access Control Lists

On Linux systems

Use the fs settacl command:

fs setacl
fs sa

  • path is the pathname of the directory
  • user is the username of an individual user, or an AFS user group
  • permission is an AFS Access Control List permission (right) to be granted

For example:
Give the user bbadger read and lookup access to all the readable files in the directory project2:

fs setacl project2 bbadger read; or
fs setacl project2 bbadger rl

Give the group username:mygroup rlidwk (read, lookup, insert, delete, write, and lock) access permissions to the directory src (username is your user name):

fs setacl src :mygroup write; or
fs setacl src :mygroup rlidwk

Remove all access permissions for the group system:anyuser:

fs setacl private system:anyuser none

On Windows systems

  • Right-click on a directory, and use the AFS | Access Control Lists menu; or
  • In a command window, use the fs setacl command as shown above for Linux systems.

Access Permissions (Rights)

The following permissions can be specified in AFS ACLs:

Permission Attribute Description
r read allows a user to look at readable files in the directory.
l lookup a user with this right may list a directory, look at an ACL or access subdirectories.
w write allows a user to modify writable files in the directory. Should never be granted to system:anyuser (see System-Defined AFS Groups)
d delete allows a user to remove files from a directory. Should never be granted to system:anyuser (see System-Defined AFS Groups)
i insert allows a user to add files to a directory. Should never be granted to system:anyuser (see System-Defined AFS Groups)
k lock allows a user to put advisory file locks on a file in the directory. Should never be granted to system:anyuser (see System-Defined AFS Groups)
a administer allows a user to change the ACLs on the directory. If a user has administer access to a directory, that user may change the ACLs of that directory and any directories within that directory. For that reason, we do not suggest granting this access to anyone for your personal directories. Should never be granted to system:anyuser (see System-Defined AFS Groups)

In addition, the following aggregate permissions can be used:

Aggregate Permission
read read (r) and lookup (l) permissions
write read (r), lookup (l), insert (i), write (w), delete (d) and lock (k) permissions. Should never be granted to system:anyuser (see System-Defined AFS Groups)
all grant all permissions (rliwdka), including administer. Should never be granted to system:anyuser (see System-Defined AFS Groups)
none remove all permissions for the specified user or group

AFS Groups

User-Managed AFS Groups

AFS allows users to create and manage their own groups of users. AFS groups can be named in ACLs (Access Control Lists), removing the need to change every ACL when someone joins or leaves a logical group of users.

AFS groups that are owned by users are named username:groupname

System-Defined AFS Groups

The AFS system defines the following system groups:

  • system:anyuser: any user using AFS anywhere in the world, regardless of authentication status
    • system:anyuser should never be granted access beyond read and lookup to locations in AFS. If you have run into a situation where you think you need to grant other permissions, please contact lab@cs.wisc.edu to discuss alternatives.
  • system:authuser: any user (or process) that has authenticated to the local AFS system
  • system:administrators: the local system administration group

AFS Group Commands

Listing AFS Groups

Use the following pts commands (run from a command prompt window on Windows workstations):

Command Description
pts membership <username> List the groups <username> is a member of
pts membership <username:group> List the members of group <username:group>
pts listowned <username> List the groups owned by <username>

For example:

  • pts membership bbadger: Lists the groups that bbadger is a member of
  • pts membership bbadger:workgroup: Lists the members of bbadger's group bbadger:workgroup

Creating/Modifying AFS Groups

Use the following pts commands (run from a command prompt window on Windows workstations):

Command Description
pts creategroup <username:group> Create group <username:group&gt;. <username> must be your username.
pts adduser <username> <username:group> Add user <username> to group <username:group>. Usually <username> is your username.
pts removeuser <username> <username:group> Remove user <username> from group <username:group>. Usually <username> is your username.

For example:

  • pts creategroup bbadger:workgroup: The user bbadger creates a group named bbadger:workgroup.
  • pts adduser bucky bbadger:workgroup: Add user bucky to bbadger's group bbadger:workgroup
  • pts removeuser bucky bbadger:workgroup: Remove user bucky from bbadger's group bbadger:workgroup

AFS Authentication

In order to use the AFS file system, you must be authenticated to AFS. AFS uses Kerberos for authentication.

Tokens

AFS uses an authentication token. Tokens are issued by the login process, and are only valid for a certain period of time.

Use the tokens command (run in a command prompt window on Windows 7 systems):
tokens: Prints the AFS user ID and token's expiration time.

Reauthentication

To reauthenticate (renew or extend an AFS token):

On Linux systems

Use the kalog command:
kalog: You will be prompted for your password.

On Windows 7 systems

  • Click on the AFS icon (yellow padlock) or Network Identity Manager icon (empty box or box with a sillouette inside) to open the Network Identity Manager
  • Select your Kerberos ticket (if present), then right-click and select Renew credentials
  • If you do not have a Kerberos ticket displayed under Network Identity Manager, you will need to go to CredentialObtain new credentials and reauthenticate. Use <LOGIN>@CS.WISC.EDU as your username where LOGIN is your login name.

AFS on Remote Computers

AFS clients are available for various platforms, including Windows, Mac, and Linux systems.

This is a brief summary of how to get a working OpenAFS client installed for a computer running Windows communicating with the AFS infrastructure in the Computer Sciences Department. It is not intended to be a general cookbook for how to configure Windows clients anywhere (although you can use the AFS client with other cells if you like).

  1. Download the necessary software:
    • Download the OpenAFS client from the OpenAFS for Windows website. You have a choice between the 32-bit and 64-bit versions; make sure you get the correct installer for your Windows installation. It is best to use 64-bit releases for 64-bit Windows; 32-bit Windows computers must use 32-bit clients. Do not install the software yet.
    • Download Kerberos for Windows for your computer. Again, you will need to make sure you get the 32-bit or 64-bit release depending on the type of Windows installation you are using. Both 32-bit and 64-bit builds of Kerberos for Windows are available from Secure Endpoints. Only 32-bit KfW releases are available from the MIT Kerberos Distribution Page.
    • Download our krb5.ini config file; this will configure your Kerberos for Windows installation to use the Computer Sciences Department kerberos authentication servers later on.
  2. Start the installation
    • Install Kerberos for Windows; choose a typical configuration if asked. This install does not require much for user input.
    • Copy the krb5.ini file you downloaded earlier to C:\WINDOWS\krb5.ini; this will replace the default krb5.ini configured to talk to authentication servers at MIT.
    • Install the OpenAFS client
      • Choose a typical installation
      • When prompted, you should change the cell name to cs.wisc.edu and disable the integrated login feature
      • Restart the computer when the installation is complete
  3. Try using the client
    • You should find an AFS lock icon in the system tray after the system reboots; opening it will give you a window where you can enter your Computer Sciences username and password
    • Computer Sciences department files are accessible under \\afs\cs.wisc.edu
    • You can make drive mappings to locations within \\afs\cs.wisc.edu if you want

Note: using OpenAFS and enabling/disabling network interfaces: VPN users -- this applies to you as well. The OpenAFS client does not operate properly after:

  • adding a network interface
  • removing a network interface
  • changing the IP address of a network interface currently in use

This includes starting or stopping a VPN session. We suggest you test this behavior; if you find your explorer window hangs, stop the OpenAFS client service before changing network interfaces or opening/closing VPN connections. You can restart the OpenAFs client service once the network connection changes have occurred. Some users report this is only a problem for 1-2 minutes after a network change; others report they can not browse files in AFS directories after this occurs. Development on the OpenAFS client code is ongoing to mitigate this bug.