CPSC 461: Copyright (C) 2003 Katrin Becker 1998-2002 Last Modified June 10, 2003 01:45 PM

UNIX File System

What is a File System? - a method for cataloging files in a computer system Examples: FAT (DOS) UNIX: NFS (Network File System) 32/64 bit

EFS (Extent File System) - SGI's 32-bit BSDFAST/UFS (Fast File System) XFS (SGI's 64-bit journalled file system; see http://www-europe.sgi.com/Technology/xfs-whitepaper.html) 4.3 BSD - defacto SUN FS 32-bit

What does it have? - files - directories - devices What is it responsible for? - organization of all stuff stored on a system - the file system is a single logical entity that controls access both to ordinary data files and to all I/O devices - on UNIX all I/O is performed on files, some of which are actually devices - the kernel maintains a single system wide file system, and all user I/O requests refer to "files" within this system - the kernel: does the physical stuff (as opposed to the logical) - 'lives' below the top level (logical view) where all files are treated as strings of bytes

UNIX contains 5 types of files:

regular files : regular finite stings of bytes (use block I/O) directories : files that provide a mapping between the names of the files and the files themselves special files : like I/O streams; printers, etc. (use character I/O)

device files can be of two types:

character (terminals and printers) block (like disks and tape)

symbolic links and other special files : (like pipes ) sockets : abstractions - endpoints for inter process communication (use network I/O) the naming syntax for the first three is the same

we'll do block I/O : normal file I/O - addressable array of fixed size (512/1024... bytes) blocks device drivers : set of routines; one for each device - like the I/O processor

Kernel and File Systems

UNIX file system is collection of files and related info (directories, ordinary files, inodes) file system is part of kernel but still separate; resides on disk and brought in as needed; makes it independent of how kernel views files provides level of indirection that allows kernel to treat very different devices in essentially the same way treats tape as block I/O - allows only one write request at a time per drive

Directories: What are they?

- way to organize files - index that tells us where to find files

- has entries that tell us about files

- it is a file of records

(size of a directory entry can be found out by looking at:

/usr/include/sys/dirent.h, part of which is displayed below:)

What does a directory entry look like?

pointer to an inode size length of name name itself (max 255)

These records are kept in chunks (== block)

- access is a single operation - makes it atomic - this is essential

- must be fast; - cannot be allowed to be interrupted

    /* Copyright (c) 1984, 1986, 1987, 1988, 1989 AT&T */

`/* All Rights Reserved */`
`/* THIS IS UNPUBLISHED PROPRIETARY SOURCE CODE OF AT&T */`
`/* The copyright notice above does not evidence any */`
`/* actual or intended publication of such source code. */`
`#ifndef _SYS_DIRENT_H` `#define _SYS_DIRENT_H`
`#pragma ident "@(#)dirent.h 1.15 94/07/26 SMI" /* SVr4.0 11.11 */`
`#include <sys/feature_tests.h>`
`#ifdef __cplusplus` `extern "C" {`
`#endif` `/*` `* File-system independent directory entry.` `*/` `struct dirent {`
`ino_t d_ino; /* "inode number" of entry */`
`off_t d_off; /* offset of disk directory entry */`
`unsigned short d_reclen; /* length of this record */`
`char d_name[1]; /* name of file */`
`};`
`typedef struct dirent dirent_t;`
`#if (!defined(_POSIX_C_SOURCE) && !defined(_XOPEN_SOURCE)) || \` `defined(__EXTENSIONS__)`
`#if !defined(_KERNEL)`
`#if defined(__STDC__)`
`int getdents(int, struct dirent *, unsigned);`
`#else` `int getdents();`
`#endif`
`#endif /* !defined(_KERNEL) */`
`#endif /* (!defined(_POSIX_C_SOURCE) && !defined(_XOPEN_SOURCE)) ... */`
`#ifdef __cplusplus` `}`
`#endif`
`#endif /* _SYS_DIRENT_H */`

What is in a directory?

- when we do ls -al we get:

. == current directory .. == parent directory others

What is a link? { ln }

- there are 2 kinds: hard and soft - hard links cannot cross file systems When you do ls -l, after the permissions is the # of hard links into the file Symbolic links can cross files system boundaries.

Try This:

mkdir foo cd foo touch a {make a file ls -li {note the inode# -the

{ first # on the line

ln a b ls -li ln -s a c ls -li la -al { look at . and .. rm a ls -li { what happens to a, b, c A hard link is the Nth (N > 1) directory entry to a file. Every directory entry points to an inode ; a hard link points to one that was previously created

Script started on Tue Jun 10 13:14:44 2003 gromit% pwd /home/profs/becker/foo gromit% ls typescript gromit% ls -li total 0 46237 -rw------- 1 becker profs 0 Jun 10 13:14 typescript gromit% touch a gromit% ls -li total 0 46493 -rw------- 1 becker profs 0 Jun 10 13:15 a 46237 -rw------- 1 becker profs 0 Jun 10 13:14 typescript gromit% ln a b gromit% ls -li total 0 46493 -rw------- 2 becker profs 0 Jun 10 13:15 a 46493 -rw------- 2 becker profs 0 Jun 10 13:15 b 46237 -rw------- 1 becker profs 0 Jun 10 13:14 typescript gromit% ls -la total 10 drwx------ 2 becker profs 96 Jun 10 13:15 ./ drwxr-x--x 54 becker profs 5120 Jun 10 13:02 ../ -rw------- 2 becker profs 0 Jun 10 13:15 a -rw------- 2 becker profs 0 Jun 10 13:15 b -rw------- 1 becker profs 0 Jun 10 13:14 typescript gromit% ls -lia total 10 13196 drwx------ 2 becker profs 96 Jun 10 13:15 ./ 10869 drwxr-x--x 54 becker profs 5120 Jun 10 13:02 ../ 46493 -rw------- 2 becker profs 0 Jun 10 13:15 a 46493 -rw------- 2 becker profs 0 Jun 10 13:15 b 46237 -rw------- 1 becker profs 0 Jun 10 13:14 typescript gromit% ls -s a c c: No such file or directory 0 a gromit% ln -s a c gromit% ls -lia total 10 13196 drwx------ 2 becker profs 96 Jun 10 13:16 ./ 10869 drwxr-x--x 54 becker profs 5120 Jun 10 13:02 ../ 46493 -rw------- 2 becker profs 0 Jun 10 13:15 a 46493 -rw------- 2 becker profs 0 Jun 10 13:15 b 46463 lrwxrwxrwx 1 becker profs 1 Jun 10 13:16 c -> a 46237 -rw------- 1 becker profs 0 Jun 10 13:14 typescript gromit% rm a rm: remove a (yes/no)? y gromit% ls -lia total 10 13196 drwx------ 2 becker profs 96 Jun 10 13:16 ./ 10869 drwxr-x--x 54 becker profs 5120 Jun 10 13:02 ../ 46493 -rw------- 1 becker profs 0 Jun 10 13:15 b 46463 lrwxrwxrwx 1 becker profs 1 Jun 10 13:16 c -> a 46237 -rw------- 1 becker profs 0 Jun 10 13:14 typescript gromit% cat a cat: cannot open a gromit% cat b gromit% cat c cat: cannot open c gromit% mv b d gromit% ls -lia total 10 13196 drwx------ 2 becker profs 96 Jun 10 13:17 ./ 10869 drwxr-x--x 54 becker profs 5120 Jun 10 13:02 ../ 46463 lrwxrwxrwx 1 becker profs 1 Jun 10 13:16 c -> a 46493 -rw------- 1 becker profs 0 Jun 10 13:15 d 46237 -rw------- 1 becker profs 0 Jun 10 13:14 typescript gromit% cat a d gromit% mv d a gromit% ls -lia total 10 13196 drwx------ 2 becker profs 96 Jun 10 13:18 ./ 10869 drwxr-x--x 54 becker profs 5120 Jun 10 13:02 ../ 46493 -rw------- 1 becker profs 0 Jun 10 13:15 a 46463 lrwxrwxrwx 1 becker profs 1 Jun 10 13:16 c -> a 46237 -rw------- 1 becker profs 0 Jun 10 13:14 typescript gromit% cat c gromit% and now.... magic happens gromit% ls -lia total 16 13196 drwx------ 2 becker profs 96 Jun 10 13:22 ./ 10869 drwxr-x--x 54 becker profs 5120 Jun 10 13:02 ../ 46460 -rw------- 1 becker profs 28 Jun 10 13:21 a 46493 -rw------- 1 becker profs 0 Jun 10 13:15 a~ 46536 -rw------- 1 becker profs 23 Jun 10 13:22 b 46463 lrwxrwxrwx 1 becker profs 1 Jun 10 13:16 c -> a 46743 -rw------- 1 becker profs 25 Jun 10 13:22 d 46237 -rw------- 1 becker profs 0 Jun 10 13:14 typescript gromit% cat a aaaaaaaaaaaaaaaaaaaaaaaaaaa gromit% cat b bbbbbbbbbbbbbbbbbbbbbb gromit% cat c aaaaaaaaaaaaaaaaaaaaaaaaaaa gromit% cat d DDDDDDDDDDDDDDDDDDDDDDDD gromit% rm a rm: remove a (yes/no)? y gromit% cat c cat: cannot open c gromit% mv d a gromit% cat c DDDDDDDDDDDDDDDDDDDDDDDD gromit% exit script done on Tue Jun 10 13:23:43 2003

A soft link has its own inode ; the inode points to a file; what's inside the file is simply a file name (essentially text) Try a move operation: ls -li; mv b d; ls -li So, what is an inode?

try /usr/include/sys/fs/ufs_inode.h it contains:

uid of owner gid of group mode u-g-o time stamps:

last mod last access last inode mod

# links byte count block count (disk blocks allocated to the file) DAB (direct access blocks)

there are 10-12 direct pointers that point to blocks of the file

IAB (indirect address blocks)

single indirect - points to a 4K block of pointers;

each of which points to a block of DATA

double indirect - points to a 4K block of indirect pointers,

each of which points to another block of pointers each of which which points to DATA

triple indirect - points to a block of pointers

each of which points to another block of pointers and each of these points to a block of direct pointers each of which which points to DATA

When you do ls -a you get:

-rwxrwxrwx what's in the first position?

d = directory l = link b = block device c = character device p = pipe

The File System decides on the block size: ~2K, 4K, 8K It turns out that by using only the first, single indirect pointer block we are able to fully address over 90% of all files. On UNIX (under BSD 4.3) the biggest file is about 2GB How big can a file be????? On the Disk:

Partition: a grouping of cylinders

- has a Superblock

- block size 8K max. - this is how much gets read in one operation

(minimum is 2K chunks, also called "fragments"

- is sub-divided into groups (16 cyl/ group) - each group has

- a copy of the superblock - space for inodes - a list of available fragments (bitmap)

(need 16bits/block since we can have 2-4-8-16 chunks) - so smallest addressable space is 512bytes

- an inode pointer can point to blocks or to chunks within blocks

aside: What is "thrashing"? aside: How come everything stops when you do a read or write from/to a disk in DOS? To save a new file:

- gets an inode - look in the free list for available fragments - if the file grows; will give contiguous fragments if possible

inodes are allocated when the file system is created (a static value)

inodes/partition = size of partition/2048

Sometimes we need to "de-frag" To mount a file system:

mount point is / inode for /usr has a "hook" which says go look on device "X" when a file system is mounted the information from the superblock is written into memory

CPSC 461: Copyright (C) 2003 Katrin Becker 1998-2002 Last Modified June 10, 2003 01:45 PM