Loading...
 

Filesystems General

Contents

Special files

Wikipedia description

Special file descriptor numbers are assigned to stdin (0), stdout (1) and stderr (2)

Redirecting of these can be done as follows:

redirect stderr to stdout - 2>&1

df and du

Why the numbers do not add up


The file system allocates some of the disk blocks in the file system to record its data. This data is referred to as meta data. Meta data is not visible to most user-level programs. Examples of meta data are inodes, disk maps, indirect blocks, and super blocks.

du is an example of a user-level program that is not aware of file system meta data, while df looks at the file system disk allocation maps and is aware of file system meta data. df obtains the true file system statistics, whereas du sees only a partial picture. For example, an empty 4MB JFS file system created with frag=4096 and nbpi=4096 has the following meta data allocated:

1 4k block for the LVM
2 4k super blocks
2 4k blocks for disk maps
2 4k blocks for inode maps
2 4k blocks for .indirect
32 4k blocks for inodes
---------
41 4k blocks for meta data on an empty 4MB file system

ls gives data on individual files based on the difference between the end-of-file (the largest offset where data is written) and the beginning-of-file, whether or not blocks were actually allocated to the file. A 32MB file (as reported by ls) may not have 32MB of data written to it if the data is not written sequentially.
du shows the blocks actually allocated to an individual file.
df shows the blocks allocated in the entire file system, including inodes and other meta data.
An example sparse file can be created fairly easily. To do so, open the file, seek to a large address, and write some data. This can be demonstrated with the dd command, as follows:

finding the differences

lsof can show unlinked files by using the ‘show less than N number of links:

  1. lsof +L1
COMMAND   PID   USER   FD   TYPE DEVICE        SIZE NLINK     NODE NAME                                                                                    
oracle  18617 oracle    2w   REG  253,0 29528674181     0 14811766 /u00/app/oracle/admin/ocarina/bdump/ocarina2_lmd0_18617.trc (deleted)
oracle  18617 oracle    5w   REG  253,0         898     0 14811763 /u00/app/oracle/admin/ocarina/udump/ocarina2_ora_18593.trc (deleted)
oracle  18617 oracle   10u   REG  253,0           0     0 13764568 /u00/app/oracle/product/10.2.0/db_1/dbs/lkinstocarina2 (deleted)


Also:

  1. ls -l /proc/18617/fd/
total 0
lr-x------ 1 oracle oinstall 64 Feb 23 04:30 0 -> /dev/null
lr-x------ 1 oracle oinstall 64 Feb 23 04:30 1 -> /dev/null
lrwx------ 1 oracle oinstall 64 Feb 23 04:30 10 -> /u00/app/oracle/product/10.2.0/db_1/dbs/lkinstocarina2 (deleted)
lr-x------ 1 oracle oinstall 64 Feb 23 04:30 11 -> /dev/zero
lrwx------ 1 oracle oinstall 64 Feb 23 04:30 12 -> /u00/app/oracle/product/10.2.0/db_1/dbs/hc_ocarina2.dat
lr-x------ 1 oracle oinstall 64 Feb 23 04:30 13 -> /dev/zero
lrwx------ 1 oracle oinstall 64 Feb 23 04:30 14 -> socket:[386827780]
lrwx------ 1 oracle oinstall 64 Feb 23 04:30 15 -> socket:[386827789]
l-wx------ 1 oracle oinstall 64 Feb 23 04:30 2 -> /u00/app/oracle/admin/ocarina/bdump/ocarina2_lmd0_18617.trc (deleted)
lr-x------ 1 oracle oinstall 64 Feb 23 04:30 3 -> /dev/null
lr-x------ 1 oracle oinstall 64 Feb 23 04:30 4 -> /dev/null
l-wx------ 1 oracle oinstall 64 Feb 23 04:30 5 -> /u00/app/oracle/admin/ocarina/udump/ocarina2_ora_18593.trc (deleted)
l-wx------ 1 oracle oinstall 64 Feb 23 04:30 6 -> /u00/app/oracle/admin/ocarina/bdump/alert_ocarina2.log
lrwx------ 1 oracle oinstall 64 Feb 23 04:30 7 -> /u00/app/oracle/product/10.2.0/db_1/dbs/hc_ocarina2.dat
l-wx------ 1 oracle oinstall 64 Feb 23 04:30 8 -> /u00/app/oracle/admin/ocarina/bdump/alert_ocarina2.log
lrwx------ 1 oracle oinstall 64 Feb 23 04:30 9 -> socket:[386827777]

Removing unlinked files

The filesystem recovers disk space through garbage collection. When files are busy the disk space is in use. When the reference count for a file is reduced to zero then the filesystem recovers the disk blocks associated with the file.

Removing files removes the directory pointer to that file but it does not remove the file from use by running processes. If a running process has the file open then the reference count will be non-zero due to use by that process and the filesystem will not be able to reclaim that disk space until the process exits.

A typical example is a program writing a log file. The log file may grow to be quite large. As long as the process is running the reference count for that file will be non-zero. Removing the file from the directory will not free up the disk space. In fact doing so means that there is no no way to access that file again. The only way
to reduce the reference count would be to kill the process that holds the file in use. To avoid this case it is better to truncate large files to free up the disk space immediately instead of removing them.

Open File Descriptors


Show PIDs with high numbers of OFFs (open file descriptors)
This is to check the number of actual file descriptors that access the filesystem (I.e. the ulimit setting)

for x in `ps --no-headers -fu ldap| awk '{ print $2 }'`;do num=`sudo ls /proc/$x/fd|wc -l|egrep "[0-9]{3,}"`;if [ $? -eq 0 ];then echo "PID = $x, OFDs = $num";fi;done
PID = 23648, OFDs = 988


for all users

[root@acontrol001 gbrown]# sudo ls /proc/*/fd|wc -l
1703
[root@acontrol001 gbrown]#

How to show what the FDs reference

for x in `ps --no-headers -fu ldap| awk '{ print $2 }'`;do num=`sudo ls /proc/$x/fd|wc -l|egrep "[0-9]{3,}"`;if [ $? -eq 0 ];then sudo ls -l /proc/$x/fd;fi;done

Show user limit

[root@acontrol001 gbrown]# cat /proc/23648/limits | grep "open files"
Max open files 1024 4096 files
[root@acontrol001 gbrown]#

Show system limit

[root@acontrol001 gbrown]# sysctl -A | egrep fs.*max
fs.file-max = 98372
fs.aio-max-nr = 65536
fs.inotify.max_user_instances = 128
fs.inotify.max_user_watches = 8192
fs.inotify.max_queued_events = 16384
fs.epoll.max_user_watches = 205250
fs.mqueue.queues_max = 256
fs.mqueue.msg_max = 10
fs.mqueue.msgsize_max = 8192
[root@acontrol001 gbrown]#