linux/fs
David Howells 9f10523f89 FS-Cache: Fix operation state management and accounting
Fix the state management of internal fscache operations and the accounting of
what operations are in what states.

This is done by:

 (1) Give struct fscache_operation a enum variable that directly represents the
     state it's currently in, rather than spreading this knowledge over a bunch
     of flags, who's processing the operation at the moment and whether it is
     queued or not.

     This makes it easier to write assertions to check the state at various
     points and to prevent invalid state transitions.

 (2) Add an 'operation complete' state and supply a function to indicate the
     completion of an operation (fscache_op_complete()) and make things call
     it.  The final call to fscache_put_operation() can then check that an op
     in the appropriate state (complete or cancelled).

 (3) Adjust the use of object->n_ops, ->n_in_progress, ->n_exclusive to better
     govern the state of an object:

	(a) The ->n_ops is now the number of extant operations on the object
	    and is now decremented by fscache_put_operation() only.

	(b) The ->n_in_progress is simply the number of objects that have been
	    taken off of the object's pending queue for the purposes of being
	    run.  This is decremented by fscache_op_complete() only.

	(c) The ->n_exclusive is the number of exclusive ops that have been
	    submitted and queued or are in progress.  It is decremented by
	    fscache_op_complete() and by fscache_cancel_op().

     fscache_put_operation() and fscache_operation_gc() now no longer try to
     clean up ->n_exclusive and ->n_in_progress.  That was leading to double
     decrements against fscache_cancel_op().

     fscache_cancel_op() now no longer decrements ->n_ops.  That was leading to
     double decrements against fscache_put_operation().

     fscache_submit_exclusive_op() now decides whether it has to queue an op
     based on ->n_in_progress being > 0 rather than ->n_ops > 0 as the latter
     will persist in being true even after all preceding operations have been
     cancelled or completed.  Furthermore, if an object is active and there are
     runnable ops against it, there must be at least one op running.

 (4) Add a remaining-pages counter (n_pages) to struct fscache_retrieval and
     provide a function to record completion of the pages as they complete.

     When n_pages reaches 0, the operation is deemed to be complete and
     fscache_op_complete() is called.

     Add calls to fscache_retrieval_complete() anywhere we've finished with a
     page we've been given to read or allocate for.  This includes places where
     we just return pages to the netfs for reading from the server and where
     accessing the cache fails and we discard the proposed netfs page.

The bugs in the unfixed state management manifest themselves as oopses like the
following where the operation completion gets out of sync with return of the
cookie by the netfs.  This is possible because the cache unlocks and returns
all the netfs pages before recording its completion - which means that there's
nothing to stop the netfs discarding them and returning the cookie.


FS-Cache: Cookie 'NFS.fh' still has outstanding reads
------------[ cut here ]------------
kernel BUG at fs/fscache/cookie.c:519!
invalid opcode: 0000 [#1] SMP
CPU 1
Modules linked in: cachefiles nfs fscache auth_rpcgss nfs_acl lockd sunrpc

Pid: 400, comm: kswapd0 Not tainted 3.1.0-rc7-fsdevel+ #1090                  /DG965RY
RIP: 0010:[<ffffffffa007050a>]  [<ffffffffa007050a>] __fscache_relinquish_cookie+0x170/0x343 [fscache]
RSP: 0018:ffff8800368cfb00  EFLAGS: 00010282
RAX: 000000000000003c RBX: ffff880023cc8790 RCX: 0000000000000000
RDX: 0000000000002f2e RSI: 0000000000000001 RDI: ffffffff813ab86c
RBP: ffff8800368cfb50 R08: 0000000000000002 R09: 0000000000000000
R10: ffff88003a1b7890 R11: ffff88001df6e488 R12: ffff880023d8ed98
R13: ffff880023cc8798 R14: 0000000000000004 R15: ffff88003b8bf370
FS:  0000000000000000(0000) GS:ffff88003bd00000(0000) knlGS:0000000000000000
CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
CR2: 00000000008ba008 CR3: 0000000023d93000 CR4: 00000000000006e0
DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
Process kswapd0 (pid: 400, threadinfo ffff8800368ce000, task ffff88003b8bf040)
Stack:
 ffff88003b8bf040 ffff88001df6e528 ffff88001df6e528 ffffffffa00b46b0
 ffff88003b8bf040 ffff88001df6e488 ffff88001df6e620 ffffffffa00b46b0
 ffff88001ebd04c8 0000000000000004 ffff8800368cfb70 ffffffffa00b2c91
Call Trace:
 [<ffffffffa00b2c91>] nfs_fscache_release_inode_cookie+0x3b/0x47 [nfs]
 [<ffffffffa008f25f>] nfs_clear_inode+0x3c/0x41 [nfs]
 [<ffffffffa0090df1>] nfs4_evict_inode+0x2f/0x33 [nfs]
 [<ffffffff810d8d47>] evict+0xa1/0x15c
 [<ffffffff810d8e2e>] dispose_list+0x2c/0x38
 [<ffffffff810d9ebd>] prune_icache_sb+0x28c/0x29b
 [<ffffffff810c56b7>] prune_super+0xd5/0x140
 [<ffffffff8109b615>] shrink_slab+0x102/0x1ab
 [<ffffffff8109d690>] balance_pgdat+0x2f2/0x595
 [<ffffffff8103e009>] ? process_timeout+0xb/0xb
 [<ffffffff8109dba3>] kswapd+0x270/0x289
 [<ffffffff8104c5ea>] ? __init_waitqueue_head+0x46/0x46
 [<ffffffff8109d933>] ? balance_pgdat+0x595/0x595
 [<ffffffff8104bf7a>] kthread+0x7f/0x87
 [<ffffffff813ad6b4>] kernel_thread_helper+0x4/0x10
 [<ffffffff81026b98>] ? finish_task_switch+0x45/0xc0
 [<ffffffff813abcdd>] ? retint_restore_args+0xe/0xe
 [<ffffffff8104befb>] ? __init_kthread_worker+0x53/0x53
 [<ffffffff813ad6b0>] ? gs_change+0xb/0xb

Signed-off-by: David Howells <dhowells@redhat.com>
2012-12-20 21:58:26 +00:00
..
9p The following changes since commit 4cbe5a555f: 2012-10-12 09:59:23 +09:00
adfs Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs 2012-10-02 20:25:04 -07:00
affs Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs 2012-10-02 20:25:04 -07:00
afs Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs 2012-10-02 20:25:04 -07:00
autofs4 Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/ebiederm/user-namespace 2012-12-17 15:44:47 -08:00
befs Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs 2012-10-02 20:25:04 -07:00
bfs Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs 2012-10-02 20:25:04 -07:00
btrfs Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/mason/linux-btrfs 2012-12-18 09:42:05 -08:00
cachefiles FS-Cache: Fix operation state management and accounting 2012-12-20 21:58:26 +00:00
ceph ceph: fix dentry reference leak in ceph_encode_fh() 2012-12-18 15:02:11 -08:00
cifs lseek: the "whence" argument is called "whence" 2012-12-17 17:15:12 -08:00
coda fs: push rcu_barrier() from deactivate_locked_super() to filesystems 2012-10-02 21:35:55 -04:00
configfs lseek: the "whence" argument is called "whence" 2012-12-17 17:15:12 -08:00
cramfs userns: Convert cramfs to use kuid/kgid where appropriate 2012-09-21 03:13:08 -07:00
debugfs fs/debugsfs: remove unnecessary inode->i_private initialization 2012-11-15 17:46:42 -08:00
devpts TTY: devpts, document devpts inode operations 2012-10-22 16:50:13 -07:00
dlm dlm: fix lvb invalidation conditions 2012-11-16 11:20:42 -06:00
ecryptfs Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs 2012-10-02 20:25:04 -07:00
efs Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs 2012-10-02 20:25:04 -07:00
exofs exofs: don't leak io_state and pages on read error 2012-12-14 12:17:32 +02:00
exportfs fs, exportfs: add exportfs_encode_inode_fh() helper 2012-12-17 17:15:27 -08:00
ext2 ext2: fix return values on parse_options() failure 2012-10-09 23:23:53 +02:00
ext3 lseek: the "whence" argument is called "whence" 2012-12-17 17:15:12 -08:00
ext4 lseek: the "whence" argument is called "whence" 2012-12-17 17:15:12 -08:00
fat fs/fat: strip "cp" prefix from codepage in display 2012-12-17 17:15:22 -08:00
freevxfs Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs 2012-10-02 20:25:04 -07:00
fscache FS-Cache: Fix operation state management and accounting 2012-12-20 21:58:26 +00:00
fuse Merge branch 'akpm' (Andrew's patch-bomb) 2012-12-17 20:58:12 -08:00
gfs2 lseek: the "whence" argument is called "whence" 2012-12-17 17:15:12 -08:00
hfs Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs 2012-10-02 20:25:04 -07:00
hfsplus Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs 2012-10-02 20:25:04 -07:00
hostfs Merge branch 'for-linus-37rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/rw/uml 2012-10-10 11:15:20 +09:00
hpfs hpfs: drop lock/unlock super 2012-10-09 23:33:38 -04:00
hppfs pidns: Use task_active_pid_ns where appropriate 2012-11-19 05:59:09 -08:00
hugetlbfs Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/jikos/trivial 2012-12-13 12:00:02 -08:00
isofs tmpfs,ceph,gfs2,isofs,reiserfs,xfs: fix fh_len checking 2012-10-09 23:33:55 -04:00
jbd Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/jikos/trivial 2012-12-13 12:00:02 -08:00
jbd2 There are two major features for this merge window. The first is 2012-12-16 17:33:01 -08:00
jffs2 jffs2: hold erase_completion_lock on exit 2012-11-18 11:59:01 +02:00
jfs jfs: Fix FITRIM argument handling 2012-10-17 09:18:38 -05:00
lockd lockd: Remove BUG_ON()s from fs/lockd/clntproc.c 2012-11-04 14:43:40 -05:00
logfs Fix misspellings of "whether" in comments. 2012-11-19 14:31:35 +01:00
minix Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs 2012-10-02 20:25:04 -07:00
ncpfs propagate name change to comments in kernel source 2012-12-06 10:39:54 +01:00
nfs NFS client updates for Linux 3.8 2012-12-18 09:36:34 -08:00
nfs_common
nfsd UAPI Disintegration 2012-10-09 2012-10-09 18:35:22 -04:00
nilfs2 mm: redefine address_space.assoc_mapping 2012-12-11 17:22:26 -08:00
nls nls: fix (and rename) mac NLS table files and config options 2012-06-01 19:51:22 -07:00
notify fs, fanotify: add @mflags field to fanotify output 2012-12-17 17:15:28 -08:00
ntfs Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs 2012-10-02 20:25:04 -07:00
ocfs2 lseek: the "whence" argument is called "whence" 2012-12-17 17:15:12 -08:00
omfs omfs: convert to use beXX_add_cpu() 2012-10-06 03:05:31 +09:00
openpromfs fs: push rcu_barrier() from deactivate_locked_super() to filesystems 2012-10-02 21:35:55 -04:00
proc Merge branch 'akpm' (Andrew's patch-bomb) 2012-12-17 20:58:12 -08:00
pstore lseek: the "whence" argument is called "whence" 2012-12-17 17:15:12 -08:00
qnx4 Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs 2012-10-02 20:25:04 -07:00
qnx6 Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs 2012-10-02 20:25:04 -07:00
quota quota: Use the pre-processor to compile out quotactl_cmd_write when !CONFIG_BLOCK 2012-12-13 16:33:24 +01:00
ramfs don't pass nameidata to ->create() 2012-07-14 16:34:47 +04:00
reiserfs reiserfs: Move quota calls out of write lock 2012-11-19 21:34:33 +01:00
romfs fs: push rcu_barrier() from deactivate_locked_super() to filesystems 2012-10-02 21:35:55 -04:00
squashfs Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs 2012-10-02 20:25:04 -07:00
sysfs Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/ebiederm/user-namespace 2012-12-17 15:44:47 -08:00
sysv sysv: drop lock/unlock super 2012-10-09 23:33:39 -04:00
ubifs ubifs: use prandom_bytes 2012-12-17 17:15:26 -08:00
udf udf: remove un-needed variable from inode_getblk 2012-12-13 16:33:23 +01:00
ufs ufs: drop lock/unlock super 2012-10-09 23:33:39 -04:00
xfs xfs: fix sparse reported log CRC endian issue 2012-12-03 12:10:59 -06:00
aio.c aio: now fput() is OK from interrupt context; get rid of manual delayed __fput() 2012-07-22 23:57:59 +04:00
anon_inodes.c
attr.c userns: Allow chown and setgid preservation 2012-11-20 04:17:24 -08:00
bad_inode.c lseek: the "whence" argument is called "whence" 2012-12-17 17:15:12 -08:00
binfmt_aout.c get rid of pt_regs argument of ->load_binary() 2012-11-28 21:53:38 -05:00
binfmt_elf_fdpic.c get rid of pt_regs argument of ->load_binary() 2012-11-28 21:53:38 -05:00
binfmt_elf.c binfmt_elf: fix corner case kfree of uninitialized data 2012-12-17 17:15:19 -08:00
binfmt_em86.c exec: use -ELOOP for max recursion depth 2012-12-17 17:15:23 -08:00
binfmt_flat.c get rid of pt_regs argument of ->load_binary() 2012-11-28 21:53:38 -05:00
binfmt_misc.c exec: use -ELOOP for max recursion depth 2012-12-17 17:15:23 -08:00
binfmt_script.c exec: use -ELOOP for max recursion depth 2012-12-17 17:15:23 -08:00
binfmt_som.c get rid of pt_regs argument of ->load_binary() 2012-11-28 21:53:38 -05:00
bio-integrity.c block: Ues bi_pool for bio_integrity_alloc() 2012-09-09 10:35:38 +02:00
bio.c vfs: fix: don't increase bio_slab_max if krealloc() fails 2012-10-22 22:00:26 +02:00
block_dev.c lseek: the "whence" argument is called "whence" 2012-12-17 17:15:12 -08:00
buffer.c fs/buffer.c: remove redundant initialization in alloc_page_buffers() 2012-12-12 17:38:35 -08:00
char_dev.c char_dev: pin parent kobject 2012-10-22 08:50:37 +03:00
compat_binfmt_elf.c coredump: extend core dump note section to contain file names of mapped files 2012-10-06 03:05:17 +09:00
compat_ioctl.c Merge 3.7-rc3 into tty-next 2012-10-29 09:00:57 -07:00
compat.c vfs: define struct filename and have getname() return it 2012-10-12 20:14:55 -04:00
coredump.c do_coredump(): get rid of pt_regs argument 2012-11-29 00:01:25 -05:00
coredump.h coredump: update coredump-related headers 2012-10-06 03:05:15 +09:00
dcache.c Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs 2012-10-02 20:25:04 -07:00
dcookies.c
direct-io.c direct-io: don't read inode->i_blkbits multiple times 2012-11-29 12:38:44 -08:00
drop_caches.c
eventfd.c fs, eventfd: add procfs fdinfo helper 2012-12-17 17:15:27 -08:00
eventpoll.c fs, epoll: add procfs fdinfo helper 2012-12-17 17:15:27 -08:00
exec.c Merge branch 'akpm' (Andrew's patch-bomb) 2012-12-17 20:58:12 -08:00
fcntl.c Fix F_DUPFD_CLOEXEC breakage 2012-10-09 15:52:31 +09:00
fhandle.c Fix misspellings of "whether" in comments. 2012-11-19 14:31:35 +01:00
fifo.c fifo: Do not restart open() if it already found a partner 2012-07-16 08:33:14 -07:00
file_table.c lglock: add DEFINE_STATIC_LGLOCK() 2012-10-10 01:15:44 -04:00
file.c Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/signal 2012-12-12 12:22:13 -08:00
filesystems.c vfs: define struct filename and have getname() return it 2012-10-12 20:14:55 -04:00
fs_struct.c kill daemonize() 2012-11-28 21:49:02 -05:00
fs-writeback.c writeback: fix a typo in comment 2012-12-12 17:38:34 -08:00
generic_acl.c userns: Pass a userns parameter into posix_acl_to_xattr and posix_acl_from_xattr 2012-09-18 01:01:35 -07:00
inode.c mm: redefine address_space.assoc_mapping 2012-12-11 17:22:26 -08:00
internal.h writeback: put unused inodes to LRU after writeback completion 2012-11-26 17:41:24 -08:00
ioctl.c switch simple cases of fget_light to fdget 2012-09-26 22:20:08 -04:00
ioprio.c
Kconfig ext4: Remove CONFIG_EXT4_FS_XATTR 2012-12-10 16:30:43 -05:00
Kconfig.binfmt coredump: make core dump functionality optional 2012-10-06 03:05:15 +09:00
libfs.c lseek: the "whence" argument is called "whence" 2012-12-17 17:15:12 -08:00
locks.c UAPI Disintegration 2012-10-09 2012-10-09 18:35:22 -04:00
Makefile coredump: make core dump functionality optional 2012-10-06 03:05:15 +09:00
mbcache.c
mount.h proc: Usable inode numbers for the namespace file descriptors. 2012-11-20 04:19:49 -08:00
mpage.c
namei.c lookup_one_len: don't accept . and .. 2012-11-29 22:17:21 -05:00
namespace.c userns: Require CAP_SYS_ADMIN for most uses of setns. 2012-12-14 16:12:03 -08:00
no-block.c
open.c vfs: Allow chroot if you have CAP_SYS_CHROOT in your user namespace 2012-11-19 05:59:17 -08:00
pipe.c pipe(2) - race-free error recovery 2012-09-26 21:08:52 -04:00
pnode.c VFS: Make clone_mnt()/copy_tree()/collect_mounts() return errors 2012-07-14 16:37:27 +04:00
pnode.h vfs: Only support slave subtrees across different user namespaces 2012-11-19 05:59:20 -08:00
posix_acl.c userns: Convert vfs posix_acl support to use kuids and kgids 2012-09-18 01:01:35 -07:00
proc_namespace.c get rid of magic in proc_namespace.c 2012-07-14 16:32:48 +04:00
read_write.c lseek: the "whence" argument is called "whence" 2012-12-17 17:15:12 -08:00
read_write.h compat: fs: Generic compat_sys_sendfile implementation 2012-10-02 21:35:55 -04:00
readdir.c switch simple cases of fget_light to fdget 2012-09-26 22:20:08 -04:00
select.c switch simple cases of fget_light to fdget 2012-09-26 22:20:08 -04:00
seq_file.c lseek: the "whence" argument is called "whence" 2012-12-17 17:15:12 -08:00
signalfd.c fs, epoll: add procfs fdinfo helper 2012-12-17 17:15:27 -08:00
splice.c writeback: remove nr_pages_dirtied arg from balance_dirty_pages_ratelimited_nr() 2012-12-11 17:22:21 -08:00
stack.c
stat.c Merge branch 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs 2012-10-02 20:25:04 -07:00
statfs.c switch simple cases of fget_light to fdget 2012-09-26 22:20:08 -04:00
super.c vfs: drop lock/unlock super 2012-10-09 23:33:39 -04:00
sync.c switch simple cases of fget_light to fdget 2012-09-26 22:20:08 -04:00
timerfd.c switch simple cases of fget_light to fdget 2012-09-26 22:20:08 -04:00
utimes.c switch simple cases of fget_light to fdget 2012-09-26 22:20:08 -04:00
xattr_acl.c userns: Fix posix_acl_file_xattr_userns gid conversion 2012-10-12 13:16:48 -07:00
xattr.c fs, xattr: fix bug when removing a name not in xattr list 2012-10-18 12:35:58 -07:00