mirror of
https://github.com/torvalds/linux.git
synced 2024-12-27 21:33:00 +00:00
docs: Bring some order to filesystem documentation
Documentation/filesystems is, like much of the rest of the kernel's documentation, a jumble of unorganized information. Split the documentation into categories and try to bring some order to the top-level index.rst files. No text changes other than a few section-introductory blurbs; this is all just moving stuff around. Cc: linux-fsdevel@vger.kernel.org Cc: Al Viro <viro@zeniv.linux.org.uk> Signed-off-by: Jonathan Corbet <corbet@lwn.net>
This commit is contained in:
Jonathan Corbet
parent
866d65b9d7
commit
4064174bec
150
Documentation/filesystems/api-summary.rst
Normal file
150
Documentation/filesystems/api-summary.rst
Normal file
@ -0,0 +1,150 @@
|
||||
=============================
|
||||
Linux Filesystems API summary
|
||||
=============================
|
||||
|
||||
This section contains API-level documentation, mostly taken from the source
|
||||
code itself.
|
||||
|
||||
The Linux VFS
|
||||
=============
|
||||
|
||||
The Filesystem types
|
||||
--------------------
|
||||
|
||||
.. kernel-doc:: include/linux/fs.h
|
||||
:internal:
|
||||
|
||||
The Directory Cache
|
||||
-------------------
|
||||
|
||||
.. kernel-doc:: fs/dcache.c
|
||||
:export:
|
||||
|
||||
.. kernel-doc:: include/linux/dcache.h
|
||||
:internal:
|
||||
|
||||
Inode Handling
|
||||
--------------
|
||||
|
||||
.. kernel-doc:: fs/inode.c
|
||||
:export:
|
||||
|
||||
.. kernel-doc:: fs/bad_inode.c
|
||||
:export:
|
||||
|
||||
Registration and Superblocks
|
||||
----------------------------
|
||||
|
||||
.. kernel-doc:: fs/super.c
|
||||
:export:
|
||||
|
||||
File Locks
|
||||
----------
|
||||
|
||||
.. kernel-doc:: fs/locks.c
|
||||
:export:
|
||||
|
||||
.. kernel-doc:: fs/locks.c
|
||||
:internal:
|
||||
|
||||
Other Functions
|
||||
---------------
|
||||
|
||||
.. kernel-doc:: fs/mpage.c
|
||||
:export:
|
||||
|
||||
.. kernel-doc:: fs/namei.c
|
||||
:export:
|
||||
|
||||
.. kernel-doc:: fs/buffer.c
|
||||
:export:
|
||||
|
||||
.. kernel-doc:: block/bio.c
|
||||
:export:
|
||||
|
||||
.. kernel-doc:: fs/seq_file.c
|
||||
:export:
|
||||
|
||||
.. kernel-doc:: fs/filesystems.c
|
||||
:export:
|
||||
|
||||
.. kernel-doc:: fs/fs-writeback.c
|
||||
:export:
|
||||
|
||||
.. kernel-doc:: fs/block_dev.c
|
||||
:export:
|
||||
|
||||
.. kernel-doc:: fs/anon_inodes.c
|
||||
:export:
|
||||
|
||||
.. kernel-doc:: fs/attr.c
|
||||
:export:
|
||||
|
||||
.. kernel-doc:: fs/d_path.c
|
||||
:export:
|
||||
|
||||
.. kernel-doc:: fs/dax.c
|
||||
:export:
|
||||
|
||||
.. kernel-doc:: fs/direct-io.c
|
||||
:export:
|
||||
|
||||
.. kernel-doc:: fs/file_table.c
|
||||
:export:
|
||||
|
||||
.. kernel-doc:: fs/libfs.c
|
||||
:export:
|
||||
|
||||
.. kernel-doc:: fs/posix_acl.c
|
||||
:export:
|
||||
|
||||
.. kernel-doc:: fs/stat.c
|
||||
:export:
|
||||
|
||||
.. kernel-doc:: fs/sync.c
|
||||
:export:
|
||||
|
||||
.. kernel-doc:: fs/xattr.c
|
||||
:export:
|
||||
|
||||
The proc filesystem
|
||||
===================
|
||||
|
||||
sysctl interface
|
||||
----------------
|
||||
|
||||
.. kernel-doc:: kernel/sysctl.c
|
||||
:export:
|
||||
|
||||
proc filesystem interface
|
||||
-------------------------
|
||||
|
||||
.. kernel-doc:: fs/proc/base.c
|
||||
:internal:
|
||||
|
||||
Events based on file descriptors
|
||||
================================
|
||||
|
||||
.. kernel-doc:: fs/eventfd.c
|
||||
:export:
|
||||
|
||||
The Filesystem for Exporting Kernel Objects
|
||||
===========================================
|
||||
|
||||
.. kernel-doc:: fs/sysfs/file.c
|
||||
:export:
|
||||
|
||||
.. kernel-doc:: fs/sysfs/symlink.c
|
||||
:export:
|
||||
|
||||
The debugfs filesystem
|
||||
======================
|
||||
|
||||
debugfs interface
|
||||
-----------------
|
||||
|
||||
.. kernel-doc:: fs/debugfs/inode.c
|
||||
:export:
|
||||
|
||||
.. kernel-doc:: fs/debugfs/file.c
|
||||
:export:
|
||||
@ -1,389 +1,43 @@
|
||||
=====================
|
||||
Linux Filesystems API
|
||||
=====================
|
||||
===============================
|
||||
Filesystems in the Linux kernel
|
||||
===============================
|
||||
|
||||
The Linux VFS
|
||||
=============
|
||||
This under-development manual will, some glorious day, provide
|
||||
comprehensive information on how the Linux virtual filesystem (VFS) layer
|
||||
works, along with the filesystems that sit below it. For now, what we have
|
||||
can be found below.
|
||||
|
||||
The Filesystem types
|
||||
--------------------
|
||||
|
||||
.. kernel-doc:: include/linux/fs.h
|
||||
:internal:
|
||||
|
||||
The Directory Cache
|
||||
-------------------
|
||||
|
||||
.. kernel-doc:: fs/dcache.c
|
||||
:export:
|
||||
|
||||
.. kernel-doc:: include/linux/dcache.h
|
||||
:internal:
|
||||
|
||||
Inode Handling
|
||||
--------------
|
||||
|
||||
.. kernel-doc:: fs/inode.c
|
||||
:export:
|
||||
|
||||
.. kernel-doc:: fs/bad_inode.c
|
||||
:export:
|
||||
|
||||
Registration and Superblocks
|
||||
----------------------------
|
||||
|
||||
.. kernel-doc:: fs/super.c
|
||||
:export:
|
||||
|
||||
File Locks
|
||||
----------
|
||||
|
||||
.. kernel-doc:: fs/locks.c
|
||||
:export:
|
||||
|
||||
.. kernel-doc:: fs/locks.c
|
||||
:internal:
|
||||
|
||||
Other Functions
|
||||
---------------
|
||||
|
||||
.. kernel-doc:: fs/mpage.c
|
||||
:export:
|
||||
|
||||
.. kernel-doc:: fs/namei.c
|
||||
:export:
|
||||
|
||||
.. kernel-doc:: fs/buffer.c
|
||||
:export:
|
||||
|
||||
.. kernel-doc:: block/bio.c
|
||||
:export:
|
||||
|
||||
.. kernel-doc:: fs/seq_file.c
|
||||
:export:
|
||||
|
||||
.. kernel-doc:: fs/filesystems.c
|
||||
:export:
|
||||
|
||||
.. kernel-doc:: fs/fs-writeback.c
|
||||
:export:
|
||||
|
||||
.. kernel-doc:: fs/block_dev.c
|
||||
:export:
|
||||
|
||||
.. kernel-doc:: fs/anon_inodes.c
|
||||
:export:
|
||||
|
||||
.. kernel-doc:: fs/attr.c
|
||||
:export:
|
||||
|
||||
.. kernel-doc:: fs/d_path.c
|
||||
:export:
|
||||
|
||||
.. kernel-doc:: fs/dax.c
|
||||
:export:
|
||||
|
||||
.. kernel-doc:: fs/direct-io.c
|
||||
:export:
|
||||
|
||||
.. kernel-doc:: fs/file_table.c
|
||||
:export:
|
||||
|
||||
.. kernel-doc:: fs/libfs.c
|
||||
:export:
|
||||
|
||||
.. kernel-doc:: fs/posix_acl.c
|
||||
:export:
|
||||
|
||||
.. kernel-doc:: fs/stat.c
|
||||
:export:
|
||||
|
||||
.. kernel-doc:: fs/sync.c
|
||||
:export:
|
||||
|
||||
.. kernel-doc:: fs/xattr.c
|
||||
:export:
|
||||
|
||||
The proc filesystem
|
||||
===================
|
||||
|
||||
sysctl interface
|
||||
----------------
|
||||
|
||||
.. kernel-doc:: kernel/sysctl.c
|
||||
:export:
|
||||
|
||||
proc filesystem interface
|
||||
-------------------------
|
||||
|
||||
.. kernel-doc:: fs/proc/base.c
|
||||
:internal:
|
||||
|
||||
Events based on file descriptors
|
||||
================================
|
||||
|
||||
.. kernel-doc:: fs/eventfd.c
|
||||
:export:
|
||||
|
||||
The Filesystem for Exporting Kernel Objects
|
||||
===========================================
|
||||
|
||||
.. kernel-doc:: fs/sysfs/file.c
|
||||
:export:
|
||||
|
||||
.. kernel-doc:: fs/sysfs/symlink.c
|
||||
:export:
|
||||
|
||||
The debugfs filesystem
|
||||
Core VFS documentation
|
||||
======================
|
||||
|
||||
debugfs interface
|
||||
-----------------
|
||||
|
||||
.. kernel-doc:: fs/debugfs/inode.c
|
||||
:export:
|
||||
|
||||
.. kernel-doc:: fs/debugfs/file.c
|
||||
:export:
|
||||
|
||||
The Linux Journalling API
|
||||
=========================
|
||||
|
||||
Overview
|
||||
--------
|
||||
|
||||
Details
|
||||
~~~~~~~
|
||||
|
||||
The journalling layer is easy to use. You need to first of all create a
|
||||
journal_t data structure. There are two calls to do this dependent on
|
||||
how you decide to allocate the physical media on which the journal
|
||||
resides. The :c:func:`jbd2_journal_init_inode` call is for journals stored in
|
||||
filesystem inodes, or the :c:func:`jbd2_journal_init_dev` call can be used
|
||||
for journal stored on a raw device (in a continuous range of blocks). A
|
||||
journal_t is a typedef for a struct pointer, so when you are finally
|
||||
finished make sure you call :c:func:`jbd2_journal_destroy` on it to free up
|
||||
any used kernel memory.
|
||||
|
||||
Once you have got your journal_t object you need to 'mount' or load the
|
||||
journal file. The journalling layer expects the space for the journal
|
||||
was already allocated and initialized properly by the userspace tools.
|
||||
When loading the journal you must call :c:func:`jbd2_journal_load` to process
|
||||
journal contents. If the client file system detects the journal contents
|
||||
does not need to be processed (or even need not have valid contents), it
|
||||
may call :c:func:`jbd2_journal_wipe` to clear the journal contents before
|
||||
calling :c:func:`jbd2_journal_load`.
|
||||
|
||||
Note that jbd2_journal_wipe(..,0) calls
|
||||
:c:func:`jbd2_journal_skip_recovery` for you if it detects any outstanding
|
||||
transactions in the journal and similarly :c:func:`jbd2_journal_load` will
|
||||
call :c:func:`jbd2_journal_recover` if necessary. I would advise reading
|
||||
:c:func:`ext4_load_journal` in fs/ext4/super.c for examples on this stage.
|
||||
|
||||
Now you can go ahead and start modifying the underlying filesystem.
|
||||
Almost.
|
||||
|
||||
You still need to actually journal your filesystem changes, this is done
|
||||
by wrapping them into transactions. Additionally you also need to wrap
|
||||
the modification of each of the buffers with calls to the journal layer,
|
||||
so it knows what the modifications you are actually making are. To do
|
||||
this use :c:func:`jbd2_journal_start` which returns a transaction handle.
|
||||
|
||||
:c:func:`jbd2_journal_start` and its counterpart :c:func:`jbd2_journal_stop`,
|
||||
which indicates the end of a transaction are nestable calls, so you can
|
||||
reenter a transaction if necessary, but remember you must call
|
||||
:c:func:`jbd2_journal_stop` the same number of times as
|
||||
:c:func:`jbd2_journal_start` before the transaction is completed (or more
|
||||
accurately leaves the update phase). Ext4/VFS makes use of this feature to
|
||||
simplify handling of inode dirtying, quota support, etc.
|
||||
|
||||
Inside each transaction you need to wrap the modifications to the
|
||||
individual buffers (blocks). Before you start to modify a buffer you
|
||||
need to call :c:func:`jbd2_journal_get_create_access()` /
|
||||
:c:func:`jbd2_journal_get_write_access()` /
|
||||
:c:func:`jbd2_journal_get_undo_access()` as appropriate, this allows the
|
||||
journalling layer to copy the unmodified
|
||||
data if it needs to. After all the buffer may be part of a previously
|
||||
uncommitted transaction. At this point you are at last ready to modify a
|
||||
buffer, and once you are have done so you need to call
|
||||
:c:func:`jbd2_journal_dirty_metadata`. Or if you've asked for access to a
|
||||
buffer you now know is now longer required to be pushed back on the
|
||||
device you can call :c:func:`jbd2_journal_forget` in much the same way as you
|
||||
might have used :c:func:`bforget` in the past.
|
||||
|
||||
A :c:func:`jbd2_journal_flush` may be called at any time to commit and
|
||||
checkpoint all your transactions.
|
||||
|
||||
Then at umount time , in your :c:func:`put_super` you can then call
|
||||
:c:func:`jbd2_journal_destroy` to clean up your in-core journal object.
|
||||
|
||||
Unfortunately there a couple of ways the journal layer can cause a
|
||||
deadlock. The first thing to note is that each task can only have a
|
||||
single outstanding transaction at any one time, remember nothing commits
|
||||
until the outermost :c:func:`jbd2_journal_stop`. This means you must complete
|
||||
the transaction at the end of each file/inode/address etc. operation you
|
||||
perform, so that the journalling system isn't re-entered on another
|
||||
journal. Since transactions can't be nested/batched across differing
|
||||
journals, and another filesystem other than yours (say ext4) may be
|
||||
modified in a later syscall.
|
||||
|
||||
The second case to bear in mind is that :c:func:`jbd2_journal_start` can block
|
||||
if there isn't enough space in the journal for your transaction (based
|
||||
on the passed nblocks param) - when it blocks it merely(!) needs to wait
|
||||
for transactions to complete and be committed from other tasks, so
|
||||
essentially we are waiting for :c:func:`jbd2_journal_stop`. So to avoid
|
||||
deadlocks you must treat :c:func:`jbd2_journal_start` /
|
||||
:c:func:`jbd2_journal_stop` as if they were semaphores and include them in
|
||||
your semaphore ordering rules to prevent
|
||||
deadlocks. Note that :c:func:`jbd2_journal_extend` has similar blocking
|
||||
behaviour to :c:func:`jbd2_journal_start` so you can deadlock here just as
|
||||
easily as on :c:func:`jbd2_journal_start`.
|
||||
|
||||
Try to reserve the right number of blocks the first time. ;-). This will
|
||||
be the maximum number of blocks you are going to touch in this
|
||||
transaction. I advise having a look at at least ext4_jbd.h to see the
|
||||
basis on which ext4 uses to make these decisions.
|
||||
|
||||
Another wriggle to watch out for is your on-disk block allocation
|
||||
strategy. Why? Because, if you do a delete, you need to ensure you
|
||||
haven't reused any of the freed blocks until the transaction freeing
|
||||
these blocks commits. If you reused these blocks and crash happens,
|
||||
there is no way to restore the contents of the reallocated blocks at the
|
||||
end of the last fully committed transaction. One simple way of doing
|
||||
this is to mark blocks as free in internal in-memory block allocation
|
||||
structures only after the transaction freeing them commits. Ext4 uses
|
||||
journal commit callback for this purpose.
|
||||
|
||||
With journal commit callbacks you can ask the journalling layer to call
|
||||
a callback function when the transaction is finally committed to disk,
|
||||
so that you can do some of your own management. You ask the journalling
|
||||
layer for calling the callback by simply setting
|
||||
``journal->j_commit_callback`` function pointer and that function is
|
||||
called after each transaction commit. You can also use
|
||||
``transaction->t_private_list`` for attaching entries to a transaction
|
||||
that need processing when the transaction commits.
|
||||
|
||||
JBD2 also provides a way to block all transaction updates via
|
||||
:c:func:`jbd2_journal_lock_updates()` /
|
||||
:c:func:`jbd2_journal_unlock_updates()`. Ext4 uses this when it wants a
|
||||
window with a clean and stable fs for a moment. E.g.
|
||||
|
||||
::
|
||||
|
||||
|
||||
jbd2_journal_lock_updates() //stop new stuff happening..
|
||||
jbd2_journal_flush() // checkpoint everything.
|
||||
..do stuff on stable fs
|
||||
jbd2_journal_unlock_updates() // carry on with filesystem use.
|
||||
|
||||
The opportunities for abuse and DOS attacks with this should be obvious,
|
||||
if you allow unprivileged userspace to trigger codepaths containing
|
||||
these calls.
|
||||
|
||||
Summary
|
||||
~~~~~~~
|
||||
|
||||
Using the journal is a matter of wrapping the different context changes,
|
||||
being each mount, each modification (transaction) and each changed
|
||||
buffer to tell the journalling layer about them.
|
||||
|
||||
Data Types
|
||||
----------
|
||||
|
||||
The journalling layer uses typedefs to 'hide' the concrete definitions
|
||||
of the structures used. As a client of the JBD2 layer you can just rely
|
||||
on the using the pointer as a magic cookie of some sort. Obviously the
|
||||
hiding is not enforced as this is 'C'.
|
||||
|
||||
Structures
|
||||
~~~~~~~~~~
|
||||
|
||||
.. kernel-doc:: include/linux/jbd2.h
|
||||
:internal:
|
||||
|
||||
Functions
|
||||
---------
|
||||
|
||||
The functions here are split into two groups those that affect a journal
|
||||
as a whole, and those which are used to manage transactions
|
||||
|
||||
Journal Level
|
||||
~~~~~~~~~~~~~
|
||||
|
||||
.. kernel-doc:: fs/jbd2/journal.c
|
||||
:export:
|
||||
|
||||
.. kernel-doc:: fs/jbd2/recovery.c
|
||||
:internal:
|
||||
|
||||
Transasction Level
|
||||
~~~~~~~~~~~~~~~~~~
|
||||
|
||||
.. kernel-doc:: fs/jbd2/transaction.c
|
||||
|
||||
See also
|
||||
--------
|
||||
|
||||
`Journaling the Linux ext2fs Filesystem, LinuxExpo 98, Stephen
|
||||
Tweedie <http://kernel.org/pub/linux/kernel/people/sct/ext3/journal-design.ps.gz>`__
|
||||
|
||||
`Ext3 Journalling FileSystem, OLS 2000, Dr. Stephen
|
||||
Tweedie <http://olstrans.sourceforge.net/release/OLS2000-ext3/OLS2000-ext3.html>`__
|
||||
|
||||
splice API
|
||||
==========
|
||||
|
||||
splice is a method for moving blocks of data around inside the kernel,
|
||||
without continually transferring them between the kernel and user space.
|
||||
|
||||
.. kernel-doc:: fs/splice.c
|
||||
|
||||
pipes API
|
||||
=========
|
||||
|
||||
Pipe interfaces are all for in-kernel (builtin image) use. They are not
|
||||
exported for use by modules.
|
||||
|
||||
.. kernel-doc:: include/linux/pipe_fs_i.h
|
||||
:internal:
|
||||
|
||||
.. kernel-doc:: fs/pipe.c
|
||||
|
||||
Encryption API
|
||||
==============
|
||||
|
||||
A library which filesystems can hook into to support transparent
|
||||
encryption of files and directories.
|
||||
|
||||
.. toctree::
|
||||
:maxdepth: 2
|
||||
|
||||
fscrypt
|
||||
|
||||
Pathname lookup
|
||||
===============
|
||||
|
||||
|
||||
This write-up is based on three articles published at lwn.net:
|
||||
|
||||
- <https://lwn.net/Articles/649115/> Pathname lookup in Linux
|
||||
- <https://lwn.net/Articles/649729/> RCU-walk: faster pathname lookup in Linux
|
||||
- <https://lwn.net/Articles/650786/> A walk among the symlinks
|
||||
|
||||
Written by Neil Brown with help from Al Viro and Jon Corbet.
|
||||
It has subsequently been updated to reflect changes in the kernel
|
||||
including:
|
||||
|
||||
- per-directory parallel name lookup.
|
||||
See these manuals for documentation about the VFS layer itself and how its
|
||||
algorithms work.
|
||||
|
||||
.. toctree::
|
||||
:maxdepth: 2
|
||||
|
||||
path-lookup.rst
|
||||
api-summary
|
||||
splice
|
||||
|
||||
binderfs
|
||||
========
|
||||
Filesystem support layers
|
||||
=========================
|
||||
|
||||
Documentation for the support code within the filesystem layer for use in
|
||||
filesystem implementations.
|
||||
|
||||
.. toctree::
|
||||
:maxdepth: 2
|
||||
|
||||
journalling
|
||||
fscrypt
|
||||
|
||||
Filesystem-specific documentation
|
||||
=================================
|
||||
|
||||
Documentation for individual filesystem types can be found here.
|
||||
|
||||
.. toctree::
|
||||
:maxdepth: 2
|
||||
|
||||
binderfs.rst
|
||||
|
||||
184
Documentation/filesystems/journalling.rst
Normal file
184
Documentation/filesystems/journalling.rst
Normal file
@ -0,0 +1,184 @@
|
||||
The Linux Journalling API
|
||||
=========================
|
||||
|
||||
Overview
|
||||
--------
|
||||
|
||||
Details
|
||||
~~~~~~~
|
||||
|
||||
The journalling layer is easy to use. You need to first of all create a
|
||||
journal_t data structure. There are two calls to do this dependent on
|
||||
how you decide to allocate the physical media on which the journal
|
||||
resides. The :c:func:`jbd2_journal_init_inode` call is for journals stored in
|
||||
filesystem inodes, or the :c:func:`jbd2_journal_init_dev` call can be used
|
||||
for journal stored on a raw device (in a continuous range of blocks). A
|
||||
journal_t is a typedef for a struct pointer, so when you are finally
|
||||
finished make sure you call :c:func:`jbd2_journal_destroy` on it to free up
|
||||
any used kernel memory.
|
||||
|
||||
Once you have got your journal_t object you need to 'mount' or load the
|
||||
journal file. The journalling layer expects the space for the journal
|
||||
was already allocated and initialized properly by the userspace tools.
|
||||
When loading the journal you must call :c:func:`jbd2_journal_load` to process
|
||||
journal contents. If the client file system detects the journal contents
|
||||
does not need to be processed (or even need not have valid contents), it
|
||||
may call :c:func:`jbd2_journal_wipe` to clear the journal contents before
|
||||
calling :c:func:`jbd2_journal_load`.
|
||||
|
||||
Note that jbd2_journal_wipe(..,0) calls
|
||||
:c:func:`jbd2_journal_skip_recovery` for you if it detects any outstanding
|
||||
transactions in the journal and similarly :c:func:`jbd2_journal_load` will
|
||||
call :c:func:`jbd2_journal_recover` if necessary. I would advise reading
|
||||
:c:func:`ext4_load_journal` in fs/ext4/super.c for examples on this stage.
|
||||
|
||||
Now you can go ahead and start modifying the underlying filesystem.
|
||||
Almost.
|
||||
|
||||
You still need to actually journal your filesystem changes, this is done
|
||||
by wrapping them into transactions. Additionally you also need to wrap
|
||||
the modification of each of the buffers with calls to the journal layer,
|
||||
so it knows what the modifications you are actually making are. To do
|
||||
this use :c:func:`jbd2_journal_start` which returns a transaction handle.
|
||||
|
||||
:c:func:`jbd2_journal_start` and its counterpart :c:func:`jbd2_journal_stop`,
|
||||
which indicates the end of a transaction are nestable calls, so you can
|
||||
reenter a transaction if necessary, but remember you must call
|
||||
:c:func:`jbd2_journal_stop` the same number of times as
|
||||
:c:func:`jbd2_journal_start` before the transaction is completed (or more
|
||||
accurately leaves the update phase). Ext4/VFS makes use of this feature to
|
||||
simplify handling of inode dirtying, quota support, etc.
|
||||
|
||||
Inside each transaction you need to wrap the modifications to the
|
||||
individual buffers (blocks). Before you start to modify a buffer you
|
||||
need to call :c:func:`jbd2_journal_get_create_access()` /
|
||||
:c:func:`jbd2_journal_get_write_access()` /
|
||||
:c:func:`jbd2_journal_get_undo_access()` as appropriate, this allows the
|
||||
journalling layer to copy the unmodified
|
||||
data if it needs to. After all the buffer may be part of a previously
|
||||
uncommitted transaction. At this point you are at last ready to modify a
|
||||
buffer, and once you are have done so you need to call
|
||||
:c:func:`jbd2_journal_dirty_metadata`. Or if you've asked for access to a
|
||||
buffer you now know is now longer required to be pushed back on the
|
||||
device you can call :c:func:`jbd2_journal_forget` in much the same way as you
|
||||
might have used :c:func:`bforget` in the past.
|
||||
|
||||
A :c:func:`jbd2_journal_flush` may be called at any time to commit and
|
||||
checkpoint all your transactions.
|
||||
|
||||
Then at umount time , in your :c:func:`put_super` you can then call
|
||||
:c:func:`jbd2_journal_destroy` to clean up your in-core journal object.
|
||||
|
||||
Unfortunately there a couple of ways the journal layer can cause a
|
||||
deadlock. The first thing to note is that each task can only have a
|
||||
single outstanding transaction at any one time, remember nothing commits
|
||||
until the outermost :c:func:`jbd2_journal_stop`. This means you must complete
|
||||
the transaction at the end of each file/inode/address etc. operation you
|
||||
perform, so that the journalling system isn't re-entered on another
|
||||
journal. Since transactions can't be nested/batched across differing
|
||||
journals, and another filesystem other than yours (say ext4) may be
|
||||
modified in a later syscall.
|
||||
|
||||
The second case to bear in mind is that :c:func:`jbd2_journal_start` can block
|
||||
if there isn't enough space in the journal for your transaction (based
|
||||
on the passed nblocks param) - when it blocks it merely(!) needs to wait
|
||||
for transactions to complete and be committed from other tasks, so
|
||||
essentially we are waiting for :c:func:`jbd2_journal_stop`. So to avoid
|
||||
deadlocks you must treat :c:func:`jbd2_journal_start` /
|
||||
:c:func:`jbd2_journal_stop` as if they were semaphores and include them in
|
||||
your semaphore ordering rules to prevent
|
||||
deadlocks. Note that :c:func:`jbd2_journal_extend` has similar blocking
|
||||
behaviour to :c:func:`jbd2_journal_start` so you can deadlock here just as
|
||||
easily as on :c:func:`jbd2_journal_start`.
|
||||
|
||||
Try to reserve the right number of blocks the first time. ;-). This will
|
||||
be the maximum number of blocks you are going to touch in this
|
||||
transaction. I advise having a look at at least ext4_jbd.h to see the
|
||||
basis on which ext4 uses to make these decisions.
|
||||
|
||||
Another wriggle to watch out for is your on-disk block allocation
|
||||
strategy. Why? Because, if you do a delete, you need to ensure you
|
||||
haven't reused any of the freed blocks until the transaction freeing
|
||||
these blocks commits. If you reused these blocks and crash happens,
|
||||
there is no way to restore the contents of the reallocated blocks at the
|
||||
end of the last fully committed transaction. One simple way of doing
|
||||
this is to mark blocks as free in internal in-memory block allocation
|
||||
structures only after the transaction freeing them commits. Ext4 uses
|
||||
journal commit callback for this purpose.
|
||||
|
||||
With journal commit callbacks you can ask the journalling layer to call
|
||||
a callback function when the transaction is finally committed to disk,
|
||||
so that you can do some of your own management. You ask the journalling
|
||||
layer for calling the callback by simply setting
|
||||
``journal->j_commit_callback`` function pointer and that function is
|
||||
called after each transaction commit. You can also use
|
||||
``transaction->t_private_list`` for attaching entries to a transaction
|
||||
that need processing when the transaction commits.
|
||||
|
||||
JBD2 also provides a way to block all transaction updates via
|
||||
:c:func:`jbd2_journal_lock_updates()` /
|
||||
:c:func:`jbd2_journal_unlock_updates()`. Ext4 uses this when it wants a
|
||||
window with a clean and stable fs for a moment. E.g.
|
||||
|
||||
::
|
||||
|
||||
|
||||
jbd2_journal_lock_updates() //stop new stuff happening..
|
||||
jbd2_journal_flush() // checkpoint everything.
|
||||
..do stuff on stable fs
|
||||
jbd2_journal_unlock_updates() // carry on with filesystem use.
|
||||
|
||||
The opportunities for abuse and DOS attacks with this should be obvious,
|
||||
if you allow unprivileged userspace to trigger codepaths containing
|
||||
these calls.
|
||||
|
||||
Summary
|
||||
~~~~~~~
|
||||
|
||||
Using the journal is a matter of wrapping the different context changes,
|
||||
being each mount, each modification (transaction) and each changed
|
||||
buffer to tell the journalling layer about them.
|
||||
|
||||
Data Types
|
||||
----------
|
||||
|
||||
The journalling layer uses typedefs to 'hide' the concrete definitions
|
||||
of the structures used. As a client of the JBD2 layer you can just rely
|
||||
on the using the pointer as a magic cookie of some sort. Obviously the
|
||||
hiding is not enforced as this is 'C'.
|
||||
|
||||
Structures
|
||||
~~~~~~~~~~
|
||||
|
||||
.. kernel-doc:: include/linux/jbd2.h
|
||||
:internal:
|
||||
|
||||
Functions
|
||||
---------
|
||||
|
||||
The functions here are split into two groups those that affect a journal
|
||||
as a whole, and those which are used to manage transactions
|
||||
|
||||
Journal Level
|
||||
~~~~~~~~~~~~~
|
||||
|
||||
.. kernel-doc:: fs/jbd2/journal.c
|
||||
:export:
|
||||
|
||||
.. kernel-doc:: fs/jbd2/recovery.c
|
||||
:internal:
|
||||
|
||||
Transasction Level
|
||||
~~~~~~~~~~~~~~~~~~
|
||||
|
||||
.. kernel-doc:: fs/jbd2/transaction.c
|
||||
|
||||
See also
|
||||
--------
|
||||
|
||||
`Journaling the Linux ext2fs Filesystem, LinuxExpo 98, Stephen
|
||||
Tweedie <http://kernel.org/pub/linux/kernel/people/sct/ext3/journal-design.ps.gz>`__
|
||||
|
||||
`Ext3 Journalling FileSystem, OLS 2000, Dr. Stephen
|
||||
Tweedie <http://olstrans.sourceforge.net/release/OLS2000-ext3/OLS2000-ext3.html>`__
|
||||
|
||||
@ -1,3 +1,18 @@
|
||||
===============
|
||||
Pathname lookup
|
||||
===============
|
||||
|
||||
This write-up is based on three articles published at lwn.net:
|
||||
|
||||
- <https://lwn.net/Articles/649115/> Pathname lookup in Linux
|
||||
- <https://lwn.net/Articles/649729/> RCU-walk: faster pathname lookup in Linux
|
||||
- <https://lwn.net/Articles/650786/> A walk among the symlinks
|
||||
|
||||
Written by Neil Brown with help from Al Viro and Jon Corbet.
|
||||
It has subsequently been updated to reflect changes in the kernel
|
||||
including:
|
||||
|
||||
- per-directory parallel name lookup.
|
||||
|
||||
Introduction to pathname lookup
|
||||
===============================
|
||||
|
||||
22
Documentation/filesystems/splice.rst
Normal file
22
Documentation/filesystems/splice.rst
Normal file
@ -0,0 +1,22 @@
|
||||
================
|
||||
splice and pipes
|
||||
================
|
||||
|
||||
splice API
|
||||
==========
|
||||
|
||||
splice is a method for moving blocks of data around inside the kernel,
|
||||
without continually transferring them between the kernel and user space.
|
||||
|
||||
.. kernel-doc:: fs/splice.c
|
||||
|
||||
pipes API
|
||||
=========
|
||||
|
||||
Pipe interfaces are all for in-kernel (builtin image) use. They are not
|
||||
exported for use by modules.
|
||||
|
||||
.. kernel-doc:: include/linux/pipe_fs_i.h
|
||||
:internal:
|
||||
|
||||
.. kernel-doc:: fs/pipe.c
|
||||
Loading…
Reference in New Issue
Block a user