mirror of
https://github.com/torvalds/linux.git
synced 2024-11-10 22:21:40 +00:00
ecae0bd517
included in this merge do the following: - Kemeng Shi has contributed some compation maintenance work in the series "Fixes and cleanups to compaction". - Joel Fernandes has a patchset ("Optimize mremap during mutual alignment within PMD") which fixes an obscure issue with mremap()'s pagetable handling during a subsequent exec(), based upon an implementation which Linus suggested. - More DAMON/DAMOS maintenance and feature work from SeongJae Park i the following patch series: mm/damon: misc fixups for documents, comments and its tracepoint mm/damon: add a tracepoint for damos apply target regions mm/damon: provide pseudo-moving sum based access rate mm/damon: implement DAMOS apply intervals mm/damon/core-test: Fix memory leaks in core-test mm/damon/sysfs-schemes: Do DAMOS tried regions update for only one apply interval - In the series "Do not try to access unaccepted memory" Adrian Hunter provides some fixups for the recently-added "unaccepted memory' feature. To increase the feature's checking coverage. "Plug a few gaps where RAM is exposed without checking if it is unaccepted memory". - In the series "cleanups for lockless slab shrink" Qi Zheng has done some maintenance work which is preparation for the lockless slab shrinking code. - Qi Zheng has redone the earlier (and reverted) attempt to make slab shrinking lockless in the series "use refcount+RCU method to implement lockless slab shrink". - David Hildenbrand contributes some maintenance work for the rmap code in the series "Anon rmap cleanups". - Kefeng Wang does more folio conversions and some maintenance work in the migration code. Series "mm: migrate: more folio conversion and unification". - Matthew Wilcox has fixed an issue in the buffer_head code which was causing long stalls under some heavy memory/IO loads. Some cleanups were added on the way. Series "Add and use bdev_getblk()". - In the series "Use nth_page() in place of direct struct page manipulation" Zi Yan has fixed a potential issue with the direct manipulation of hugetlb page frames. - In the series "mm: hugetlb: Skip initialization of gigantic tail struct pages if freed by HVO" has improved our handling of gigantic pages in the hugetlb vmmemmep optimizaton code. This provides significant boot time improvements when significant amounts of gigantic pages are in use. - Matthew Wilcox has sent the series "Small hugetlb cleanups" - code rationalization and folio conversions in the hugetlb code. - Yin Fengwei has improved mlock()'s handling of large folios in the series "support large folio for mlock" - In the series "Expose swapcache stat for memcg v1" Liu Shixin has added statistics for memcg v1 users which are available (and useful) under memcg v2. - Florent Revest has enhanced the MDWE (Memory-Deny-Write-Executable) prctl so that userspace may direct the kernel to not automatically propagate the denial to child processes. The series is named "MDWE without inheritance". - Kefeng Wang has provided the series "mm: convert numa balancing functions to use a folio" which does what it says. - In the series "mm/ksm: add fork-exec support for prctl" Stefan Roesch makes is possible for a process to propagate KSM treatment across exec(). - Huang Ying has enhanced memory tiering's calculation of memory distances. This is used to permit the dax/kmem driver to use "high bandwidth memory" in addition to Optane Data Center Persistent Memory Modules (DCPMM). The series is named "memory tiering: calculate abstract distance based on ACPI HMAT" - In the series "Smart scanning mode for KSM" Stefan Roesch has optimized KSM by teaching it to retain and use some historical information from previous scans. - Yosry Ahmed has fixed some inconsistencies in memcg statistics in the series "mm: memcg: fix tracking of pending stats updates values". - In the series "Implement IOCTL to get and optionally clear info about PTEs" Peter Xu has added an ioctl to /proc/<pid>/pagemap which permits us to atomically read-then-clear page softdirty state. This is mainly used by CRIU. - Hugh Dickins contributed the series "shmem,tmpfs: general maintenance" - a bunch of relatively minor maintenance tweaks to this code. - Matthew Wilcox has increased the use of the VMA lock over file-backed page faults in the series "Handle more faults under the VMA lock". Some rationalizations of the fault path became possible as a result. - In the series "mm/rmap: convert page_move_anon_rmap() to folio_move_anon_rmap()" David Hildenbrand has implemented some cleanups and folio conversions. - In the series "various improvements to the GUP interface" Lorenzo Stoakes has simplified and improved the GUP interface with an eye to providing groundwork for future improvements. - Andrey Konovalov has sent along the series "kasan: assorted fixes and improvements" which does those things. - Some page allocator maintenance work from Kemeng Shi in the series "Two minor cleanups to break_down_buddy_pages". - In thes series "New selftest for mm" Breno Leitao has developed another MM self test which tickles a race we had between madvise() and page faults. - In the series "Add folio_end_read" Matthew Wilcox provides cleanups and an optimization to the core pagecache code. - Nhat Pham has added memcg accounting for hugetlb memory in the series "hugetlb memcg accounting". - Cleanups and rationalizations to the pagemap code from Lorenzo Stoakes, in the series "Abstract vma_merge() and split_vma()". - Audra Mitchell has fixed issues in the procfs page_owner code's new timestamping feature which was causing some misbehaviours. In the series "Fix page_owner's use of free timestamps". - Lorenzo Stoakes has fixed the handling of new mappings of sealed files in the series "permit write-sealed memfd read-only shared mappings". - Mike Kravetz has optimized the hugetlb vmemmap optimization in the series "Batch hugetlb vmemmap modification operations". - Some buffer_head folio conversions and cleanups from Matthew Wilcox in the series "Finish the create_empty_buffers() transition". - As a page allocator performance optimization Huang Ying has added automatic tuning to the allocator's per-cpu-pages feature, in the series "mm: PCP high auto-tuning". - Roman Gushchin has contributed the patchset "mm: improve performance of accounted kernel memory allocations" which improves their performance by ~30% as measured by a micro-benchmark. - folio conversions from Kefeng Wang in the series "mm: convert page cpupid functions to folios". - Some kmemleak fixups in Liu Shixin's series "Some bugfix about kmemleak". - Qi Zheng has improved our handling of memoryless nodes by keeping them off the allocation fallback list. This is done in the series "handle memoryless nodes more appropriately". - khugepaged conversions from Vishal Moola in the series "Some khugepaged folio conversions". -----BEGIN PGP SIGNATURE----- iHUEABYIAB0WIQTTMBEPP41GrTpTJgfdBJ7gKXxAjgUCZULEMwAKCRDdBJ7gKXxA jhQHAQCYpD3g849x69DmHnHWHm/EHQLvQmRMDeYZI+nx/sCJOwEAw4AKg0Oemv9y FgeUPAD1oasg6CP+INZvCj34waNxwAc= =E+Y4 -----END PGP SIGNATURE----- Merge tag 'mm-stable-2023-11-01-14-33' of git://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm Pull MM updates from Andrew Morton: "Many singleton patches against the MM code. The patch series which are included in this merge do the following: - Kemeng Shi has contributed some compation maintenance work in the series 'Fixes and cleanups to compaction' - Joel Fernandes has a patchset ('Optimize mremap during mutual alignment within PMD') which fixes an obscure issue with mremap()'s pagetable handling during a subsequent exec(), based upon an implementation which Linus suggested - More DAMON/DAMOS maintenance and feature work from SeongJae Park i the following patch series: mm/damon: misc fixups for documents, comments and its tracepoint mm/damon: add a tracepoint for damos apply target regions mm/damon: provide pseudo-moving sum based access rate mm/damon: implement DAMOS apply intervals mm/damon/core-test: Fix memory leaks in core-test mm/damon/sysfs-schemes: Do DAMOS tried regions update for only one apply interval - In the series 'Do not try to access unaccepted memory' Adrian Hunter provides some fixups for the recently-added 'unaccepted memory' feature. To increase the feature's checking coverage. 'Plug a few gaps where RAM is exposed without checking if it is unaccepted memory' - In the series 'cleanups for lockless slab shrink' Qi Zheng has done some maintenance work which is preparation for the lockless slab shrinking code - Qi Zheng has redone the earlier (and reverted) attempt to make slab shrinking lockless in the series 'use refcount+RCU method to implement lockless slab shrink' - David Hildenbrand contributes some maintenance work for the rmap code in the series 'Anon rmap cleanups' - Kefeng Wang does more folio conversions and some maintenance work in the migration code. Series 'mm: migrate: more folio conversion and unification' - Matthew Wilcox has fixed an issue in the buffer_head code which was causing long stalls under some heavy memory/IO loads. Some cleanups were added on the way. Series 'Add and use bdev_getblk()' - In the series 'Use nth_page() in place of direct struct page manipulation' Zi Yan has fixed a potential issue with the direct manipulation of hugetlb page frames - In the series 'mm: hugetlb: Skip initialization of gigantic tail struct pages if freed by HVO' has improved our handling of gigantic pages in the hugetlb vmmemmep optimizaton code. This provides significant boot time improvements when significant amounts of gigantic pages are in use - Matthew Wilcox has sent the series 'Small hugetlb cleanups' - code rationalization and folio conversions in the hugetlb code - Yin Fengwei has improved mlock()'s handling of large folios in the series 'support large folio for mlock' - In the series 'Expose swapcache stat for memcg v1' Liu Shixin has added statistics for memcg v1 users which are available (and useful) under memcg v2 - Florent Revest has enhanced the MDWE (Memory-Deny-Write-Executable) prctl so that userspace may direct the kernel to not automatically propagate the denial to child processes. The series is named 'MDWE without inheritance' - Kefeng Wang has provided the series 'mm: convert numa balancing functions to use a folio' which does what it says - In the series 'mm/ksm: add fork-exec support for prctl' Stefan Roesch makes is possible for a process to propagate KSM treatment across exec() - Huang Ying has enhanced memory tiering's calculation of memory distances. This is used to permit the dax/kmem driver to use 'high bandwidth memory' in addition to Optane Data Center Persistent Memory Modules (DCPMM). The series is named 'memory tiering: calculate abstract distance based on ACPI HMAT' - In the series 'Smart scanning mode for KSM' Stefan Roesch has optimized KSM by teaching it to retain and use some historical information from previous scans - Yosry Ahmed has fixed some inconsistencies in memcg statistics in the series 'mm: memcg: fix tracking of pending stats updates values' - In the series 'Implement IOCTL to get and optionally clear info about PTEs' Peter Xu has added an ioctl to /proc/<pid>/pagemap which permits us to atomically read-then-clear page softdirty state. This is mainly used by CRIU - Hugh Dickins contributed the series 'shmem,tmpfs: general maintenance', a bunch of relatively minor maintenance tweaks to this code - Matthew Wilcox has increased the use of the VMA lock over file-backed page faults in the series 'Handle more faults under the VMA lock'. Some rationalizations of the fault path became possible as a result - In the series 'mm/rmap: convert page_move_anon_rmap() to folio_move_anon_rmap()' David Hildenbrand has implemented some cleanups and folio conversions - In the series 'various improvements to the GUP interface' Lorenzo Stoakes has simplified and improved the GUP interface with an eye to providing groundwork for future improvements - Andrey Konovalov has sent along the series 'kasan: assorted fixes and improvements' which does those things - Some page allocator maintenance work from Kemeng Shi in the series 'Two minor cleanups to break_down_buddy_pages' - In thes series 'New selftest for mm' Breno Leitao has developed another MM self test which tickles a race we had between madvise() and page faults - In the series 'Add folio_end_read' Matthew Wilcox provides cleanups and an optimization to the core pagecache code - Nhat Pham has added memcg accounting for hugetlb memory in the series 'hugetlb memcg accounting' - Cleanups and rationalizations to the pagemap code from Lorenzo Stoakes, in the series 'Abstract vma_merge() and split_vma()' - Audra Mitchell has fixed issues in the procfs page_owner code's new timestamping feature which was causing some misbehaviours. In the series 'Fix page_owner's use of free timestamps' - Lorenzo Stoakes has fixed the handling of new mappings of sealed files in the series 'permit write-sealed memfd read-only shared mappings' - Mike Kravetz has optimized the hugetlb vmemmap optimization in the series 'Batch hugetlb vmemmap modification operations' - Some buffer_head folio conversions and cleanups from Matthew Wilcox in the series 'Finish the create_empty_buffers() transition' - As a page allocator performance optimization Huang Ying has added automatic tuning to the allocator's per-cpu-pages feature, in the series 'mm: PCP high auto-tuning' - Roman Gushchin has contributed the patchset 'mm: improve performance of accounted kernel memory allocations' which improves their performance by ~30% as measured by a micro-benchmark - folio conversions from Kefeng Wang in the series 'mm: convert page cpupid functions to folios' - Some kmemleak fixups in Liu Shixin's series 'Some bugfix about kmemleak' - Qi Zheng has improved our handling of memoryless nodes by keeping them off the allocation fallback list. This is done in the series 'handle memoryless nodes more appropriately' - khugepaged conversions from Vishal Moola in the series 'Some khugepaged folio conversions'" [ bcachefs conflicts with the dynamically allocated shrinkers have been resolved as per Stephen Rothwell in https://lore.kernel.org/all/20230913093553.4290421e@canb.auug.org.au/ with help from Qi Zheng. The clone3 test filtering conflict was half-arsed by yours truly ] * tag 'mm-stable-2023-11-01-14-33' of git://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm: (406 commits) mm/damon/sysfs: update monitoring target regions for online input commit mm/damon/sysfs: remove requested targets when online-commit inputs selftests: add a sanity check for zswap Documentation: maple_tree: fix word spelling error mm/vmalloc: fix the unchecked dereference warning in vread_iter() zswap: export compression failure stats Documentation: ubsan: drop "the" from article title mempolicy: migration attempt to match interleave nodes mempolicy: mmap_lock is not needed while migrating folios mempolicy: alloc_pages_mpol() for NUMA policy without vma mm: add page_rmappable_folio() wrapper mempolicy: remove confusing MPOL_MF_LAZY dead code mempolicy: mpol_shared_policy_init() without pseudo-vma mempolicy trivia: use pgoff_t in shared mempolicy tree mempolicy trivia: slightly more consistent naming mempolicy trivia: delete those ancient pr_debug()s mempolicy: fix migrate_pages(2) syscall return nr_failed kernfs: drop shared NUMA mempolicy hooks hugetlbfs: drop shared NUMA mempolicy pretence mm/damon/sysfs-test: add a unit test for damon_sysfs_set_targets() ...
382 lines
12 KiB
C
382 lines
12 KiB
C
// SPDX-License-Identifier: GPL-2.0
|
|
/*
|
|
* Copyright (c) 2000-2005 Silicon Graphics, Inc.
|
|
* All Rights Reserved.
|
|
*/
|
|
#ifndef __XFS_BUF_H__
|
|
#define __XFS_BUF_H__
|
|
|
|
#include <linux/list.h>
|
|
#include <linux/types.h>
|
|
#include <linux/spinlock.h>
|
|
#include <linux/mm.h>
|
|
#include <linux/fs.h>
|
|
#include <linux/dax.h>
|
|
#include <linux/uio.h>
|
|
#include <linux/list_lru.h>
|
|
|
|
extern struct kmem_cache *xfs_buf_cache;
|
|
|
|
/*
|
|
* Base types
|
|
*/
|
|
struct xfs_buf;
|
|
|
|
#define XFS_BUF_DADDR_NULL ((xfs_daddr_t) (-1LL))
|
|
|
|
#define XBF_READ (1u << 0) /* buffer intended for reading from device */
|
|
#define XBF_WRITE (1u << 1) /* buffer intended for writing to device */
|
|
#define XBF_READ_AHEAD (1u << 2) /* asynchronous read-ahead */
|
|
#define XBF_NO_IOACCT (1u << 3) /* bypass I/O accounting (non-LRU bufs) */
|
|
#define XBF_ASYNC (1u << 4) /* initiator will not wait for completion */
|
|
#define XBF_DONE (1u << 5) /* all pages in the buffer uptodate */
|
|
#define XBF_STALE (1u << 6) /* buffer has been staled, do not find it */
|
|
#define XBF_WRITE_FAIL (1u << 7) /* async writes have failed on this buffer */
|
|
|
|
/* buffer type flags for write callbacks */
|
|
#define _XBF_INODES (1u << 16)/* inode buffer */
|
|
#define _XBF_DQUOTS (1u << 17)/* dquot buffer */
|
|
#define _XBF_LOGRECOVERY (1u << 18)/* log recovery buffer */
|
|
|
|
/* flags used only internally */
|
|
#define _XBF_PAGES (1u << 20)/* backed by refcounted pages */
|
|
#define _XBF_KMEM (1u << 21)/* backed by heap memory */
|
|
#define _XBF_DELWRI_Q (1u << 22)/* buffer on a delwri queue */
|
|
|
|
/* flags used only as arguments to access routines */
|
|
/*
|
|
* Online fsck is scanning the buffer cache for live buffers. Do not warn
|
|
* about length mismatches during lookups and do not return stale buffers.
|
|
*/
|
|
#define XBF_LIVESCAN (1u << 28)
|
|
#define XBF_INCORE (1u << 29)/* lookup only, return if found in cache */
|
|
#define XBF_TRYLOCK (1u << 30)/* lock requested, but do not wait */
|
|
#define XBF_UNMAPPED (1u << 31)/* do not map the buffer */
|
|
|
|
|
|
typedef unsigned int xfs_buf_flags_t;
|
|
|
|
#define XFS_BUF_FLAGS \
|
|
{ XBF_READ, "READ" }, \
|
|
{ XBF_WRITE, "WRITE" }, \
|
|
{ XBF_READ_AHEAD, "READ_AHEAD" }, \
|
|
{ XBF_NO_IOACCT, "NO_IOACCT" }, \
|
|
{ XBF_ASYNC, "ASYNC" }, \
|
|
{ XBF_DONE, "DONE" }, \
|
|
{ XBF_STALE, "STALE" }, \
|
|
{ XBF_WRITE_FAIL, "WRITE_FAIL" }, \
|
|
{ _XBF_INODES, "INODES" }, \
|
|
{ _XBF_DQUOTS, "DQUOTS" }, \
|
|
{ _XBF_LOGRECOVERY, "LOG_RECOVERY" }, \
|
|
{ _XBF_PAGES, "PAGES" }, \
|
|
{ _XBF_KMEM, "KMEM" }, \
|
|
{ _XBF_DELWRI_Q, "DELWRI_Q" }, \
|
|
/* The following interface flags should never be set */ \
|
|
{ XBF_LIVESCAN, "LIVESCAN" }, \
|
|
{ XBF_INCORE, "INCORE" }, \
|
|
{ XBF_TRYLOCK, "TRYLOCK" }, \
|
|
{ XBF_UNMAPPED, "UNMAPPED" }
|
|
|
|
/*
|
|
* Internal state flags.
|
|
*/
|
|
#define XFS_BSTATE_DISPOSE (1 << 0) /* buffer being discarded */
|
|
#define XFS_BSTATE_IN_FLIGHT (1 << 1) /* I/O in flight */
|
|
|
|
/*
|
|
* The xfs_buftarg contains 2 notions of "sector size" -
|
|
*
|
|
* 1) The metadata sector size, which is the minimum unit and
|
|
* alignment of IO which will be performed by metadata operations.
|
|
* 2) The device logical sector size
|
|
*
|
|
* The first is specified at mkfs time, and is stored on-disk in the
|
|
* superblock's sb_sectsize.
|
|
*
|
|
* The latter is derived from the underlying device, and controls direct IO
|
|
* alignment constraints.
|
|
*/
|
|
typedef struct xfs_buftarg {
|
|
dev_t bt_dev;
|
|
struct bdev_handle *bt_bdev_handle;
|
|
struct block_device *bt_bdev;
|
|
struct dax_device *bt_daxdev;
|
|
u64 bt_dax_part_off;
|
|
struct xfs_mount *bt_mount;
|
|
unsigned int bt_meta_sectorsize;
|
|
size_t bt_meta_sectormask;
|
|
size_t bt_logical_sectorsize;
|
|
size_t bt_logical_sectormask;
|
|
|
|
/* LRU control structures */
|
|
struct shrinker *bt_shrinker;
|
|
struct list_lru bt_lru;
|
|
|
|
struct percpu_counter bt_io_count;
|
|
struct ratelimit_state bt_ioerror_rl;
|
|
} xfs_buftarg_t;
|
|
|
|
#define XB_PAGES 2
|
|
|
|
struct xfs_buf_map {
|
|
xfs_daddr_t bm_bn; /* block number for I/O */
|
|
int bm_len; /* size of I/O */
|
|
unsigned int bm_flags;
|
|
};
|
|
|
|
/*
|
|
* Online fsck is scanning the buffer cache for live buffers. Do not warn
|
|
* about length mismatches during lookups and do not return stale buffers.
|
|
*/
|
|
#define XBM_LIVESCAN (1U << 0)
|
|
|
|
#define DEFINE_SINGLE_BUF_MAP(map, blkno, numblk) \
|
|
struct xfs_buf_map (map) = { .bm_bn = (blkno), .bm_len = (numblk) };
|
|
|
|
struct xfs_buf_ops {
|
|
char *name;
|
|
union {
|
|
__be32 magic[2]; /* v4 and v5 on disk magic values */
|
|
__be16 magic16[2]; /* v4 and v5 on disk magic values */
|
|
};
|
|
void (*verify_read)(struct xfs_buf *);
|
|
void (*verify_write)(struct xfs_buf *);
|
|
xfs_failaddr_t (*verify_struct)(struct xfs_buf *bp);
|
|
};
|
|
|
|
struct xfs_buf {
|
|
/*
|
|
* first cacheline holds all the fields needed for an uncontended cache
|
|
* hit to be fully processed. The semaphore straddles the cacheline
|
|
* boundary, but the counter and lock sits on the first cacheline,
|
|
* which is the only bit that is touched if we hit the semaphore
|
|
* fast-path on locking.
|
|
*/
|
|
struct rhash_head b_rhash_head; /* pag buffer hash node */
|
|
|
|
xfs_daddr_t b_rhash_key; /* buffer cache index */
|
|
int b_length; /* size of buffer in BBs */
|
|
atomic_t b_hold; /* reference count */
|
|
atomic_t b_lru_ref; /* lru reclaim ref count */
|
|
xfs_buf_flags_t b_flags; /* status flags */
|
|
struct semaphore b_sema; /* semaphore for lockables */
|
|
|
|
/*
|
|
* concurrent access to b_lru and b_lru_flags are protected by
|
|
* bt_lru_lock and not by b_sema
|
|
*/
|
|
struct list_head b_lru; /* lru list */
|
|
spinlock_t b_lock; /* internal state lock */
|
|
unsigned int b_state; /* internal state flags */
|
|
int b_io_error; /* internal IO error state */
|
|
wait_queue_head_t b_waiters; /* unpin waiters */
|
|
struct list_head b_list;
|
|
struct xfs_perag *b_pag; /* contains rbtree root */
|
|
struct xfs_mount *b_mount;
|
|
struct xfs_buftarg *b_target; /* buffer target (device) */
|
|
void *b_addr; /* virtual address of buffer */
|
|
struct work_struct b_ioend_work;
|
|
struct completion b_iowait; /* queue for I/O waiters */
|
|
struct xfs_buf_log_item *b_log_item;
|
|
struct list_head b_li_list; /* Log items list head */
|
|
struct xfs_trans *b_transp;
|
|
struct page **b_pages; /* array of page pointers */
|
|
struct page *b_page_array[XB_PAGES]; /* inline pages */
|
|
struct xfs_buf_map *b_maps; /* compound buffer map */
|
|
struct xfs_buf_map __b_map; /* inline compound buffer map */
|
|
int b_map_count;
|
|
atomic_t b_pin_count; /* pin count */
|
|
atomic_t b_io_remaining; /* #outstanding I/O requests */
|
|
unsigned int b_page_count; /* size of page array */
|
|
unsigned int b_offset; /* page offset of b_addr,
|
|
only for _XBF_KMEM buffers */
|
|
int b_error; /* error code on I/O */
|
|
|
|
/*
|
|
* async write failure retry count. Initialised to zero on the first
|
|
* failure, then when it exceeds the maximum configured without a
|
|
* success the write is considered to be failed permanently and the
|
|
* iodone handler will take appropriate action.
|
|
*
|
|
* For retry timeouts, we record the jiffie of the first failure. This
|
|
* means that we can change the retry timeout for buffers already under
|
|
* I/O and thus avoid getting stuck in a retry loop with a long timeout.
|
|
*
|
|
* last_error is used to ensure that we are getting repeated errors, not
|
|
* different errors. e.g. a block device might change ENOSPC to EIO when
|
|
* a failure timeout occurs, so we want to re-initialise the error
|
|
* retry behaviour appropriately when that happens.
|
|
*/
|
|
int b_retries;
|
|
unsigned long b_first_retry_time; /* in jiffies */
|
|
int b_last_error;
|
|
|
|
const struct xfs_buf_ops *b_ops;
|
|
struct rcu_head b_rcu;
|
|
};
|
|
|
|
/* Finding and Reading Buffers */
|
|
int xfs_buf_get_map(struct xfs_buftarg *target, struct xfs_buf_map *map,
|
|
int nmaps, xfs_buf_flags_t flags, struct xfs_buf **bpp);
|
|
int xfs_buf_read_map(struct xfs_buftarg *target, struct xfs_buf_map *map,
|
|
int nmaps, xfs_buf_flags_t flags, struct xfs_buf **bpp,
|
|
const struct xfs_buf_ops *ops, xfs_failaddr_t fa);
|
|
void xfs_buf_readahead_map(struct xfs_buftarg *target,
|
|
struct xfs_buf_map *map, int nmaps,
|
|
const struct xfs_buf_ops *ops);
|
|
|
|
static inline int
|
|
xfs_buf_incore(
|
|
struct xfs_buftarg *target,
|
|
xfs_daddr_t blkno,
|
|
size_t numblks,
|
|
xfs_buf_flags_t flags,
|
|
struct xfs_buf **bpp)
|
|
{
|
|
DEFINE_SINGLE_BUF_MAP(map, blkno, numblks);
|
|
|
|
return xfs_buf_get_map(target, &map, 1, XBF_INCORE | flags, bpp);
|
|
}
|
|
|
|
static inline int
|
|
xfs_buf_get(
|
|
struct xfs_buftarg *target,
|
|
xfs_daddr_t blkno,
|
|
size_t numblks,
|
|
struct xfs_buf **bpp)
|
|
{
|
|
DEFINE_SINGLE_BUF_MAP(map, blkno, numblks);
|
|
|
|
return xfs_buf_get_map(target, &map, 1, 0, bpp);
|
|
}
|
|
|
|
static inline int
|
|
xfs_buf_read(
|
|
struct xfs_buftarg *target,
|
|
xfs_daddr_t blkno,
|
|
size_t numblks,
|
|
xfs_buf_flags_t flags,
|
|
struct xfs_buf **bpp,
|
|
const struct xfs_buf_ops *ops)
|
|
{
|
|
DEFINE_SINGLE_BUF_MAP(map, blkno, numblks);
|
|
|
|
return xfs_buf_read_map(target, &map, 1, flags, bpp, ops,
|
|
__builtin_return_address(0));
|
|
}
|
|
|
|
static inline void
|
|
xfs_buf_readahead(
|
|
struct xfs_buftarg *target,
|
|
xfs_daddr_t blkno,
|
|
size_t numblks,
|
|
const struct xfs_buf_ops *ops)
|
|
{
|
|
DEFINE_SINGLE_BUF_MAP(map, blkno, numblks);
|
|
return xfs_buf_readahead_map(target, &map, 1, ops);
|
|
}
|
|
|
|
int xfs_buf_get_uncached(struct xfs_buftarg *target, size_t numblks,
|
|
xfs_buf_flags_t flags, struct xfs_buf **bpp);
|
|
int xfs_buf_read_uncached(struct xfs_buftarg *target, xfs_daddr_t daddr,
|
|
size_t numblks, xfs_buf_flags_t flags, struct xfs_buf **bpp,
|
|
const struct xfs_buf_ops *ops);
|
|
int _xfs_buf_read(struct xfs_buf *bp, xfs_buf_flags_t flags);
|
|
void xfs_buf_hold(struct xfs_buf *bp);
|
|
|
|
/* Releasing Buffers */
|
|
extern void xfs_buf_rele(struct xfs_buf *);
|
|
|
|
/* Locking and Unlocking Buffers */
|
|
extern int xfs_buf_trylock(struct xfs_buf *);
|
|
extern void xfs_buf_lock(struct xfs_buf *);
|
|
extern void xfs_buf_unlock(struct xfs_buf *);
|
|
#define xfs_buf_islocked(bp) \
|
|
((bp)->b_sema.count <= 0)
|
|
|
|
static inline void xfs_buf_relse(struct xfs_buf *bp)
|
|
{
|
|
xfs_buf_unlock(bp);
|
|
xfs_buf_rele(bp);
|
|
}
|
|
|
|
/* Buffer Read and Write Routines */
|
|
extern int xfs_bwrite(struct xfs_buf *bp);
|
|
|
|
extern void __xfs_buf_ioerror(struct xfs_buf *bp, int error,
|
|
xfs_failaddr_t failaddr);
|
|
#define xfs_buf_ioerror(bp, err) __xfs_buf_ioerror((bp), (err), __this_address)
|
|
extern void xfs_buf_ioerror_alert(struct xfs_buf *bp, xfs_failaddr_t fa);
|
|
void xfs_buf_ioend_fail(struct xfs_buf *);
|
|
void xfs_buf_zero(struct xfs_buf *bp, size_t boff, size_t bsize);
|
|
void __xfs_buf_mark_corrupt(struct xfs_buf *bp, xfs_failaddr_t fa);
|
|
#define xfs_buf_mark_corrupt(bp) __xfs_buf_mark_corrupt((bp), __this_address)
|
|
|
|
/* Buffer Utility Routines */
|
|
extern void *xfs_buf_offset(struct xfs_buf *, size_t);
|
|
extern void xfs_buf_stale(struct xfs_buf *bp);
|
|
|
|
/* Delayed Write Buffer Routines */
|
|
extern void xfs_buf_delwri_cancel(struct list_head *);
|
|
extern bool xfs_buf_delwri_queue(struct xfs_buf *, struct list_head *);
|
|
extern int xfs_buf_delwri_submit(struct list_head *);
|
|
extern int xfs_buf_delwri_submit_nowait(struct list_head *);
|
|
extern int xfs_buf_delwri_pushbuf(struct xfs_buf *, struct list_head *);
|
|
|
|
static inline xfs_daddr_t xfs_buf_daddr(struct xfs_buf *bp)
|
|
{
|
|
return bp->b_maps[0].bm_bn;
|
|
}
|
|
|
|
void xfs_buf_set_ref(struct xfs_buf *bp, int lru_ref);
|
|
|
|
/*
|
|
* If the buffer is already on the LRU, do nothing. Otherwise set the buffer
|
|
* up with a reference count of 0 so it will be tossed from the cache when
|
|
* released.
|
|
*/
|
|
static inline void xfs_buf_oneshot(struct xfs_buf *bp)
|
|
{
|
|
if (!list_empty(&bp->b_lru) || atomic_read(&bp->b_lru_ref) > 1)
|
|
return;
|
|
atomic_set(&bp->b_lru_ref, 0);
|
|
}
|
|
|
|
static inline int xfs_buf_ispinned(struct xfs_buf *bp)
|
|
{
|
|
return atomic_read(&bp->b_pin_count);
|
|
}
|
|
|
|
static inline int
|
|
xfs_buf_verify_cksum(struct xfs_buf *bp, unsigned long cksum_offset)
|
|
{
|
|
return xfs_verify_cksum(bp->b_addr, BBTOB(bp->b_length),
|
|
cksum_offset);
|
|
}
|
|
|
|
static inline void
|
|
xfs_buf_update_cksum(struct xfs_buf *bp, unsigned long cksum_offset)
|
|
{
|
|
xfs_update_cksum(bp->b_addr, BBTOB(bp->b_length),
|
|
cksum_offset);
|
|
}
|
|
|
|
/*
|
|
* Handling of buftargs.
|
|
*/
|
|
struct xfs_buftarg *xfs_alloc_buftarg(struct xfs_mount *mp,
|
|
struct bdev_handle *bdev_handle);
|
|
extern void xfs_free_buftarg(struct xfs_buftarg *);
|
|
extern void xfs_buftarg_wait(struct xfs_buftarg *);
|
|
extern void xfs_buftarg_drain(struct xfs_buftarg *);
|
|
extern int xfs_setsize_buftarg(struct xfs_buftarg *, unsigned int);
|
|
|
|
#define xfs_getsize_buftarg(buftarg) block_size((buftarg)->bt_bdev)
|
|
#define xfs_readonly_buftarg(buftarg) bdev_read_only((buftarg)->bt_bdev)
|
|
|
|
int xfs_buf_reverify(struct xfs_buf *bp, const struct xfs_buf_ops *ops);
|
|
bool xfs_verify_magic(struct xfs_buf *bp, __be32 dmagic);
|
|
bool xfs_verify_magic16(struct xfs_buf *bp, __be16 dmagic);
|
|
|
|
#endif /* __XFS_BUF_H__ */
|