10 Feb
2022
10 Feb
'22
3 p.m.
On 2022/2/10 2:24 下午, Baolin Wang wrote:
No, I mean it should prevent memory allocation thp in pmem node.
zhong jiang
<zhongjiang-ali(a)linux.alibaba.com> writes:
I think you misunderstood the change, the change is in
get_page_from_freelist(), not in zone_allows_reclaim().
From my understanding, Zhongjiang is trying to disable the memory
allocation fallback for THP, right? But that will cause more demotion
if we can not fallback to PMEM node?
On 2022/2/10 8:58 H, Huang, Ying wrote:
Maybe we are looking at different source code :-). In latest upstream
code, zone_allows_reclaim() is to control node reclaiming (or zone
reclaim) only. Which repo should I look?
zhongjiang-ali
<zhongjiang-ali(a)linux.alibaba.com> writes:
> Currently, Mysql testcase show that a large number of thp are
> migrated
> from pmem node to toptier node, it will bring in more
> pgpromote_demoted
> and migrated failiure. because pmem node memory is marked as
> prot_none,
> it will be migrated by cpu access as soon as possible when it is hot,
> and it is unnesscessary to migrate thp to dram when dram memory is
> not
> enough, which will bring in more demoted and promoted.
>
> Hence, the patch forbid the thp to produce in pmem node. the
> result show
> about 3% improvements. the relative statistics is as follows.
>
> before appling patch:
> mysql prepare:
> pgpromote_demoted 908267
> pgmigrate_fail_dst_node_fail 428223
> pgmigrate_fail_numa_isolate_fail 460480
>
> mysql run:
> pgpromote_demoted 2901105
> pgmigrate_fail_dst_node_fail 5653776
> pgmigrate_fail_numa_isolate_fail 5686052
>
> after appling patch:
> mysql prepare:
> pgpromote_demoted 839297
> pgmigrate_fail_dst_node_fail 36585
> pgmigrate_fail_numa_isolate_fail 36585
>
> mysql run:
> pgpromote_demoted 913828
> pgmigrate_fail_dst_node_fail 235863
> pgmigrate_fail_numa_isolate_fail 235870
>
> Signed-off-by: zhongjiang-ali <zhongjiang-ali(a)linux.alibaba.com>
> ---
> mm/page_alloc.c | 14 ++++++++++++++
> 1 file changed, 14 insertions(+)
>
> diff --git a/mm/page_alloc.c b/mm/page_alloc.c
> index 8cfce92..4fff3cd 100644
> --- a/mm/page_alloc.c
> +++ b/mm/page_alloc.c
> @@ -461,6 +461,17 @@ static __always_inline int
> get_pfnblock_migratetype(struct page *page, unsigned
> return __get_pfnblock_flags_mask(page, pfn, PB_migrate_end,
> MIGRATETYPE_MASK);
> }
> +static inline bool allow_hugepage_allocation(int nid, unsigned
> int order)
> +{
> + if (node_is_toptier(nid))
> + return true;
> +
> + if (order != HPAGE_PMD_ORDER)
> + return true;
> +
> + return false;
> +}
> +
> /**
> * set_pfnblock_flags_mask - Set the requested group of flags
> for a pageblock_nr_pages block of pages
> * @page: The page within the block of interest
> @@ -3689,6 +3700,9 @@ static bool zone_allows_reclaim(struct zone
> *local_zone, struct zone *zone)
> }
> }
> + if (!allow_hugepage_allocation(zone_to_nid(zone),
> order))
> + continue;
> +
It appears that this will disable node reclaiming for THP allocation.
So more pages will be allocated in PMEM node because of allocation
fallback?
We just allow normal pages allocate in pmem node, hence, thp
allocation will fallback to produce more normal pages.
Mysql testcase show that too many thps is promoted to toptier ,
due to toptier memory is not enough, it will bring in
more pgpromote_deomted and dst_node_full counter increasing. In
that case, we prefer to remote access rather
than migrate thp between pmem and toptier node frequently, which
will make performance decrease.
Memory tiering is challenging for THP. There's no many free pages (a
little more than high watermark) in DRAM node, so it's hard to allocate
THP there except workloads start up.
Just curiously, whether disabling THP helps your workload?
From our previous testing, enabling THP can benefit the performance.
And our ECS environment enabled the THP by default.