SQL Server的MAXDOP设置算法

在设置新的SQL Server时,我使用以下代码来确定MAXDOP设置的一个好的起点:
/* 
   This will recommend a MAXDOP setting appropriate for your machine's NUMA memory
   configuration.  You will need to evaluate this setting in a non-production 
   environment before moving it to production.

   MAXDOP can be configured using:  
   EXEC sp_configure 'max degree of parallelism',X;
   RECONFIGURE

   If this instance is hosting a Sharepoint database, you MUST specify MAXDOP=1 
   (URL wrapped for readability)
   http://blogs.msdn.com/b/rcormier/archive/2012/10/25/
   you-shall-configure-your-maxdop-when-using-sharepoint-2013.aspx

   Biztalk (all versions, including 2010): 
   MAXDOP = 1 is only required on the BizTalk Message Box
   database server(s), and must not be changed; all other servers hosting other 
   BizTalk Server databases may return this value to 0 if set.
   http://support.microsoft.com/kb/899000
*/


DECLARE @CoreCount int;
DECLARE @NumaNodes int;

SET @CoreCount = (SELECT i.cpu_count from sys.dm_os_sys_info i);
SET @NumaNodes = (
    SELECT MAX(c.memory_node_id) + 1 
    FROM sys.dm_os_memory_clerks c 
    WHERE memory_node_id < 64
    );

IF @CoreCount > 4 /* If less than 5 cores, don't bother. */
BEGIN
    DECLARE @MaxDOP int;

    /* 3/4 of Total Cores in Machine */
    SET @MaxDOP = @CoreCount * 0.75; 

    /* if @MaxDOP is greater than the per NUMA node
       Core Count, set @MaxDOP = per NUMA node core count
    */
    IF @MaxDOP > (@CoreCount / @NumaNodes) 
        SET @MaxDOP = (@CoreCount / @NumaNodes) * 0.75;

    /*
        Reduce @MaxDOP to an even number 
    */
    SET @MaxDOP = @MaxDOP - (@MaxDOP % 2);

    /* Cap MAXDOP at 8, according to Microsoft */
    IF @MaxDOP > 8 SET @MaxDOP = 8;

    PRINT 'Suggested MAXDOP = ' + CAST(@MaxDOP as varchar(max));
END
ELSE
BEGIN
    PRINT 'Suggested MAXDOP = 0 since you have less than 4 cores total.';
    PRINT 'This is the default setting, you likely do not need to do';
    PRINT 'anything.';
END
我意识到这有点主观,并且可能会因为很多事情而有所不同;然而,我正在尝试创建一个紧凑的通用代码片段,作为新服务器的起点。 有人对这段代码有什么意见吗?
8个回答

最好的方法是使用coreinfo(sysinternals提供的实用程序),因为这将为您提供
a. Logical to Physical Processor Map
b. Logical Processor to Socket Map
c. Logical Processor to NUMA Node Map as below :

Logical to Physical Processor Map:
**----------------------  Physical Processor 0 (Hyperthreaded)
--**--------------------  Physical Processor 1 (Hyperthreaded)
----**------------------  Physical Processor 2 (Hyperthreaded)
------**----------------  Physical Processor 3 (Hyperthreaded)
--------**--------------  Physical Processor 4 (Hyperthreaded)
----------**------------  Physical Processor 5 (Hyperthreaded)
------------**----------  Physical Processor 6 (Hyperthreaded)
--------------**--------  Physical Processor 7 (Hyperthreaded)
----------------**------  Physical Processor 8 (Hyperthreaded)
------------------**----  Physical Processor 9 (Hyperthreaded)
--------------------**--  Physical Processor 10 (Hyperthreaded)
----------------------**  Physical Processor 11 (Hyperthreaded)
 
Logical Processor to Socket Map:
************------------  Socket 0
------------************  Socket 1
 
Logical Processor to NUMA Node Map:
************------------  NUMA Node 0
------------************  NUMA Node 1
根据上述信息,理想的 MaxDop 设置应该按以下方式计算。
a.  It has 12 CPU’s which are hyper threaded giving us 24 CPUs.
b.  It has 2 NUMA node [Node 0 and 1] each having 12 CPU’s with Hyperthreading ON.
c.  Number of sockets are 2 [socket 0 and 1] which are housing 12 CPU’s each.

Considering all above factors, the max degree of Parallelism should be set to 6 which is ideal value for server with above configuration.
所以答案是——根据您的处理器占用和NUMA配置而定,下表将总结我上面解释的内容:
8 or less processors    ===> 0 to N (where N= no. of processors)
More than 8 processors  ===> 8
NUMA configured         ===> MAXDOP should not exceed no of CPU’s assigned to each 
                                 NUMA node with max value capped to 8
Hyper threading Enabled ===> Should not exceed the number of physical processors.
编辑:下面是一个快速而简单的TSQL脚本,用于生成MAXDOP设置的建议。
/*************************************************************************
Author          :   Kin Shah
Purpose         :   Recommend MaxDop settings for the server instance
Tested RDBMS    :   SQL Server 2008R2

**************************************************************************/
declare @hyperthreadingRatio bit
declare @logicalCPUs int
declare @HTEnabled int
declare @physicalCPU int
declare @SOCKET int
declare @logicalCPUPerNuma int
declare @NoOfNUMA int

select @logicalCPUs = cpu_count -- [Logical CPU Count]
    ,@hyperthreadingRatio = hyperthread_ratio --  [Hyperthread Ratio]
    ,@physicalCPU = cpu_count / hyperthread_ratio -- [Physical CPU Count]
    ,@HTEnabled = case 
        when cpu_count > hyperthread_ratio
            then 1
        else 0
        end -- HTEnabled
from sys.dm_os_sys_info
option (recompile);

select @logicalCPUPerNuma = COUNT(parent_node_id) -- [NumberOfLogicalProcessorsPerNuma]
from sys.dm_os_schedulers
where [status] = 'VISIBLE ONLINE'
    and parent_node_id < 64
group by parent_node_id
option (recompile);

select @NoOfNUMA = count(distinct parent_node_id)
from sys.dm_os_schedulers -- find NO OF NUMA Nodes 
where [status] = 'VISIBLE ONLINE'
    and parent_node_id < 64

-- Report the recommendations ....
select
    --- 8 or less processors and NO HT enabled
    case 
        when @logicalCPUs < 8
            and @HTEnabled = 0
            then 'MAXDOP setting should be : ' + CAST(@logicalCPUs as varchar(3))
                --- 8 or more processors and NO HT enabled
        when @logicalCPUs >= 8
            and @HTEnabled = 0
            then 'MAXDOP setting should be : 8'
                --- 8 or more processors and HT enabled and NO NUMA
        when @logicalCPUs >= 8
            and @HTEnabled = 1
            and @NoofNUMA = 1
            then 'MaxDop setting should be : ' + CAST(@logicalCPUPerNuma / @physicalCPU as varchar(3))
                --- 8 or more processors and HT enabled and NUMA
        when @logicalCPUs >= 8
            and @HTEnabled = 1
            and @NoofNUMA > 1
            then 'MaxDop setting should be : ' + CAST(@logicalCPUPerNuma / @physicalCPU as varchar(3))
        else ''
        end as Recommendations

编辑:对于未来的访问者,你可以查看test-dbamaxdop PowerShell函数(以及其他非常有用的DBA函数(全部免费!!))。

在设置MAXDOP时,通常希望将其限制在NUMA节点中的核心数量。这样,调度程序就不会尝试跨NUMA节点访问内存。

通过查看MSDN团队的帖子,我找到了一种可靠的方法来获取机器的物理核心数,并使用它来确定一个良好的MAXDOP设置。

所谓“良好”,是指保守的。也就是说,我的要求是在NUMA节点中最多使用75%的核心,或者总共最多使用8个核心。

SQL Server 2016(13.x)SP2及以上版本以及所有SQL Server 2017及以上版本都会显示每个插槽的物理核心数、插槽数和NUMA节点数的详细信息,从而可以整洁地确定新SQL Server安装的基线MAXDOP设置。

对于上述版本,此代码将建议将MAXDOP设置保守为NUMA节点中物理核心数的75%:

DECLARE @socket_count int;
DECLARE @cores_per_socket int;
DECLARE @numa_node_count int;
DECLARE @memory_model nvarchar(120);
DECLARE @hyperthread_ratio int;

SELECT @socket_count = dosi.socket_count
       , @cores_per_socket = dosi.cores_per_socket
       , @numa_node_count = dosi.numa_node_count
       , @memory_model = dosi.sql_memory_model_desc
       , @hyperthread_ratio = dosi.hyperthread_ratio
FROM sys.dm_os_sys_info dosi;

SELECT [Socket Count] = @socket_count
       , [Cores Per Socket] = @cores_per_socket
       , [Number of NUMA nodes] = @numa_node_count
       , [Hyperthreading Enabled] = CASE WHEN @hyperthread_ratio > @cores_per_socket THEN 1 ELSE 0 END
       , [Lock Pages in Memory granted?] = CASE WHEN @memory_model = N'CONVENTIONAL' THEN 0 ELSE 1 END;

DECLARE @MAXDOP int = @cores_per_socket;
SET @MAXDOP = @MAXDOP * 0.75;
IF @MAXDOP >= 8 SET @MAXDOP = 8;

SELECT [Recommended MAXDOP setting] = @MAXDOP
       , [Command] = 'EXEC sys.sp_configure N''max degree of parallelism'', ' + CONVERT(nvarchar(10), @MAXDOP) + ';RECONFIGURE;';
在SQL Server 2017之前的版本或SQL Server 2016 SP2之前的版本中,无法从 sys.dm_os_sys_info 中获取每个NUMA节点的核心数。相反,我们可以使用PowerShell来确定物理核心数量。
powershell -OutputFormat Text -NoLogo -Command "& {Get-WmiObject -namespace 
"root\CIMV2" -class Win32_Processor -Property NumberOfCores} | select NumberOfCores"
可以使用PowerShell来确定逻辑核心的数量,如果启用了超线程技术,那么逻辑核心的数量很可能是物理核心数量的两倍。
powershell -OutputFormat Text -NoLogo -Command "& {Get-WmiObject -namespace 
"root\CIMV2" -class Win32_Processor -Property NumberOfCores} 
| select NumberOfLogicalProcessors"
T-SQL:
/* 
   This will recommend a MAXDOP setting appropriate for your machine's NUMA memory
   configuration.  You will need to evaluate this setting in a non-production 
   environment before moving it to production.

   MAXDOP can be configured using:  
   EXEC sp_configure 'max degree of parallelism',X;
   RECONFIGURE

   If this instance is hosting a Sharepoint database, you MUST specify MAXDOP=1 
   (URL wrapped for readability)
   http://blogs.msdn.com/b/rcormier/archive/2012/10/25/
   you-shall-configure-your-maxdop-when-using-sharepoint-2013.aspx

   Biztalk (all versions, including 2010): 
   MAXDOP = 1 is only required on the BizTalk Message Box
   database server(s), and must not be changed; all other servers hosting other 
   BizTalk Server databases may return this value to 0 if set.
   http://support.microsoft.com/kb/899000
*/
SET NOCOUNT ON;

DECLARE @CoreCount int;
SET @CoreCount = 0;
DECLARE @NumaNodes int;

/*  see if xp_cmdshell is enabled, so we can try to use 
    PowerShell to determine the real core count
*/
DECLARE @T TABLE (
    name varchar(255)
    , minimum int
    , maximum int
    , config_value int
    , run_value int
);
INSERT INTO @T 
EXEC sp_configure 'xp_cmdshell';
DECLARE @cmdshellEnabled BIT;
SET @cmdshellEnabled = 0;
SELECT @cmdshellEnabled = 1 
FROM @T
WHERE run_value = 1;
IF @cmdshellEnabled = 1
BEGIN
    CREATE TABLE #cmdshell
    (
        txt VARCHAR(255)
    );
    INSERT INTO #cmdshell (txt)
    EXEC xp_cmdshell 'powershell -OutputFormat Text -NoLogo -Command "& {Get-WmiObject -namespace "root\CIMV2" -class Win32_Processor -Property NumberOfCores} | select NumberOfCores"';
    SELECT @CoreCount = CONVERT(INT, LTRIM(RTRIM(txt)))
    FROM #cmdshell
    WHERE ISNUMERIC(LTRIM(RTRIM(txt)))=1;
    DROP TABLE #cmdshell;
END
IF @CoreCount = 0 
BEGIN
    /* 
        Could not use PowerShell to get the corecount, use SQL Server's 
        unreliable number.  For machines with hyperthreading enabled
        this number is (typically) twice the physical core count.
    */
    SET @CoreCount = (SELECT i.cpu_count from sys.dm_os_sys_info i); 
END

SET @NumaNodes = (
    SELECT MAX(c.memory_node_id) + 1 
    FROM sys.dm_os_memory_clerks c 
    WHERE memory_node_id < 64
    );

DECLARE @MaxDOP int;

/* 3/4 of Total Cores in Machine */
SET @MaxDOP = @CoreCount * 0.75; 

/* if @MaxDOP is greater than the per NUMA node
    Core Count, set @MaxDOP = per NUMA node core count
*/
IF @MaxDOP > (@CoreCount / @NumaNodes) 
    SET @MaxDOP = (@CoreCount / @NumaNodes) * 0.75;

/*
    Reduce @MaxDOP to an even number 
*/
SET @MaxDOP = @MaxDOP - (@MaxDOP % 2);

/* Cap MAXDOP at 8, according to Microsoft */
IF @MaxDOP > 8 SET @MaxDOP = 8;

PRINT 'Suggested MAXDOP = ' + CAST(@MaxDOP as varchar(max));

一般来说,对于OLAP系统,使用较高的DOP(并行度),而对于OLTP系统,使用较低(或没有)DOP。许多系统介于两者之间,因此要找到一个适当的平衡点,以便偶尔出现的大工作负载能够获得足够的CPU资源快速完成,同时不会妨碍OLTP工作负载。 此外,使用cpu_count列获取核心数时要小心。如果启用了超线程技术,该列似乎反映的是暴露出来的逻辑处理器数量。一般来说,DOP不应超过物理核心数。将重负载的并行工作分散到逻辑处理器上只会增加开销,没有实际好处。 还有一个hyperthread_ratio列,但我不确定它代表什么。文档也没有很清楚地说明。我们系统上看到的数字表明它可能是整个系统中的物理核心数,或者每个芯片的逻辑处理器数量。文档声称我应该看到完全不同的数字。

1我相信hyperthread_ratio是每个处理器的逻辑核心数量。我前段时间遇到了这个问题,如果我记得正确的话,那就是我得出的结论。也许@AaronBertrand有更多关于此的信息。在验证之前,不要将其视为确定无疑的事实。 - Thomas Stringer
@ThomasStringer 根据文档所述,并通过在多台机器上运行,看起来就是这样。然而,从那列很难确定超线程是否真的已启用。例如,在我其中一台服务器上,它报告为8 - 该服务器有2个物理CPU,每个CPU上有4个核心,并启用了超线程。在没有超线程的机器上,在同样的情况下报告为4,但如果不重新启动(关闭超线程),你永远也看不到这种变化! - Hannah Vernon

我也遇到了这篇文章http://support.microsoft.com/kb/2806535,但是找不到与上面的脚本相关的内容。 另外,我想知道为什么存在"@logicalCPUs >= 8 and @HTEnabled = 1 and @NoofNUMA = 1"和"@logicalCPUs >= 8 and @HTEnabled = 1 and @NoofNUMA > 1"之间的区别,因为结果是相同的。 最后,我自己写了一段代码来匹配上面的文章,尽管在那里我希望能有更精确的定义和/或区分"处理器"、"CPU"和"物理处理器"。 随意发表你的见解。
/*************************************************************************
Author          :   Dennis Winter (Thought: Adapted from a script from "Kin Shah")
Purpose         :   Recommend MaxDop settings for the server instance
Tested RDBMS    :   SQL Server 2008R2

**************************************************************************/
declare @hyperthreadingRatio bit
declare @logicalCPUs int
declare @HTEnabled int
declare @physicalCPU int
declare @SOCKET int
declare @logicalCPUPerNuma int
declare @NoOfNUMA int
declare @MaxDOP int

select @logicalCPUs = cpu_count -- [Logical CPU Count]
    ,@hyperthreadingRatio = hyperthread_ratio --  [Hyperthread Ratio]
    ,@physicalCPU = cpu_count / hyperthread_ratio -- [Physical CPU Count]
    ,@HTEnabled = case 
        when cpu_count > hyperthread_ratio
            then 1
        else 0
        end -- HTEnabled
from sys.dm_os_sys_info
option (recompile);

select @logicalCPUPerNuma = COUNT(parent_node_id) -- [NumberOfLogicalProcessorsPerNuma]
from sys.dm_os_schedulers
where [status] = 'VISIBLE ONLINE'
    and parent_node_id < 64
group by parent_node_id
option (recompile);

select @NoOfNUMA = count(distinct parent_node_id)
from sys.dm_os_schedulers -- find NO OF NUMA Nodes 
where [status] = 'VISIBLE ONLINE'
    and parent_node_id < 64

IF @NoofNUMA > 1 AND @HTEnabled = 0
    SET @MaxDOP= @logicalCPUPerNuma 
ELSE IF  @NoofNUMA > 1 AND @HTEnabled = 1
    SET @MaxDOP=round( @NoofNUMA  / @physicalCPU *1.0,0)
ELSE IF @HTEnabled = 0
    SET @MaxDOP=@logicalCPUs
ELSE IF @HTEnabled = 1
    SET @MaxDOP=@physicalCPU

IF @MaxDOP > 10
    SET @MaxDOP=10
IF @MaxDOP = 0
    SET @MaxDOP=1

PRINT 'logicalCPUs : '         + CONVERT(VARCHAR, @logicalCPUs)
PRINT 'hyperthreadingRatio : ' + CONVERT(VARCHAR, @hyperthreadingRatio) 
PRINT 'physicalCPU : '         + CONVERT(VARCHAR, @physicalCPU) 
PRINT 'HTEnabled : '           + CONVERT(VARCHAR, @HTEnabled)
PRINT 'logicalCPUPerNuma : '   + CONVERT(VARCHAR, @logicalCPUPerNuma) 
PRINT 'NoOfNUMA : '            + CONVERT(VARCHAR, @NoOfNUMA)
PRINT '---------------------------'
Print 'MAXDOP setting should be : ' + CONVERT(VARCHAR, @MaxDOP)

不错的代码。我不确定你是否意识到sys.dm_os_sys_info中的hyperthread_ratio列是误导性的...例如,在我的工作站上,我有一个启用了超线程的单个4核CPU - 任务管理器看到8个逻辑CPU,而你的代码报告超线程比率为1。 - Hannah Vernon
顺便提一下,我的代码对这台机器的推荐是6,即使在最紧张的并行查询下,还会有2个核心可用。 - Hannah Vernon
hyperthread_ratio确实是一个问题,尽管我不知道是否有更好的解决办法。请参阅此博客以获取更多详细信息:http://sqlblog.com/blogs/kalen_delaney/archive/2007/12/08/hyperthreaded-or-not.aspx至于您的第二个帖子 - 很想知道您为您的机器选择了哪个“最大并行度”值。 :-D 我对这个话题还很陌生 - 只是因为之前不知道并且需要这些信息才偶然发现的。所以您的结论是,2个核心仍然可用是好事还是坏事? - Dennis Winter

这个版本使用现有的MAXDOP设置为您提供一个漂亮的单一结果集,并且不需要使用xp_cmdshell,可以在SQL 2008-2017版本上保持稳定。

select
[ServerName]                    = @@SERVERNAME
, [ComputerName]                = SERVERPROPERTY('ComputerNamePhysicalNetBIOS') 
, [LogicalCPUs]             
, hyperthread_ratio 
, [PhysicalCPU]             
, [HTEnabled]               
, LogicalCPUPerNuma
, [NoOfNUMA]
, [MaxDop_Recommended]          = convert(int,case when [MaxDop_RAW] > 10 then 10 else [MaxDop_RAW] end)
, [MaxDop_Current]              = sc.value
, [MaxDop_RAW]
, [Number of Cores] 
from
(
select
     [LogicalCPUs]              
    , hyperthread_ratio 
    , [PhysicalCPU]             
    , [HTEnabled]               
    , LogicalCPUPerNuma
    , [NoOfNUMA]
    , [Number of Cores] 
    , [MaxDop_RAW]              = 
        case
            when [NoOfNUMA] > 1 AND HTEnabled = 0 then logicalCPUPerNuma 
            when [NoOfNUMA] > 1 AND HTEnabled = 1 then convert(decimal(9,4),[NoOfNUMA]/ convert(decimal(9,4),Res_MAXDOP.PhysicalCPU) * convert(decimal(9,4),1))
            when HTEnabled = 0 then  Res_MAXDOP.LogicalCPUs
            when HTEnabled = 1 then  Res_MAXDOP.PhysicalCPU
        end
from
(
    select
         [LogicalCPUs]              = osi.cpu_count
        , osi.hyperthread_ratio 
        , [PhysicalCPU]             = osi.cpu_count/osi.hyperthread_ratio
        , [HTEnabled]               = case when osi.cpu_count > osi.hyperthread_ratio then 1 else 0 end
        , LogicalCPUPerNuma
        , [NoOfNUMA]
        , [Number of Cores] 
    from 
    (
        select
            [NoOfNUMA]  = count(res.parent_node_id)
            ,[Number of Cores]  = res.LogicalCPUPerNuma/count(res.parent_node_id)
            ,res.LogicalCPUPerNuma
        from
        (
            Select
                s.parent_node_id
                ,LogicalCPUPerNuma  = count(1)
            from
                sys.dm_os_schedulers s
            where
                s.parent_node_id < 64
                and
                s.status = 'VISIBLE ONLINE'
            group by 
                s.parent_node_id
        ) Res
        group by
            res.LogicalCPUPerNuma
    ) Res_NUMA
    cross apply sys.dm_os_sys_info osi
) Res_MAXDOP
)Res_Final
cross apply sys.sysconfigures sc
where sc.comment = 'maximum degree of parallelism'
option (recompile);

不错的脚本,但是kb文章:http://support.microsoft.com/kb/2806535与你的代码并不完全一致。我错过了什么吗? 服务器1 HTEnabled: 1 超线程比例: 12 逻辑处理器数: 24 物理处理器数: 2 每个NUMA的逻辑处理器数: 12 NUMA的数量: 2 MaxDop设置应为: 6 服务器2 HTEnabled: 2 超线程比例: 16 逻辑处理器数: 64 物理处理器数: 4 每个NUMA的逻辑处理器数: 16 NUMA的数量: 4 MaxDop设置应为: 4 我知道这只是建议,但对我来说,似乎有些不对劲,一个拥有4个处理器而不是2个,每个物理CPU有8个核心而不是6个的服务器(#2),竟然推荐将MAXDOP设置为4,而不是对于性能较低的服务器设置为6。 上面的kbb文章建议了8个我的场景。"对于配置了NUMA和启用了超线程的服务器,MAXDOP值不应超过每个NUMA节点的物理处理器数量。"

如果您将MAXDOP设置得高于核心/NUMA节点的数量,那么您将面临调用远程内存的情况,而这种调用比调用近程内存慢得多。这是因为每个NUMA节点都有自己的内存;如果一个查询使用的线程数超过单个NUMA节点中的核心数,那么CPU负载将分布在多个核心和多个内存节点上。 - Hannah Vernon
我建议将MAXDOP设置为适合您服务器负载的设置。只有您能确定最适合您特定负载的设置;本文仅供参考。 - Hannah Vernon