SQL Server 2014 & 2016 Developer Edition
Microsoft SQL Server has been in developer edition as far back as 2005 (if my memory serves me); however each license was $50.00. Now with SQL Server 2014 & 2016, Microsoft has made developer edition free.
There is no better time to get on board with SQL Server 2016, there are lots of new functionality and features available. That can help your organization reach new levels of performance, scalability, and data insight.
Download and start playing with SQL 2014 here.
SQL Server 2016 RC3 Released
SQL Server 2016 RC3 has been released, it is the final release before RTM. In this release no major functionality is introduced.
Download from here.
SQLPASS Edmonton (Apr. 9, 2016) – Key Note Presentation
Friday/Saturday was a great day, got to meet lots of wonderful people from US/Canada at the Alberta’s first SQL Saturday. I delivered they Key Note speech for SQL Server 2016. I would love to hear back from people, so please provide evaluation on SQL Saturday site and you can download the PowerPoint deck here.
How to get orphaned logins for all databases?
Following script gives you all user database users, that do not have a server login mapped or where database user name does not match server login name. This is follow up post to How get all users and their role mappings from all databases? I posted few days ago.
IF EXISTS(SELECT * FROM tempdb.sys.tables WHERE name like '#Output%')
DROP TABLE #Output
GO
CREATE TABLE #Output (DatabaseName VARCHAR(255), UserLoginSID varbinary(128), ServerLoginName VARCHAR(255), DatabaseUserName VARCHAR(255), UserType VARCHAR(50))
GO
sp_msforeachdb '
INSERT INTO #Output
SELECT ''?''
, DP.sid
, SP.name
, DP.name
, DP.type_desc
FROM sys.database_principals DP
LEFT JOIN sys.server_principals SP
ON DP.sid = SP.sid
WHERE DP.type_desc IN (''SQL_USER'',''WINDOWS_USER'')
AND DP.sid IS NOT NULL'
GO
SELECT *
FROM #Output
WHERE ServerLoginName IS NULL
OR ServerLoginName <> DatabaseUserName
ORDER BY DatabaseName, ServerLoginName
GO
This post is cross posted on my SQLCAN Blog, MSDN Blog, and SQL server Consultation blog.
How get all users and their role mappings from all databases?
IF EXISTS(SELECT * FROM tempdb.sys.tables WHERE name like '#Output%')
DROP TABLE #Output
GO
CREATE TABLE #Output (DatabaseName VARCHAR(255), RoleName VARCHAR(255), UserName VARCHAR(255))
GO
sp_msforeachdb '
INSERT INTO #Output
SELECT ''?''
, DR.name
, DP.name
FROM [?].sys.database_principals AS DR
JOIN [?].sys.database_role_members AS DRM
ON DR.principal_id = DRM.role_principal_id
JOIN [?].sys.database_principals AS DP
ON DP.principal_id = DRM.member_principal_id
WHERE (DP.name <> ''dbo'')'
GO
SELECT *
FROM #Output
GO
Microsoft SQL Server 2016 RC2 Released
In SQL Server 2016 RC 2, enhancements include:
- R Services setup – the setup process for R Services is much more integrated into SQL Server setup. There is no longer a need to manually download and install Microsoft R open and R Server if the SQL Server is connected to the Internet; it becomes part of the SQL Server install sequence.
- SQL Server Management Studio (SSMS) – This release of SSMS features an update to the Visual Studio 2015 shell bringing enhancements such as the quick launch toolbar and improved theming support.
- Mobile reports – Brand Packages will now be downloaded to the mobile report publisher from a server running RC2 and available for use in report creation. Basic mobile report content migration between servers is now supported.
Changing @@SERVERNAME causes SQL Server Backups to Fail for AlwaysOn Availability Group
One of customers changed the value returned from @@SERVERNAME. SQL Server works no problem, however an unexpected behavior appeared. Changing the value for @@SERVERNAME, caused the backups to fail.
Looking at the maintenance jobs, found all jobs completed successfully and without issues. However, upon looking at the database’s statics it states no backups completed.
Because the database in question is part of AlwaysOn Availability Group (AG); SQL Server executes sys.fn_hadr_backup_is_preferred_replica to determine if the backup should take place on the current node. However, it returns value of 0 for all databases, if the preferred replica is set. Because, the script makes a check that is running on the server that is preferred. It does this by comparing the value to @@SERVERNAME to value of replica_server_name in sys.availability_replicas. Because value will never match, it skips the database on both primary and secondary replica.
I have created a Microsoft Connect article (link); asking this little bit of information to be added to Books Online article (link). There was a request submitted by Ola Hallengren (Blog | Twitter), which was closed as Won’t Fix (link). Please vote!
This post is cross posted on my SQLCAN Blog, MSDN Blog, and SQL server Consultation blog.
sys.foreign_keys does not have matching row in sys.indexes
Running DBCC CHECKDB you are getting following error message:
Check Catalog Msg 3853, State 1: Attribute (referenced_object_id=194099732,key_index_id=7) of row (object_id=2040565179) in sys.foreign_keys does not have a matching row (object_id=194099732,index_id=7) in sys.indexes
This error means, that Unique key constraint (index_id 7) in the primary table (object_id 194099732) is missing, which was referenced by child table’s FK constraint (FK object_id 2040565179). This should not happen, SQL Server will not allow you to drop a constraint that is referenced by FK. If attempted should get following error message:
Msg 3723, Level 16, State 6, Line 1
An explicit DROP INDEX is not allowed on index ‘dbo.a.NonClusteredIndex-20151119-085219’. It is being used for FOREIGN KEY constraint enforcement.
So if we are suppose to get errors? Why do we have corruption; simple answer, someone be making updates to system tables directly, which is not allowed or supported!
Actually we are not able to update system tables in SQL Server 2005+ (ref), however in SQL 2000 days, we had setting called allow updates in sp_configure options. Also supported by the KB2787112.
So question is how do you fix it?
First, identify the child table name from sys.foreign_keys:
SELECT object_name(parent_object_id) AS TableName FROM sys.foreign_keys WHERE name = 'FK_b_a'
Second, script our constraint definition:
- Find the table, we got in SQL Statement above.
- Go to Keys.
- Right click on FK constraint name.
- Script Key As.
- Create To.
- New Query Window.
Third, drop the FK constraint:
ALTER TABLE [schema].[tablename] DROP CONSTRAINT [fk_constraint_name]
Fourth, Re-create the constraint, with script generated in Step 2.
If it was issue of someone playing around in system tables, this should resolve it. However, if you get error similar to below:
Msg 1776, Level 16, State 0, Line 1
There are no primary or candidate keys in the referenced table ‘dbo.a’ that match the referencing column list in the foreign key ‘FK_b_a’.
This means, that the key is missing in parent table and appropriate index needs be created before FK constraint can be created. Since SQL doesn’t allow the index to be dropped there most likely are other corruption issues that have gone unnoticed. If that is an issue, you will have to rely on your backups for recovery.
This post is cross posted on my SQLCAN Blog, MSDN Blog, and SQL server Consultation blog.
Index Creation Date
Often when troubleshooting performance related issues, it is beneficial to know when new indexes were introduced. However currently there is no easy way to keep track of this information or attain this information. That’s why one of my colleagues and a good friend, Mohamed Sharaf (Blog | Twitter), has submitted a request on Microsoft Connect, Expose index creation date in sys.indexes, Link. Please take a few minutes to vote if you agree 
.
This post is cross posted on my SQLCAN Blog, MSDN Blog, and SQL server Consultation blog.
SQL Server 2012 NUMA Node Imbalance–Conclusion
Final post in the series, I hope to summarize my findings for everyone. Thanks for all the comments, feedback and comments I have received to-date on these posts.
Before providing the summary of all the testing, research and reading, I want to provide some internals for folks.
NUAM Memory Allocation
If a server has NUMA configuration, when we define the max memory setting. All memory gets allocated evenly from all NUMA nodes. For example, if I allocate 8GB from a server that has 4 NUMA nodes, it will allocate 2GB from each NUMA node for the instance in question. So be careful, if you are using CPU affinity. As even using CPU affinity, lets say you only want to use NUMA 1 for an instance; this can lead to foreign memory access. NUMA allocation still happens across all NUMA nodes (at least in SQL Server 2014). So even though you might only have subset of schedulers online, memory will still be accessed across all NUMA Nodes. Even though with new hardware foreign memory access does not have major impact on performance. Still recommend avoiding it. If you must use CPU affinity, then allocate sub-set of the processor from each NUMA node.
Lazy Writer Responsibility
Lazy Writer is back ground process and a thread is created per physical NUMA node (for Soft-NUMA, we only get one thread, you can read details here by Jonathan Kehayias (Blog|Twitter)). Lazy Writer responsibility is to make sure sufficient number of free pages, about 640 pages, are always available for new data to be loaded. If Lazy Writer thread is spinning, that is usually a clear signal of internal memory pressure.
Note: Internal Memory pressure does not mean server doesn’t have enough memory. There are multiple factors that can contribute to this, one fact is low Max Server Memory, which is what has been explored in these articles.
Original Scenario: SQL Server 2012 Latest Build + Windows 2008 R2 Latest Build + Locked Pages In Memory + Low Max Server Memory + AMD Processors + NUMA Configuration
In this build, I can consistently reproduce the issue. If I set the Max Memory low enough, it caused the Lazy Writer on NUMA node 0 to spin non-stop, as I documented in SQL Server 2012 NUMA Node Imbalance – Cont’d. To further to this I understand some additional internals around this thanks to colleague who helped here. As I was not true sure what is happening.
If we looked at the DBCC MEMORYSTATUS, in particular we notice following for Node 0:
Memory node Id = 0 KB
—————————————- ———–
VM Reserved 127086340
VM Committed 642452
Locked Pages Allocated 55996
Pages Allocated 54008
Pages Free 0
Target Committed 512000
Current Committed 698448
Foreign Committed 0
Away Committed 0
Taken Away Committed 0
Review the two numbers high-lighted above, Target memory is what this NUMA node is allocated and Current is what is NUMA consuming, therefore the Page Free count is 0. Thus Lazy Writer spinning to free up memory. However it is unable, thus spinning indefinitely causing CPU to pin. Question to answer, why can’t Lazy Writer free up memory from NUMA Node 0?
Looking further into MEMORYSTATUS, we also notice, that most of the memory allocated into NUMA Node 0 is for buffer cache:
MEMORYCLERK_SQLBUFFERPOOL (node 0) KB
—————————————- ———–
VM Reserved 103971820
VM Committed 524288
Locked Pages Allocated 2524
SM Reserved 0
SM Committed 0
Pages Allocated 8
Test Scenario #1: Windows 10 + SQL Server 2012 RTM + Low Max Server Memory + Intel Processors + No-NUMA (Physical)
In this, I was just trying to reproduce the issue on my desktop, with no luck. Memory allocation or such were nominal.
Test Scenario #2: Windows 2012 + SQL Server 2014 RTM + Low Max Server Memory + Intel Processors + No-NUMA (Hyper-V)
Again no issue, as per the title of the series, NUMA Imbalance. It was obvious and expected this two scenarios don’t show any issues. But I wanted to confirm it.
Test Scenario #3: SQL Server 2008 R2 + Windows 2008 R2 + Low Max Server Memory + AMD Processors
These test was on same servers as original scenario, however we did not notice any issue.
*** During testing, we realized an interesting fact about NUMA configuration in SQL Server 2008 R2. SQL Server swaps NUMA 0 and NUMA 1 CPU mapping when creating a logical mapping for schedulers. For example, if you have 2 NUMA Nodes each with 4 cores; SQL Server binds NUMA 0 – Processor 0 – 3 to Scheduler 4 – 7 and NUMA 1 – Processor 4 – 7 to Scheduler 0 – 3. This is considered NUMA Swap, documented in this MSDN blog article here by CSS Team. However this behavior does not seem to persist in SQL Server 2012.
Test Scenario #4: SQL Server 2014 + Windows 2012 R2 + Low Max Server Memory + Intel Processors + NUMA (Azure IaaS Virtual Machine)
This was interesting case, when I set the Max Server Memory to low value, the NUMA node 0’s processor time started to increase. However it was not as swear as SQL Server 2012, however these are different processors. I had same behavior as noticed above, Current Committed was higher then Target Committed, therefore NUMA thread on Node 0 was spinning. However interesting thing note, it was not as aggressive as SQL Server 2012. It was not continuously spinning, it allowed CPU to breath. However would wake up frequently as low as ever 20ms.
!!! Eliminated Locked Pages in Memory as a factor for NUMA Imbalance.
Test Scenario #5: SQL Server 2014 + Windows 2012 R2 + Low Max Server Memory + Intel Processors + Locked Pages In Memory + NUMA (Azure IaaS Virtual Machine)
Exact same behavior as above was noticed, however, after restart, SQL Server no longer had Current Committed higher then Target Committed. Even after lowering the max memory even further then Scenario #4. It did not cause as dramatic shift.
*** Another interesting thing to note in SQL Server 2014. Up-to SQL Server 2012, Lazy Writer Thread bound to CPU 0 of each NUMA Node. In SQL Server 2014, Lazy Writer Thread is binding to last CPU of each NUMA Node.
Conclusion & Things to Consider
- Configuring SQL Server Max Server Memory low memory will cause Lazy Writer to spin due to internal memory pressure issue.
- There is no value at which point this behavior starts, it depends on the load of the server. For example, in Original Scenario, due to server load under 4000MB it started spinning the Lazy Writer thread; however in Test Scenario #4, I had to go down to as much as 1000MB. Therefore on your servers look at Lazy Writer thread in conjunction with CPU and DBCC MEMORYSTATUS to understand what is truly happening.
- This behavior is consistent in whether I have Locked Pages in Memory or not.
- In Windows 2012 R2 NUMA allocation seem to be much better then Windows 2008 R2.
- SQL Server 2014 Lazy Writer thread does not seem to be as aggressive as SQL Server 2012.
- Few KB articles came up during my research in SQL Server 2012 that fix NUMA related issues, KB2819662 and KB2926223.
- There as been some fixes released for NUMA for Windows 2008 R2 operating system also, please review KB2155311 and KB2510206.
- In addition, if you are running a server with more then 64-logical cores, please review K-Group Configuration, for details please reference this article.
- Last but not least verify that memory is distributed evenly on physical server. That is the memory banks have been evenly allocated, i.e. you do not have more memory allocated in one bank versus another, as you might also see one NUMA node working extra harder if there is significant amount of foreign memory access.
Outstanding Question
Question to answer, why can’t Lazy Writer free up memory from NUMA Node 0?
SQL Server Lazy Writer is only responsible for clearing memory from Buffer Pool, it cannot remove or clear memory from any other memory clerk. Therefore, it is possible, that other internal components have memory allocated on NUMA Node 0 only, therefore NUMA Node 0 is unable to free up enough memory for buffer pool only. Looking at DBCC MEMORYSTATUS we can see which memory clerks allocate from NODE Node 0 only, there are few, for example, MEMORYCLERK_SQLCLR, MEMORYCLERK_SOSMEMMANAGER, OBJECTSTORE_LOCK_MANAGER, and MEMORYCLERK_XE_BUFFER all have allocation from NODE Node 0 only.
This post is cross posted on my SQLCAN Blog, MSDN Blog, and SQL server Consultation blog.
