Recommended Reading: Windows Internals, 5th Edition
 |
This topic is a recommended reading. This article indicates a suggested or recommended reading that will provide you with accurate and important information for a better understanding of the covered topic. |
 |
Windows Internals, 5th Edition |
| Authors: Mark E. Russinovich, David A. Solomon, Alex Ionescu | |
| Publisher: Microsoft Press | |
| Released: June 15, 2009 | |
| Pages: 1264 | |
| Web pages: http://technet.microsoft.com/en-us/sysinternals/bb963901 http://shop.oreilly.com/product/9780735625303.do |
|
| Errata: http://oreilly.com/catalog/errata.csp?isbn=9780735625303 |
Book's Contents
Copyright
Dedication
Foreword
Acknowledgments
Introduction
Chapter 1. Concepts and Tools
Chapter 2. System Architecture
Chapter 3. System Mechanisms
Chapter 4. Management Mechanisms
Chapter 5. Processes, Threads, and Jobs
Chapter 6. Security
Chapter 7. I/O System
Chapter 8. Storage Management
Chapter 9. Memory Management
Chapter 10. Cache Manager
Chapter 11. File Systems
Chapter 12. Networking
Chapter 13. Startup and Shutdown
Chapter 14. Crash Dump Analysis
Glossary
Index
Why Reading?
Windows Internals is a must for everyone who wants to know how the Windows operating system works. You can see how the core components of the Windows operating system work behind the scenes and delve inside Windows architecture and internals:
- Understand how the core system and management mechanisms work from the object manager to services to the registry
- Explore internal system data structures using tools like the kernel debugger
- Grasp the scheduler's priority and CPU placement algorithms
- Go inside the Windows security model to see how it authorizes access to data
- Understand how Windows manages physical and virtual memory
- Tour the Windows networking stack from top to bottom including APIs, protocol drivers, and network adapter drivers
- Troubleshoot file-system access problems and system boot problems
- Learn how to analyze crashes
If you want to get key architectural insights on system design, debugging performance and support (along with hands-on experiments) this is the right book for you.
And of course you'll get precious information that will be useful to write TechNet Wiki Articles too: for example, chapter 2 ("System Architecture") made me know about the existence of a special Windows "feature" I wasn't aware of, the Checked Build (described in this article) and provided valuable information about the functionality provided by the different Windows components and subsystems (like the POSIX Subsystem, as you can read here); chapter 11 ("File Systems"), heavily and deeply describes the Windows file systems formats (as you can see in this article); finally, chapter 14 ("Crash Dump Analysis"), provides valuable information to understand what's behind a Windows bug check, how you can identify the Stop error that made a system crash and how you can troubleshoot it (as you can read here).
Some Words About the Authors...
Mark Russinovich is a technical fellow in the Windows Core Operating System Division. He is a member of the core team that provides architectural direction and oversight across Windows, with a focus on security and virtualization. Mark is currently working on the technical direction and architectural plan for Windows 8. He also continues developing tools for the Windows Sysinternals Web site, the most popular TechNet subsite, with 2 million downloads per month. Mark consistently delivers the top-rated sessions at major IT and developer conferences, including Microsoft TechEd, TechReady, and WinHEC. Mark has written dozens of magazine articles and serves as contributing editor for both Microsoft TechNet and Windows IT Pro magazines. Mark joined Microsoft in 2006 when the 85-person company he cofounded 10 years earlier, Winternals Software, was acquired along with Sysinternals. At Winternals he was chief software architect, defining the business and technical direction for the company.
David Solomon, president of David Solomon Expert Seminars (www.solsem.com), has focused on explaining the internals of the Microsoft Windows NT operating system line since 1992. He has taught his world-renowned Windows internals classes to thousands of developers and IT professionals worldwide. His clients include all the major software and hardware companies, including Microsoft. He was nominated a Microsoft Most Valuable Professional in 1993 and from 2005–2008. Prior to starting his own company, David worked for nine years as a project leader and developer in the VMS operating system development group at Digital Equipment Corporation. His first book was entitled Windows NT for Open VMS Professionals (Digital Press/Butterworth Heinemann, 1996). It explained Windows NT to VMS-knowledgeable programmers and system administrators. His second book, Inside Windows NT, Second Edition (Microsoft Press, 1998), covered the internals of Windows NT 4 0. Since the third edition (Inside Windows 2000) David has coauthored this book series with Mark Russinovich. In addition to organizing and teaching seminars, David is a regular speaker at technical conferences such as Microsoft TechEd and Microsoft PDCs. He has also served as technical chair for several past Windows NT conferences.
Alex Ionescu is the founder of Winsider Seminars & Solutions Inc., specializing in low-level system software for administrators and developers. He also teaches Windows internals courses for David Solomon Expert Seminars, including at Microsoft. Alex was the lead kernel developer for ReactOS, an open source clone of Windows XP/Windows Server 2003 written from scratch, for which he wrote most of the Windows NT–based kernel. Alex is also very active in the security research community, discovering and reporting several vulnerabilities related to the Windows kernel and presenting talks at conferences such as Blackhat and Recon. Alex’s experience in operating system design and kernel coding dates back to his early adolescence, when he first played with John Fine’s educational operating system and kernel and boot loader code. Since then he has been active in the area of NT kernel development, offering help and advice for driver developers, as well as in the NT reverse engineering and security felds, where he has published a number of articles and source code, such as documentation for the Linux NTFS project, extensive papers on the Visual Basic metadata and pseudo-code format and NTFS structures and data streams. In the last three years, he has contributed to patches and development in two major commercially used operating system kernels.
...and From the Authors: Q & A
I've asked some questions to the book's authors, talking about some of the covered topics and trying to relate them to Windows actual state and evolution.
Important: these answers are the authors' and do not represent the view of Microsoft.
Q.: "Windows Kernel versus MinWin", a full featured operating system versus something minimal but fully bootable, 25-40 MB of disk space, that contains the executive systems, memory management, networking, and optional file system drivers. Could the future of Windows kernel be something that could be modularized and built in various ways, according to variable parameters and factors?
Alex Ionescu: The Windows kernel was always designed in such a way that it was modularized – it’s actually built as a variety of libraries, and only linked into one big binary at the end. Strict separation between the modules usually exists, and each has their own set of headers, structures, etc, that usually other modules do not have access to, unless there are internal APIs from one module to the other. So in one way, the benefits of a modularized architecture already exist. The kernel is also built in various ways, there are PAE builds (for Physical Address Extension and No-Execute support), there were UP vs MP builds (uni vs multi-processor), and some of the kernel modules (such as Cm – registry) are always built into the boot loader and some other tools. But past that, there wouldn’t be any major benefits into actually making separate binaries for the kernel (it’s already a small module, less than 5MB). Minwin is more about modularizing and creating dependencies in user-mode, where things are much less well structured and designed as in kernel-mode.
Q.: Windows provides support for several file system formats, but the primary recommended choice is always NTFS. We've heard something about two "mysterious" entities in this area, whose names are WinFS and Protogon, but everything seems to be disappeared. What can we say about the future of Windows file systems? Will NTFS continue to be the primary choice or there's something new and improved that is going to come?
Alex Ionescu: WinFS was officially cancelled (and part of an object-oriented file system model that Microsoft has been pursuing ever since Windows 2000) but much of the technology behind it did make it into other products. As for Protogon, such a file system does seem to be part of Windows 8 but I can’t provide any information on it – developer previews often have technologies which may or may not make it into the final product, and I’m sure if something better than NTFS will come along and ship, Microsoft will make an official announcement to prepare everyone for the changes. It’s worth nothing that what we call “NTFS” today is quite different from the NTFS that shipped initially, so even if we give the file system the same name, it has gone through several stages of improvements, such as the self-healing and transactional capabilities added in Vista, and the constant performance improvements.
Q.: Sometimes users may think that a Windows crash is someway a sort of weakness of the operating system while facing an unexpected problem; we also know that the OS "...adopts a fail fast policy in attempting to prevent the corruption in RAM from spreading to disk" (quoting chapter 14). According to data related to the crash dump analysis, the main causes for system to crash are related to page faults, power management issues and access violations: drivers are often the main suspects. What are the improvements that the OS can provide to reduce the impact of these crashes and what is still up to software developers only?
Alex Ionescu: Adopting a micro-kernel approach where drivers run in an isolated environment is academically one of the preferred solutions. However, the performance implications of such a design change are quite drastic (they have been getting better, however) and the application compatibility issues, for Windows, may be too severe to ever go down this path. An interim solution, which still needs to mature a bit more, is the User-Mode Driver Framework that is part of Windows Driver Foundation (WDF) – this is a mechanism which allows developers to write C++ COM drivers that run in a user-mode hosting process, instead of running in the kernel. UMDF is still limited to only a few classes of devices however, but it can still help reduce the amount of blue screens of death on a Windows machine. Another step in this direction is the Kernel-Mode Driver Framework also part of WDF – although this still lets developers push their drivers into the kernel, most of the “hard work” that WDM driver writers had to do (and which was crash-prone) is now done by the framework, which is written by Microsoft (and thus has overall a higher quality). Making the code that driver developers have to write a lot easier to write, and able to run in a more constrained environment can definitely reduce catastrophic crashes as well. Finally, producing tools to test, fuzz, and exercise drivers allows driver developers to catch crashes before they ship, and tools such as Driver Verifier and the WHQL process at Microsoft are doing just that. In fact, you will probably find today that most driver crashes are either going to be in old, legacy drivers that are no longer maintained, malicious drivers (such as rootkits), and video card drivers (simply because of the immense complexity required to build such a driver – case in point, the video driver on my machine is 14MB, three times larger than the kernel itself!).
Q: Sysinternals tools constitute a set of programs that are absolutely a must for any system power user and/or administrator and that complete the operating system's programs equipment. Do you think that they could be integrated as part of the operating system in a future release?
Mark Russinovich: I joined Microsoft in 2006, so didn’t have a chance to impact Vista, but you can see the influence of the Sysinternals tools in utilities like the Resource Monitor. You can search for a file handle, for example, a feature derived from the handle search in Process Explorer that enables you to find the process keeping a file locked, for example. Windows 8’s Task Manager shows even more influence from the tools and from Windows Internals.
Q.: Windows Internals is still, year after year, a key reference for any Windows users who wants to know "how it is made and how it works": could you tell us something about the next edition?
Mark Russinovich: The next edition, the 6th, covers Windows 7 and Windows Server 2008 R2. The previous edition, at over 1200 pages, already stretched the bounds of a single book, so with the additional content we’ve had to split the book into two parts. Part 1 will come out in early 2012 with part 2 following a couple of months later. Because of the complexity and additional length required to describe changes across multiple versions of the operating system, we’ve decided to make this edition focus exclusively on these latest versions and leave the 5th edition for those interested in Windows XP and Windows Vista-generation releases.
Acknowledgements
While waiting to read the new 6th edition, I want to thank in a special way Mark Russinovich, David Solomon and Alex Ionescu for their precious help in writing this page: their answers are the value added of this article.