Google Sponsors Improvements to FreeBSD's Performance Measurement Toolkit

By Murray Stokely, Software Engineer and FreeBSD Committer

Most CPUs today have built in performance measurement counters (PMCs) that can measure interesting low-level hardware events such as cache misses, branch mispredicts, and tlb misses. FreeBSD's HWPMC(4) driver "virtualizes" these hardware counters, allowing multiple processes to use them, and for multiple hardware counters to be concurrently active. Both simple counting and sampling (profiling) are supported, along with multi-CPU operation. FreeBSD's performance measurement toolset, PmcTools, is built using HWPMC.

PmcTools helps answer the following broad questions:

• What is the system doing at this point of time? (e.g. "What hardware events are being seen in unusual numbers?")
  
• Which part of the system are the symptoms associated with? (e.g. "Which are the 'hot' locations in the source?")

Recently, Google sponsored the development of an oft requested enhancement to FreeBSD's PmcTools: that of capturing the call chains leading to "hot" locations in the code. Call chains provide additional insight into the behavior of the system; in addition to determining the "hot" locations in the code, developers gain insight into why these locations became "hot" in the first place.

HWPMC and associated userland tools have been invaluable to the FreeBSD community in improving the scalability and performance of the upcoming FreeBSD 7 release. Kris Kennaway of the FreeBSD Project notes that "hwpmc is one of our most powerful tools for measuring and understanding CPU performance on FreeBSD. Support for profiling of call graphs was an important missing piece that will simplify the ability of developers to analyze performance bottlenecks in the kernel and in application code". Kip Macy notes that hwpmc has been invaluable in his 10 Gigabit Ethernet tuning efforts, and Arun Sharma notes that this work was particularly successful because it was quickly merged and is available out of the box to users of FreeBSD.

Check out this latest and greatest addition to PmcTools and let the FreeBSD community know what you think!