Performance Issues with Transparent Huge Pages (THP)
By Michele Casey on Sep 17, 2013
By Michele Casey on Sep 17, 2013
One key reason customers choose Oracle Linux is the investment Oracle makes in optimizing Linux for demanding workloads. Part of this work includes extensive testing for I/O intensive workloads, like the Oracle Database. A common test used at Oracle to measure I/O throughput and performance is the Oracle Orion Calibration Tool, which simulates the I/O workloads of an Oracle Database. Recently, the Oracle Linux team discovered a significant lag in I/O performance, which was later traced to a feature in the Linux kernel, Transparent Huge Pages (also known as THP). During these tests, our engineering teams discovered THP introduced a 30% performance degradation in I/O throughput. When Transparent Huge Pages were removed from the kernel, performance values returned to their expected levels.
Our engineers have been working with the community to identify the root cause of this performance problem, which also impacts other projects, such as Hadoop. Oracle’s kernel engineers have traced the performance issues with Transparent Huge Pages to its use of swappable hugepages. Prior to THP, hugepages were not swappable. It appears the overhead generated by this implementation is the root cause of the deterioration seen in performance when the feature is enabled (compiled) in the kernel. Our kernel engineers are working on several changes to the code which they believe will improve performance with THP, but the final resolution will take time to develop, test and release.
Due to the impact this feature has on performance with applications like the Oracle Database and the timing of a final fix, Oracle Linux will disable Transparent Huge Pages in the Unbreakable Enterprise Kernel, beginning with Quarterly Update 5 for the Unbreakable Enterprise Kernel, Release 2 (version 2.6.39-400.209.1). To maintain binary compatibility with the Red Hat distribution, the Red Hat Compatible kernel will be left as is, with Transparent Huge Pages enabled. In addition, the Oracle Database team has released an alert recommending all customers disable THP in all Linux systems running RAC and even those systems running single instances, due to instability. Please note that THP is not used by the Oracle Database, the recommendation is to use huge pages. Additional information for huge pages can be found in your Oracle Database documentation or on My Oracle Support.
Our engineers will continue to work with others in the community on a final resolution to the performance issues seen with Transparent Huge Pages. Once a fix is available, we will re-enable this feature in the Unbreakable Enterprise Kernel.
