MySQL PERFORMANCE_SCHEMA implementation.

Introduction

The PERFORMANCE_SCHEMA is a way to introspect the internal execution of the server at runtime. The performance schema focuses primarily on performance data, as opposed to the INFORMATION_SCHEMA whose purpose is to inspect metadata.

From a user point of view, the performance schema consists of:

a dedicated database schema, named PERFORMANCE_SCHEMA,
SQL tables, used to query the server internal state or change configuration settings.

From an implementation point of view, the performance schema is a dedicated Storage Engine which exposes data collected by 'Instrumentation Points' placed in the server code.

Multiple interfaces

The performance schema exposes many different interfaces, for different components, and for different purposes.

Instrumenting interface

All the data representing the server internal state exposed in the performance schema must be first collected: this is the role of the instrumenting interface. The instrumenting interface is a coding interface provided by implementors (of the performance schema) to implementors (of the server or server components).

This interface is available to:

C implementations
C++ implementations
the core SQL layer (/sql)
the mysys library (/mysys)
MySQL plugins, including storage engines,
third party plugins, including third party storage engines.

For details, see the instrumentation interface page.

Compiling interface

The implementation of the performance schema can be enabled or disabled at build time, when building MySQL from the source code.

When building with the performance schema code, some compilation flags are available to change the default values used in the code, if required.

For more details, see:

./configure --help

To compile with the performance schema:

./configure --with-perfschema

The implementation of all the compiling options is located in

./storage/perfschema/plug.in

Server startup interface

The server startup interface consists of the "./mysqld ..." command line used to start the server. When the performance schema is compiled in the server binary, extra command line options are available.

These extra start options allow the DBA to:

enable or disable the performance schema
specify some sizing parameters.

To see help for the performance schema startup options, see:

./sql/mysqld --verbose --help

The implementation of all the startup options is located in

./sql/mysqld.cc, my_long_options[]

Server bootstrap interface

The bootstrap interface is a private interface exposed by the performance schema, and used by the SQL layer. Its role is to advertise all the SQL tables natively supported by the performance schema to the SQL server. The code consists of creating MySQL tables for the performance schema itself, and is used in './mysql –bootstrap' mode when a server is installed.

The implementation of the database creation script is located in

./scripts/mysql_system_tables.sql

Runtime configuration interface

When the performance schema is used at runtime, various configuration parameters can be used to specify what kind of data is collected, what kind of aggregations are computed, what kind of timers are used, what events are timed, etc.

For all these capabilities, not a single statement or special syntax was introduced in the parser. Instead of new SQL statements, the interface consists of DML (SELECT, INSERT, UPDATE, DELETE) against special "SETUP" tables.

For example:

mysql> update performance_schema.SETUP_INSTRUMENTS
  set ENABLED='YES', TIMED='YES';
Query OK, 234 rows affected (0.00 sec)
Rows matched: 234  Changed: 234  Warnings: 0

Internal audit interface

The internal audit interface is provided to the DBA to inspect if the performance schema code itself is functioning properly. This interface is necessary because a failure caused while instrumenting code in the server should not cause failures in the MySQL server itself, so that the performance schema implementation never raises errors during runtime execution.

This auditing interface consists of:

SHOW ENGINE PERFORMANCE_SCHEMA STATUS;

It displays data related to the memory usage of the performance schema, as well as statistics about lost events, if any.

The SHOW STATUS command is implemented in

./storage/perfschema/pfs_engine_table.cc

Query interface

The query interface is used to query the internal state of a running server. It is provided as SQL tables.

For example:

mysql> select * from performance_schema.EVENTS_WAITS_CURRENT;

Design principles

No behavior changes

The primary goal of the performance schema is to measure (instrument) the execution of the server. A good measure should not cause any change in behavior.

To achieve this, the overall design of the performance schema complies with the following very severe design constraints:

The parser is unchanged. There are no new keywords, no new statements. This guarantees that existing applications will run the same way with or without the performance schema.

All the instrumentation points return "void", there are no error codes. Even if the performance schema internally fails, execution of the server code will proceed.

None of the instrumentation points allocate memory. All the memory used by the performance schema is pre-allocated at startup, and is considered "static" during the server life time.

None of the instrumentation points use any pthread_mutex, pthread_rwlock, or pthread_cond (or platform equivalents). Executing the instrumentation point should not cause thread scheduling to change in the server.

In other words, the implementation of the instrumentation points, including all the code called by the instrumentation points, is:

malloc free
mutex free
rwlock free

TODO: All the code located in storage/perfschema is malloc free, but unfortunately the usage of LF_HASH introduces some memory allocation. This should be revised if possible, to use a lock-free, malloc-free hash code table.

No performance hit

The instrumentation of the server should be as fast as possible. In cases when there are choices between:

doing some processing when recording the performance data in the instrumentation,
doing some processing when retrieving the performance data,

priority is given in the design to make the instrumentation faster, pushing some complexity to data retrieval.

As a result, some parts of the design, related to:

the setup code path,
the query code path,

might appear to be sub-optimal.

The criterion used here is to optimize primarily the critical path (data collection), possibly at the expense of non-critical code paths.

Unintrusive instrumentation

For the performance schema in general to be successful, the barrier of entry for a developer should be low, so it's easy to instrument code.

In particular, the instrumentation interface:

is available for C and C++ code (so it's a C interface),
does not require parameters that the calling code can't easily provide,
supports partial instrumentation (for example, instrumenting mutexes does not require that every mutex is instrumented)

Extendable instrumentation

As the content of the performance schema improves, with more tables exposed and more data collected, the instrumentation interface will also be augmented to support instrumenting new concepts. Existing instrumentations should not be affected when additional instrumentation is made available, and making a new instrumentation available should not require existing instrumented code to support it.

Versioned instrumentation

Given that the instrumentation offered by the performance schema will be augmented with time, when more features are implemented, the interface itself should be versioned, to keep compatibility with previous instrumented code.

For example, after both plugin-A and plugin-B have been instrumented for mutexes, read write locks and conditions, using the instrumentation interface, we can anticipate that the instrumentation interface is expanded to support file based operations.

Plugin-A, a file based storage engine, will most likely use the expanded interface and instrument its file usage, using the version 2 interface, while Plugin-B, a network based storage engine, will not change its code and not release a new binary.

When later the instrumentation interface is expanded to support network based operations (which will define interface version 3), the Plugin-B code can then be changed to make use of it.

Note, this is just an example to illustrate the design concept here. Both mutexes and file instrumentation are already available since version 1 of the instrumentation interface.

Easy deployment

Internally, we might want every plugin implementation to upgrade the instrumented code to the latest available, but this will cause additional work and this is not practical if the code change is monolithic.

Externally, for third party plugin implementors, asking implementors to always stay aligned to the latest instrumentation and make new releases, even when the change does not provide new functionality for them, is a bad idea.

For example, requiring a network based engine to re-release because the instrumentation interface changed for file based operations, will create too many deployment issues.

So, the performance schema implementation must support concurrently, in the same deployment, multiple versions of the instrumentation interface, and ensure binary compatibility with each version.

In addition to this, the performance schema can be included or excluded from the server binary, using build time configuration options.

Regardless, the following types of deployment are valid:

a server supporting the performance schema + a storage engine that is not instrumented
a server not supporting the performance schema + a storage engine that is instrumented