Skip to content

MDEV-19629 Implement MySQL 8.0 native functions: format_bytes(), format_pico_time() and ps_thread_id() - #6080

Open
Shubhamnish1030 wants to merge 2 commits into
MariaDB:mainfrom
Shubhamnish1030:mdev-19629-ps-thread-id
Open

Shubhamnish1030 wants to merge 2 commits into
MariaDB:mainfrom
Shubhamnish1030:mdev-19629-ps-thread-id

Conversation

@Shubhamnish1030

@Shubhamnish1030 Shubhamnish1030 commented Nov 5, 2026 •

Copy link
Copy Markdown

Description

Jira: MDEV-19629

FORMAT_PICO_TIME() and FORMAT_BYTES() from this ticket are already native. This PR adds the remaining MySQL 8.0.16 functions:

Function Returns
PS_THREAD_ID(connection_id) Performance Schema THREAD_ID of that connection, NULL if there is none
PS_CURRENT_THREAD_ID() THREAD_ID of the current connection

Behaviour follows MySQL 8.4, which I checked against a local 8.4.11 server before writing the code:

  • result type BIGINT UNSIGNED
  • NULL for NULL, 0, negative and non-integer arguments and for unknown connection ids ('abc' also raises the usual 1292 truncation warning, same as MySQL)
  • ER_WRONG_PERFSCHEMA_USAGE ("The Performance Schema is not enabled.") when performance_schema=OFF
  • like CONNECTION_ID(), allowed in VIRTUAL but not in PERSISTENT generated columns

Implementation (reworked after review)

  • PS_THREAD_ID() and PS_CURRENT_THREAD_ID() are MariaDB_FUNCTION_PLUGINs declared by the performance schema plugin library: storage/perfschema/pfs_native_func.cc, registered in ha_perfschema.cc. They read PFS_thread directly (PFS_thread::get_current_thread(), and a scan of global_thread_container under the optimistic lock, the same way table_threads reads it).
  • No PSI interface change and nothing in sql/. Without the performance schema (-DPLUGIN_PERFSCHEMA=NO) the functions don't exist. Each one can be switched off with --ps-thread-id=OFF / --ps-current-thread-id=OFF.
  • Both builders set safe_to_cache_query= 0 and BINLOG_STMT_UNSAFE_SYSTEM_FUNCTION.
  • sys.ps_thread_id() keeps its interface (NULL = current connection, NULL when performance_schema is off) but calls the native functions instead of selecting from performance_schema.threads (MariaDB's part of MySQL's WL#13439; the format_bytes()/format_pico_time() part is already in sys). Calling it emits Note 1585, like sys.format_bytes(), so 4 sysschema results gain that note, and fn_ps_thread_id/fn_ps_thread_account disable the ps/cursor/view protocols around the call, like fn_format_bytes.
  • mariadb-test-run.pl does not make the two plugins optional, because sys uses them (same exception as the InnoDB I_S plugins sys uses).

Per-row cost, filtering events_statements_history_long (20,000 rows, Debug build, so only the ratios matter):

WHERE THREAD_ID = ... time
sys.ps_thread_id(NULL), before ~1080 ms
sys.ps_thread_id(NULL), this PR ~500 ms
PS_CURRENT_THREAD_ID() ~19 ms

How can this PR be tested?

New tests:

  • perfschema.native_func_thread_id: the plugins are listed in information_schema.PLUGINS, results match performance_schema.threads for every connection, a second connection before and after disconnect, NULL cases, prepared statements, result type, argument count, generated columns, sys.ps_thread_id() on top of the native functions
  • perfschema.native_func_thread_id_off (--loose-performance-schema=OFF): the error path, and sys.ps_thread_id() still returns NULL

Both source include/have_perfschema.inc.

Run locally (macOS arm64, Debug build of main @ 7b96ca4, head 4b5eb03):

  • perfschema.native_func_thread_id, perfschema.native_func_thread_id_off: pass, also with --ps-protocol, --cursor-protocol and --view-protocol (same for the changed sysschema tests)
  • --suite=perfschema,sysschema,plugins: 555/556 pass; the one failure, perfschema.socket_summary_by_instance_func, passes on its own (socket timing under 6-way parallel load)
  • main.mysqld--help (the two plugin switches are listed with the other optional plugins), main.plugin, main.plugin_load, main.information_schema, main.information_schema_all_engines, main.func_misc, main.sp, main.ps: pass
  • -DPLUGIN_PERFSCHEMA=NO build: compiles and links; the perfschema-dependent tests skip themselves
  • ninja abi_check: passes, PSI headers unchanged

Basing the PR against the correct MariaDB version

  • This is a new feature, so the PR is based against main.

Copyright

I am contributing this under the BSD-new (3-clause) license.

@CLAassistant

CLAassistant commented Nov 5, 2026 •

Copy link
Copy Markdown

CLA assistant check
All committers have signed the CLA.

…at_pico_time() and ps_thread_id()

FORMAT_PICO_TIME() and FORMAT_BYTES() are already native. This adds
the remaining MySQL 8.0.16 functions:

  PS_THREAD_ID(connection_id)  Performance Schema THREAD_ID of the
                               connection, NULL if there is none
  PS_CURRENT_THREAD_ID()       THREAD_ID of the current connection

Behaviour follows MySQL 8.4: BIGINT UNSIGNED, NULL for NULL, zero,
negative and non-integer arguments and for unknown connections, and
ER_WRONG_PERFSCHEMA_USAGE when the Performance Schema is disabled.
Like CONNECTION_ID(), both are allowed in VIRTUAL but not in
PERSISTENT generated columns.

The lookup goes through two new PSI thread calls, get_thread_by_id()
and get_thread_internal_id(), appended to PSI_v1 like
set_thread_peer_port(), so the result always matches
performance_schema.threads, including threads that never call
THD::set_psi().

The sys schema keeps its own sys.ps_thread_id(), where NULL means the
current connection. Calling it now emits Note 1585 on first use, the
same as sys.format_bytes() after FORMAT_BYTES() became native, so the
affected sysschema results are updated.

Signed-off-by: Shubham Chandrawanshi <179670574+Shubhamnish1030@users.noreply.github.com>
@Shubhamnish1030
Shubhamnish1030 force-pushed the mdev-19629-ps-thread-id branch from 488f9be to d7d4074 Compare November 5, 2026 14:53
@gkodinov gkodinov added the External Contribution All PRs from entities outside of MariaDB Foundation, Corporation, Codership agreements. label Nov 6, 2026
@gkodinov gkodinov self-assigned this Nov 6, 2026

@gkodinov gkodinov left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for your contribution. But I fail to see the usefulness of it, given its current implementation.
Can you please reply my questions below?

Comment thread include/mysql/psi/psi.h Outdated
end_metadata_wait_v1_t end_metadata_wait;

set_thread_peer_port_v1_t set_thread_peer_port;
/** @sa get_thread_by_id_v1_t. */

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You can't just add functions without changing the PSI interface version. That's not the intent. The PSI interfaces are supposed to be immutable and then when you need to change the interface you bump the version (v1, v2 etc).

But this brings me to my next question: Why do you insist on having the function call down to perfschema from the SQL layer?

That's not what Chris did in mysql/mysql-server@260e09e#diff-bbb008e05393ffaad9f35966758d0aa1988a9adff23f40489ab187860dc62e7d.

He implemented the functions in the SQL layer: there he has everything needed to do that.

And, finally, what is the point in implementing these functions to start with?

Chris implemented these to speed up the SYS schema joins. Do you need to speed up the sys schema joins? Then why is this not part of the fix, as it is with Chris's change above?

Do you need these functions for something? What is it?

There's no point in just dumping it into MariaDB because MySQL did it. IMHO at least.

If you really want to implement them regardless, why not just do them as pluggable functions? So that at least they are optional and not baked into the core server.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed. In 4b5eb03 the PSI headers are back to what main has. The functions are now function plugins inside the perfschema library and read PFS_thread directly. Details, including the sys schema / WL#13439 part: #6080 (comment)

@vaintroub

vaintroub commented Nov 6, 2026 •

Copy link
Copy Markdown
Member

It failed when compiled without perfschema, on Appveyor. Please also ensure that tests that need perfschema have --source include/have_perfschema.inc in them. To compile without perfschema yourself for testing, pass -DPLUGIN_PERFSCHEMA=NO to cmake

…hema function plugins

Review: no PSI interface change, no SQL layer change, and the server
must build with -DPLUGIN_PERFSCHEMA=NO.

PS_THREAD_ID(connection_id) and PS_CURRENT_THREAD_ID() are now
function plugins declared by the performance schema plugin library
(storage/perfschema/pfs_native_func.cc):

- They read PFS_thread directly, so the PSI interface is unchanged,
  nothing is added to the SQL layer, and the functions do not exist in
  a server built without the performance schema. Each one can be
  switched off with --ps-thread-id=OFF / --ps-current-thread-id=OFF.

- Behaviour follows MySQL 8.4: BIGINT UNSIGNED, NULL for NULL, zero,
  negative and non-integer arguments and for unknown connections, and
  ER_WRONG_PERFSCHEMA_USAGE when the performance schema is disabled.
  Like CONNECTION_ID(), both are allowed in VIRTUAL but not in
  PERSISTENT generated columns.

sys.ps_thread_id() keeps its interface (NULL means the current
connection, and it returns NULL when the performance schema is
disabled), but it now calls the native functions instead of selecting
from performance_schema.threads. That makes it about twice as fast
when it is evaluated per row, and it no longer adds a statement event
of its own for every call. Calling it emits Note 1585, the same as
sys.format_bytes() since FORMAT_BYTES() became native, so the affected
sysschema results are updated.

mtr does not treat the two plugins as optional, because the sys schema
uses them, the same as the InnoDB I_S plugins that sys uses.

Signed-off-by: Shubham Chandrawanshi <179670574+Shubhamnish1030@users.noreply.github.com>
@Shubhamnish1030

Copy link
Copy Markdown
Author

Thanks for the review, @gkodinov and @vaintroub. I reworked the PR in 4b5eb03, pushed on top of the original commit so the change is easy to review. I'll squash it into one commit whenever you prefer.

PSI interface. You're right, appending to PSI_v1 was wrong. The PSI headers, pfs.cc and psi_noop.c are back to what main has. For the record, Chris's Item_func_pfs_thread_id in 260e09ea does call PSI_THREAD_CALL(get_thread_by_id) and get_thread_internal_id, but those calls were already in MySQL's thread interface, which is why that commit touches no PSI files. MariaDB's PSI_v1 doesn't have them. Rather than bump the PSI version for two lookups, I followed your suggestion and made the functions pluggable.

Pluggable functions. PS_THREAD_ID() and PS_CURRENT_THREAD_ID() are now MariaDB_FUNCTION_PLUGINs declared by the performance schema plugin library (storage/perfschema/pfs_native_func.cc, registered in ha_perfschema.cc). They read PFS_thread directly, using the same optimistic lock as table_threads. Nothing is added to sql/, and each one can be turned off with --ps-thread-id=OFF / --ps-current-thread-id=OFF.

Building without perfschema (@vaintroub). With -DPLUGIN_PERFSCHEMA=NO the functions don't exist, so nothing outside storage/perfschema refers to them. I built that configuration locally (Debug, macOS): it builds, and the perfschema-dependent tests skip themselves there. The tests moved to perfschema.native_func_thread_id and perfschema.native_func_thread_id_off, and both source include/have_perfschema.inc. The sysschema suite already requires the perfschema engine.

What it's for, and WL#13439. I read your comment on the Jira and went through 7e4f32bc. In MariaDB the format half of WL#13439 is already done: the sys views and procedures call the native format_bytes() and format_pico_time(). For the thread id, MySQL's change only removed sys.ps_thread_id(), because no sys object joins on it, and that's true in MariaDB too. So the piece left here was sys.ps_thread_id() itself. Instead of removing it (it's documented, and NULL means "current connection" there), it now calls the native functions and still returns NULL when performance_schema is off.

The difference shows when the function is evaluated per row, for example when filtering events_statements_history_long (20,000 rows, Debug build, so only the ratios matter):

WHERE THREAD_ID = ... time
sys.ps_thread_id(NULL), before ~1080 ms
sys.ps_thread_id(NULL), this PR ~500 ms
PS_CURRENT_THREAD_ID() ~19 ms

The old function also runs a SELECT for every row, and with the history consumers on, those statements land in the same history table the query is reading.

Apart from sys, the reasons in the ticket still apply: scripts written for MySQL 8.0.16+ call PS_CURRENT_THREAD_ID() and PS_THREAD_ID() directly.

If you'd rather drop sys.ps_thread_id() the way MySQL did, or keep these plugins out of the perfschema library, I'm happy to change it.

The amd64-windows failure was innodb.avoid_deadlock_with_blocked again, the same lock-timing failure that showed up on the earlier push. It doesn't touch these functions.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

External Contribution All PRs from entities outside of MariaDB Foundation, Corporation, Codership agreements.

Development

Successfully merging this pull request may close these issues.

4 participants