Saving the client IP address once per the lifetime of a backend. This avoid
the expensive operation multiple times, and hence improving performance
significantly.
This bug uncovered serious issues with how the data was being stored by PSGM.
So it require a complete redesign.
pg_stat_monitor now stores the data locally within the backend process's local
memory. The data is only stored when the query completes. This reduces the
number of lock acquisitions that were previously needed during various stages
of the execution. Also, this avoids data loss in case the current bucket
changes during execution. Also, the unavailability of jumble state during later
stages of executions was causing pg_stat_monitor to save non-normalized query.
This was a major problem as well.
pg_stat_monitor specific memory context is implemented. It is used for saving
data locally. The context memory callback helps us clear the locally saved data
so that we do not store it multiple times in the shared hash.
As part of this major rewrite, pgss reference in function and variable names
is changed to pgsm. Memory footprint for the entries is reduced, data types
are corrected where needed, and we've removed unused variables, functions and
macros.
This patch was mutually created by:
Co-authored-by: Hamid Akhtar <hamid.akhtar@percona.com>
Co-authored-by: Muhammad Usama <muhammad.usama@percona.com>
Added buckets for queries that take less than minimum histogram time
and one for the ones taking more than the max value specified.
Also, in case the buckets end up overlapping, on server start, an
error will be thrown informing the user of this issue and requesting
a rectification.
Refactored the code to consolidate the calculations in a single
function.
* PG-543: pg_stat_monitor: PostgreSQL's pg_stat_statements compatible view.
The view now carries all the columns as pg_stat_statements. This required fixing
data types of some of the columns, renaming a few, as well inclusion of new
columns to make the view fully compatible with pg_stat_statements.
* PG-543: pg_stat_monitor: PostgreSQL's pg_stat_statements compatible view.
Updating the upgrade sql file from 1.0 to 2.0 version linked with this issue
changes.
* PG-543: pg_stat_monitor: PostgreSQL's pg_stat_statements compatible view.
Updating datum calls to use UInt64 rather than Int64.
Regardless of the database or the user, the same query will yield the
same query ID. As part of this, a new column, 'pgsm_query_id', is added.
* pgsm_query_id:
pgsm_query_id has the same data type of int8 as the queryid column. If
the incoming SQL command includes any constants, it internally normalizes
the query to remove those constant values with placeholders. Otherwise,
it uses the query directly to generate the query hash.
Since we no longer depend on the server's parse tree mechanism, we can
generate the same hash for the same query text for all server versions.
Also, it is important to note that the hash being calculated is a database,
schema and user independent. So same query text in different databases
will generate the same hash.
This column is not part of the key; rather, for observability purposes only.
* Regression
SQL test case pgsm_query_id.sql is added to the SQL regression.
Removing the view for 2.0. Updating the required SQL files to manage
the upgrade. Downgrade from 2.x to 1.x is not supported.
Also part of this fix is the SQL regression. This does not update the
tap test cases.
In line with pg_stat_statments for PG15, This commit adds eight new cumulative
counters for jit operations, making it easier to diagnose how JIT is used in an
installation. And two new columns, temp_blk_read_time, and temp_blk_write_time,
respectively, show the time spent reading and writing temporary file blocks
on disk.
Moreover, The commit also contains a few indentations and API adjustments.
A new column is added to mention that bucket is active or done. there is
some timing based adjustment was required with that too.
Co-authored-by: Hamid Akhtar <hamid.akhtar@percona.com>
Creating the infrastructure that'll allow using newer versions
of the loadable module with old SQL declarations.
Also updating the build version to 2.0.0-dev
It will be a security problem to provide the internal function access to PUBLIC.
This commit will revoke all permission of internal functions from PUBLIC.
There was a typo while checking the PostgreSQL version in the SQL file. This commit
will fix the typo, and only the necessary columns will be visible in the view.
The query status monitoring code was used to track the current query state, for example,
parsing, executing and finishing. After careful review, we have figured out that
it does not make sense while a lot of time same query is running. Therefore it
is also consuming resources. This commit will remove that feature. The upgrade
SQL from 1.0 - 2.0 is also updated.
pg_stat_monitor is a bit longer; therefore, it requires some code cleanup.
Therefore I decided to turn these tasks into multiple commits and PR to avoid
various changes in one PR. This will ease the review and Q/A process.
In this commit, I have done these tasks.
1 - Delete all the SQL.in files because these version-dependent files
are becoming significant in quantity. Now added a single SQL file for which
contains the dynamic SQL based on the PostgreSQL Version.
2 - New SQL files (pg_stat_monitor--2.0.sql) added for pg_stat_monitor version 2.
3 - A new SQL file (pg_stat_monitor--1.0--2.0.sql) is created, which will be
used to upgrade from version 1.0 to 2.0. Currently, this file is empty. But
whenever we add some API changes into 2.0, we need to update that file too.
4 - The control file (pg_stat_monitor.control) is updated for version 2.0.
This change will make the CREATE EXTENSION default to pg_stat_monitor version 2.0
Hamid Akhtar
Muhammad Usama
Naeem Akhter
Ibrar Ahmed
Muhammad Usama